Use this aggregate for bedding for the patio

If your brick is level, and moves water away from your house, I wouldn't replace any of it. It takes a lot of time and expertise to get it to set just right, and probably isn't worth the trouble.

If you are really bent on making it look uniform, take a paint scraper, something simple you can get from your local hardware store like this. The pointed end is great for clearing debris between bricks and the like. Use the scraper to clean out as much of the dirt and junk from between the bricks. You can then use a standard leveling sand (which you can also get from your local hardware store), and sweep it into the cracks.

Normally, the purpose of the sand is to fill in the cracks between the bricks to keep them from moving. However, if you get a sand which is lighter in color, then it would probably match well.

It's hard to tell from the picture but it may be -1/4" crushed gravel that was used as bedding. We used that quite often in place of a mortar type sand because it didn't compress as much but still allowed easy grading and workability.