Drywall – How to fix a floating wall shelf on an uneven wall

I'd agree with the masonry bit (as well, stucco is rough on drill bits anyway) of whatever size is recommended for the wall anchors you're using.

I'd been reluctant to post a response, as I had no idea what do do for the wall without flattening down that section, but I have a completely untested idea --

If the stucco isn't too extreme in its texture, and you're not going to be putting lots of load on the shelf, you might be able to cut a piece of foam to use as backing, and then tighten it to the wall to compress the foam so it's not too bouncy. You're likely going to want extra (or heavier) bolts for this; there won't be friction against the wall, so the bolts are going to be taking the whole load in sheer. Also, you won't have the same support against moment.

I was thinking some of the stiffer spray-foam might work, but if you wanted to make it removable without damage (and if you didn't, you'd just flatten the wall), you'd need to add some release agent, which might stain the wall, or go through some hoops to pull this off:

1. Make a frame the size of the bracket, open on the side against the wall, and an extra hole (see below).
2. Figure out where you want to mount the bracket.
3. Tape a plastic bag large enough to full the frame on the end of a can of spray foam.
4. Stuff the bag in the frame through the hole.
5. Hold the frame where you want to mount the bracket, and inject the spray foam.
6. Wait for it to set up.
7. Repeat steps 2 through 6 for each additional bracket location.

You might be able to do something similar with other products to mold to the wall, but all of the ones I can think of have slower set times which would make it really slow going.

You may not have a choice if the shelf locks in by sliding forward. If that's the case, you mount to the shelf first and then hang it on the wall. Otherwise, the wall blocks you from installing the shelf. And even if it's not necessary, you may want to do it that way to get the shelf tight to the wall. Mount the brackets 1/8" or less from the back of the shelf, so the shelf touches the wall just before they do. Then, after hanging, when you tighten the screws on the wall, the shelf will get pulled tight without any gaps.