Can you put a matte polyurethane over a semi-gloss polyurethane to make it a matte finish

I would keep them the same if it's not too inconvenient.

Different manufacturers (and even different batches from the same manufacturer) may have different drying properties (the Satin may come out differently). They may also have trouble adhering to one another which would lead to cracking between layers and may also yellow differently, producing a blotchy effect later on especially in areas that didn't cover well.

Your safest bet would have been to buy as much poly as you needed and mix them all together before you started in a seperate container. If you expect the bookshelf to last more than five years then go get the same stuff. Otherwise, it probably doesn't matter.

What type of paint/glaze? Oil or water based poly? Check with manufacturer of the poly to make sure it's compatible with previous top coats. The ones I've called in the past have always been very helpful.

How long did you let paint/glaze cure before poly? Even though paint dries within hours it takes a while to fully cure. If you apply the poly before it cures you'll have problems with the poly curing. Should wait at least 7 days before applying poly. More depending on climate.