Painting with a roller leaves an "orange peel" type texture that you can't match exactly with a brush. When you brushed your new paint on, the orange peel texture in the unsanded areas just telegraphed through your new brushed paint, but the smoothly sanded parts had no such base texture so they look visibly smoother.
To fix this I'd go over your patches again with a small roller and you should get the same sheen and texture as in the original paint.
Yes, you can and should sand latex if you are painting over it. You cannot expect the next coat of paint to grip if it doesn't have a roughed surface to grip to. Painting a glossy or glazed surface is like painting glass - it has no adhesion and will lift right up.
Fresh latex is hard to sand because it hasn't cured yet. You have to wait it out. If it won't sand, it's definitely too soon to overcoat it with anything but itself. (and on that, check the cans for recoat rules). You really need to leave decent curing time between dissimilar products - especially oil over latex.
There is no such thing as paint-primer. Paint companies tell you that to take your money. Their marketing claim is based on painting surfaces which are close to ideal - that is to say, well prepared and primed, with no material variations.
By "material variations" I mean material, color, porosity, chemical differences, anything which might "print through" because the topcoat reacts differently to it. I'm not referring to physical roughness like brush stipple or high spots. The purpose of primer is to bind to the underlying surfaces no matter what they are, seal them, and make them "equal" to the topcoat, so the topcoat applies uniformly and reflects exactly the physical terrain, without visible glitches for any other reason.
Done right, you will not see where wood meets marble, an area was previously painted, or where a cat peed on bare drywall.
You also need to remove surface contaminants which would foul the primer (oil, acids) or prevent adhesion (wax, silicones, linseed oil). In machinery paint, that's the classic 2-cloth wipedown with solvent. Water is a solvent, but too many contaminants are immune to it, so you need to use a chemical solvent or strong soaps, e.g. the classic TSP. You need to get the soap residue off.
Invariably when paint fails, somebody skipped one of these steps.
Why are the oil brush strokes still visible? Is this contamination like acid which is fouling each layer? (remove it). Or is it physical irregularity/bumps (sand it down). A build primer is specifically meant to be sanded. It will leave brush stipple marks far more than a topcoat will, but it's easy to sand.
Best Answer
In the paint business they call it holdout. It is a reference to how paints with a sheen attain a uniform sheen on a particular surface. If a surface has different absorptive properties (e.g. figure-8s of primer) then the gloss will not be uniform.
An additional coat of paint may provide a uniform gloss. Be aware that if the primer you partially applied changed the surface texture, then you will likely still see the figure-8s. A possible fix for that would be to apply another coat but use a thick nap roller (or other texturing strategy) to create a new and more pronounced texture that will hide the difference beneath.
Glossy paints are notorious for showing off imperfections in the substrate. Pro decorators are extremely conscientious with their surface preparations when applying them.