[RPG] If you move into melee range and then back out on the same turn, does it provoke an opportunity attack

A creature that, because of the creature's size, provokes attacks of opportunity for entering a foe's square provokes attacks of opportunity when entering that foe's square even if the creature enters that square by taking a 5-ft. step...

The rules cited in the question are from two different places about two different things. The 5-ft. step rules don't care about the creature size rules, and the creature size rules don't care about the 5-ft. step rules. The two rules are managed independently and don't interact. Thus, when a Tiny, Diminutive, or Fine creature enters an opponent's square, it provokes an attack of opportunity even if it entered that opponent's square by taking a 5-ft. step. That's a function of the creature's size exclusive of the creature's 5-ft. step.

Example 1
A Tiny, Diminutive, or Fine creature takes a 5-ft. step when adjacent to opponent and within that opponent's threatened area that doesn't provoke an attack of opportunity (because of the 5 ft. step) and moves into his opponent's square which does provoke attacks of opportunity (because of the creature's size).

When a Tiny, Diminutive, or Fine creature takes a move action to move within its opponent's threatened area then enter its opponent's space, the creature probably should provoke attacks of opportunity twice, once for the movement within the creature's threatened area and once for entering the creature's square, except that the FAQ (see below) says to treat the opportunity granted by moving within a creature's threatened area as part of the same movement needed to enter a creature's space. Thus a typical Medium creature gains only one attack of opportunity against a Tiny foe that takes a move action to enter the creature's square.

Example 2
A Tiny, Diminutive, or Fine creature takes a move action to travel 30 ft. to enter an opponent's square. The creature provokes an attack of opportunity when moving within the opponent's threatened area as normal then can enter the opponent's square with impunity.

It's sort unfortunate that two attacks of opportunity aren't provoked. It might've saved the Dexterity 13 commoner with the feat Combat Reflexes from death by house cat.

...And the FAQ agrees

The Pathfinder Roleplaying Game Core Rulebook FAQ contains the following exchange:

Question: In the section on Tiny and smaller creatures, it says that entering a creature’s space provokes an attack of opportunity, but typically 5-foot steps don’t provoke an attack of opportunity. If a Tiny or smaller creature took a 5-foot step into a creature’s space, would it provoke an attack of opportunity?
Answer: Yes. Even with a 5-foot step, a Tiny or smaller creature entering a creature’s space provokes an attack of opportunity (unless it is using a more specific ability to avoid the attack of opportunity such as the Monkey Shine feat). This doesn’t mean that a Tiny or smaller creature entering a creature’s space and moving out of a threatened square with a move action provokes two attacks of opportunity from that creature, for the same reason that moving out of multiple of a creature’s threatened squares in the same move action doesn’t provoke two attacks of opportunity.

This exchange was added to the FAQ in Jan. 2016.

Yes

I'm not sure why there is any doubt about this:

You can make an opportunity attack when a hostile creature that you can see moves out of your reach.

If the putative rogue has a reach of 5 feet or more, a reaction available and the target is visible then if the target leave's the rogue's reach using their own movement then they rogue can launch an Attack of Opportunity.