A Medium humanoid druid that employs the supernatural ability wild shape to assume a Medium alternate form can, for example, consume a potion of enlarge person to gain that spell's benefits—like becoming Large and a +2 size bonus to Strength—while in that alternate form.
If that same (normally Medium but currently Large) humanoid (but having assumed an alternate form) druid then opts to end the wild shape effect while the enlarge person effect's duration continues, the druid resumes his normal original form except that the druid's Large as per the spell enlarge person until the enlarge person effect ends. (N.b. the spell enlarge person is dismissible.)
Were that same druid to then employ wild shape again to assume a Medium form again, he would assume that Medium form, the special ability wild shape setting his new size category overruling the ongoing enlarge person spell effect (q.v. One Effect Makes Another Irrelevant and—more tellingly—the omitted-from-the-SRD examples on PH 172).
So you know, this DM would likely rule that were the Medium humanoid druid to employ initially the supernatural ability wild shape to assume a Large or bigger alternate form, consuming the potion of enlarge person would have no effect. The enlarge person spell's description says, "Multiple magical effects that increase size do not stack" (PH 226), and the supernatural ability wild shape is, indeed, magical, therefore having the same constraints on its interaction with the spell enlarge person as any other magical effect.
However, this player wouldn't argue with a DM that ruled that because the supernatural ability wild shape sets a druid's alternate form to particular size that an enlarge person effect could work normally on that druid. That is, the supernatural ability wild shape doesn't change the druid's size but puts the druid's size at certain point and the enlarge person effect works from that point. This player may point out that this gives the druid an edge when employing size-changing magic—which is already pretty awesome—, but, since druids are involved, I don't think this ruling would even crack the top 10 list of Things I Worry About When Fighting Druids.
Yes, they would still benefit from Expertise in Wild Shape
As stated in the part of the description of the druid's Wild Shape feature you quote:
You also retain all of your skill and saving throw proficiencies, in addition to gaining those of the creature. If the creature has the same proficiency as you and the bonus in its stat block is higher than yours, use the creature’s bonus instead of yours.
In addition, another bullet point in the Wild Shape feature description says:
You retain the benefit of any features from your class, race, or other source and can use them if the new form is physically capable of doing so. However, you can’t use any of your special senses, such as darkvision, unless your new form also has that sense.
So, to see whether your Expertise class feature is retained, we need to look at the description of the rogue's Expertise feature:
At 1st level, choose two of your skill proficiencies, or one of your skill proficiencies and your proficiency with thieves’ tools. Your proficiency bonus is doubled for any ability check you make that uses either of the chosen proficiencies.
Nothing in the feature description specifies a particular anatomy needed to benefit from Expertise (...it'd be weird if it did, honestly). Thus, you do retain the benefit of the Expertise feature.
Putting it all together: You still benefit from the Expertise feature in Wild Shape. You retain all your skill proficiencies in Wild Shape, in addition to those of your new form. And if the creature is also proficient in the same skill, you use whichever total bonus to the skill is higher between yours and the beast's (for a skill you have Expertise in, your own bonus will likely be higher).
Note: Your own modifier for a skill may change depending on the ability score that the skill is associated with. Per the Wild Shape description:
Your game statistics are replaced by the statistics of the beast, but you retain your alignment, personality, and Intelligence, Wisdom, and Charisma scores.
Thus, your own Strength, Dexterity, and Constitution scores are replaced by those of your new form while in Wild Shape. For those skills that are tied to these three scores (or rather two, since Constitution isn't associated with any skills), your modifiers are also changed accordingly.
For instance, if your own Strength modifier is +1 but your new form's Strength modifier is +5, your Athletics skill modifier will increase by 4 - assuming the beast is not also proficient in Athletics with a higher modifier (in which case you would use the beast's Athletics modifier instead of yours).
Correspondingly, your modifier for a skill may actually decrease if the new form is not proficient in the skill and its corresponding ability score is lower. For instance, if you have a Dexterity modifier of +3, and the beast's Dexterity modifier is -2, your modifier for the Stealth skill would actually decrease by 5.
You can determine your new modifiers for the skills associated with Strength and Dexterity by recalculating them based on your new ability score, and then comparing them to the beast's statblock. If the beast is not proficient, you use your own recalculated modifier. If it is proficient, then you compare your recalculated modifier to the modifier given in the statblock; use whichever of the two is higher.
Best Answer
No checks are necessary unless you try to do something decisively non-goat-like.
There is some guidance in the Dungeon Master's Guide section Using Ability Scores:
You have to ask yourself, "does my enemy have any reason to believe that this goat that is physically indistinguishable from a goat is not really a goat? Remember, your physical characteristics are exactly that of a goat, therefore unless you say you are doing something non-goat-like, there is a reasonable expectation that you are moving and acting as a goat would. Think about it this way: there's obviously no check required to see if you know how to use this totally unique musculo-skeletal system you now possess to walk around. The ability to move and use your body as a goat naturally would is part-and-parcel to Wildshape.
Therefore, the answer to the first question in the DMG guidance would be "yes, a goat acting like a goat is so easy and so free of stress there should be no chance of failure." It is when a goat starts acting like not-a-goat that checks need to be made: How well can I make this non-goat-like action appear to be goat-like?
To quote enkryptor from the comments: