Learn English – the origin of the term “Couch Potato”

Back in the olden days, back where there were houses made of sod on the American prairie that people actually lived in, about the time that Laura Ingalls Wilder was still Laura Ingalls, ladies used to wear dresses, men wore trousers (or pants). Also, ladies were treated as second class people. Ladies could not own property of their own. Ladies could not vote. Ladies could not enter into contracts. If you wanted to get someone in a household that had the ability to enter into a binding agreement, you needed a man. The only way you could tell a man from a woman in those days was to check to see what they were wearing. If the person was wearing pants, it was a man, and you could do serious business with him. If the person was wearing a dress, you did not bother with them.

Thus, the person in a household who makes decisions and is otherwise in charge "wears the pants".

According to phrases.org, this idiom was well known in the late 19th century. The first known appearance in print was in an November 1880 article in The Manitoba Daily Free Press where the phrase "wear the trousers" was used.

Hot potato -- Dictionary

(informal) a situation or issue that is difficult, unpleasant, or risky to deal with.

From the same page, you'll find the origins explained (emphasis and links mine):

This term, dating from the mid-1800s, alludes to the only slightly older expression drop like a hot potato, meaning “to abandon something or someone quickly” (lest one be burned). The idiom alludes to the fact that cooked potatoes retain considerable heat because they contain a lot of water.