Learn English – What’s the origin of the idiom “to be left holding the bag”

Back in the olden days, back where there were houses made of sod on the American prairie that people actually lived in, about the time that Laura Ingalls Wilder was still Laura Ingalls, ladies used to wear dresses, men wore trousers (or pants). Also, ladies were treated as second class people. Ladies could not own property of their own. Ladies could not vote. Ladies could not enter into contracts. If you wanted to get someone in a household that had the ability to enter into a binding agreement, you needed a man. The only way you could tell a man from a woman in those days was to check to see what they were wearing. If the person was wearing pants, it was a man, and you could do serious business with him. If the person was wearing a dress, you did not bother with them.

Thus, the person in a household who makes decisions and is otherwise in charge "wears the pants".

According to phrases.org, this idiom was well known in the late 19th century. The first known appearance in print was in an November 1880 article in The Manitoba Daily Free Press where the phrase "wear the trousers" was used.

The use of south as in the phrase go south stems from the 1920s (from the Oxford English Dictionary):

colloq. (orig. Stock Market). Downward or lower in value, price, or quality; in or into a worse condition or position. Esp. in to head (also go) south.

1920 Elgin (Illinois) Dairy Rep. 13 Nov., Meat, grains and provisions generally, are like Douglas Fairbanks, headed south—in other words, going down.

The reason South is correlated with down is because of its use in the standard Western set of cardinal directions:

According to the Oxford English Dictionary, the meaning of telling someone to "go west" is not related to the phrase "going south". Instead, it means:

go west, young man: used as an encouragement to seek fortune in the American West; also in extended use. Attributed to Horace Greeley, who, according to Josiah Bushnell Grinnell, gave the latter this advice in September 1851 (see quot. 1891)

The phrase go west on its own also refers figuratively to death (as the sun sets in the West).

There is no similar idiom "go east", unless one is actually telling someone to go in the cardinal direction of east.

The direction "north" has a figurative meaning as well:

Higher; esp. in north of (a figure, cost, etc.): higher than, in excess of.

Though north and south have related meanings (higher and lower, respectively), they are used in different ways. North is usually used in respect to a give figure or amount. For example,

2001 San Francisco Business Times (Nexis) 9 Nov. 21 What's your average deal size? It's gone north of $250,000 per contract even as high as $300,000 per contract.

However, south is usually used as a general figure of speech. For example,

2003 R. B. Parker Stone Cold (2004) xl. 154 But your marriage went south and you had a drinking problem.

That being said, the two can be used both to refer to figure amounts, although in my experience this usage is rarer:

1986 Financial Times (Nexis) 5 July i. 6 With oil heading south of $10‥the London stock market today stands less than 4 per cent below its highest ever level.