Learn English – Usage of “last evening”

First question: "Last night" is perfectly normal English for "yesterday between 5 p.m. and midnight" (night time). Yesterday night is something only non-native speakers of English would use.

Second question: There should be no problem understanding that "last night" means yesterday night. If your friend had meant that it was her last night in NYC because she was going to move somewhere else the next day, she might have said my last night in New York. OTOH, she might have been using shorthand (headline English), which is not unusual for photo captions. Her communication problem rather than your comprehension problem.

Third question (unasked but latent, nascent, and imminent): "At New York" is not idiomatic American English. It has to be in New York.

Last and next are sometimes used as determiners in deictic time expressions (cf. this):

Last week I went to the store.
This week I went to the store.
Next week I'll go to the store.
I ate pizza last time.
I'll eat pizza this time.
I'll eat pizza next time.

But your noun phrase last argument is not a deictic time expression, so in that phrase last cannot be a determiner. It must instead be an attributive adjective, modifying the following head noun. That means your noun phrase last argument is incomplete—it needs a determiner, and the appropriate one to add is the:

The last argument implies that …

Previous is never a determiner. Like last, it would be attributive in your example, and again that would leave the determiner slot empty. Let's add the:

The previous argument implies that …

Either of these would work. I think last is neutral with respect to formality, while previous is markedly formal. Please see the comments for additional discussion.