This is simply a question of style. Wikipedia has a huge article on the subject. The passages most relevant to your question are:
Quotations and speech
Single or double quotation marks denote either speech or a quotation. Neither style—single or double—is an absolute rule, though double quotation marks are preferred in the United States, and both single and double quotation marks are used in the United Kingdom. A publisher’s or author’s style may take precedence over national general preferences. The important rule is that the style of opening and closing quotation marks must be matched[.]
[...]
Use–mention distinction
Either quotation marks or italic type can emphasize that an instance of a word refers to the word itself rather than its associated concept. [...] A three-way distinction is occasionally made between normal use of a word (no quotation marks), referring to the concept behind the word (single quotation marks), and the word itself (double quotation marks). [...] In common usage, there may be a distinction between the single and double quotation marks in this context; often, single quotation marks are used to embrace single characters, while double quotation marks enclose whole words or phrases[.]
Emphasis mine. Read the entire article for further insight.
Thanks to the OP for amending his question to
how do I know when to use them? Or,
when I saw them, how do I know what
they stand for?
Unfortunately, as with so much in English (or any language really), a lot of it is just observation and rote practice.
It might help to remember that your first two examples are (somewhat) representative of elision in slurred colloquial speech. Jammin', or anything ending in -in', is pretty clearly an elision of -ing. You could use this for any verb and be well understood.
O' is a little less clear. It seems to have stronger historical roots (e.g., the luck o' the Irish) but in contemporary usage it represents of slurred into /ə/ (schwa). If I actually pronounce it /ow/, it's usually for comic effect such as the Irish example or in various cases of hipster irony.
'Cause is just a fairly standard colloquial form; I can't think of any pattern that applies to anything similar.
Can I create omissions arbitrarily?
As long as they're intelligible, sure.
But your next question is going to be how to tell if they're intelligible. Then we're back to where we started: Make a note of helpful patterns, listen for how native speakers elide certain sounds in speech, and experiment just to see if anyone understands you. :)
Best Answer
A backtick would be my last recommendation. A straight single-quote is acceptable, and a curly close quote can be substituted as an improvement. In other words, of these three:
I would avoid the first, accept the second, and consider the third to be superior to the other two.
One website says the backtick:
and goes on to say that the closing single quote is:
It also mentions that the ASCII apostrophe: