These sentences are a little confusing, because they seem to be very technical. A set containing elements is usually only used when describing things mathematically - a set of prime numbers, for instance. Technical fields will often have specific grammar expectations that may be unique, so it is important to provide context.
In terms of English language usage, "of" means the things that make up the set, while "with" means things that are contained within the set. "Of" is the entire set, while "with" may be part of the set.
For instance, let's say that I have a basket containing puppies.
If I describe it as "a basket of puppies", you could infer that the only thing in the basket is puppies. The basket might contain other objects like a blanket or a chew toy, but as far as the speaker is concerned the only significant objects in the basket are the puppies.
If I describe it as "a basket with puppies", there is a possibility that the basket contains other important things that are being overlooked. For instance, there could also be a sleeping cat.
If I hand my friend a basket with puppies (and also a cat), and she has an allergic reaction to the cat, I can claim to have been truthful. If I tell her it is a basket of puppies and don't mention the cat, she can claim I misled her.
First off, I wouldn't lose any sleep over it. I imagine you could swap the two in almost any context and I doubt anyone (regardless of origin) would notice.
That said, for all meanings not concerning computers, "disc" is probably more common in British English, and "disk" in American English (that's the OED's take, and matches my experience), but use whichever you prefer. You might think disc would be older, since the word derives from the Latin discus, but there are some pretty old "disk" quotes listed in the OED (one from the Philosophical Transactions of the Royal Society of London, 1665--so no American influence there).
For computers, I have to concede that the Apple page does have a point (though calling it a "distinct difference" is carrying it way too far towards prescription on what's clearly a descriptive issue--for that matter, so is devoting a support page to it). One pretty much always sees "compact disc". "Floppy disks" (for those that remember them) were sometimes called "diskettes"; which term came first, I don't know, but the association between the two probably caused the K spelling to stick. Personally, I would also spell "hard disk" with a K as well--since the term originated by analogy to "floppy disk", that makes sense. I don't think the optical/magnetic split between disc/disk is based on anything like logic, but it does seem to be the way things have developed (mostly: I'd still call a CD drive a disk drive). The OED even acknowledges this: "Some writers have used the spelling disk for magnetic ones (‘hard disk’, ‘floppy disk’), and disc for optical ones (‘compact disc’, ‘laser disc’).". But then, it also lists "compact disc (also disk)".
There might be some technical applications where one is used over the other, but it'll never be based on any logic, since the two words mean exactly the same thing, so if you run into that, just look around to see which is more common. ("Disc" brakes gets the wikipedia entry, but no one would wonder for a second what "disk brakes" were.)
As to why both exist: who knows? Both spellings have been around a long time, so it is surprising neither has died out (since there never seems to have been a difference in meaning between them, at least until Apple got involved). If I were to engage in rank speculation, I would wonder if "disc" didn't come into the language directly from Latin, while "disk" meandered in via the French disque.
Best Answer
Cautious means (roughly) the same as "careful"
The prepositions have some of their usual meanings:
You should be careful when using trump cards in a card game (since they are important). The word "with" implies the use of something
You are also careful of things that could be dangerous.
I don't use "with snakes" as I am not using them.
You can use "cautious in a location". For example "He is cautious in the forest." But also there is the "in doing" pattern, with a gerund:
Meaning you should be careful and not sign something that you haven't read. "Cautious about signing a document" also works here.