Learn English – word for numbers and letters, but not punctuation, etc

• quaternary (adjective): fourth in order or rank; belonging to the fourth order.
• quinary (adjective): of or relating to the number five
• senary (adjective, rare): relating to or based on the number six
• septenary (adjective)
• octonary (adjective, rare)
• nonary (adjective, rare)
• denary (adjective)

(source: New Oxford American Dictionary)

I seem to remember the old askoxford.com site said either was acceptable: CDs and CD's.

But now the replacement Lexico powered by Oxford firmly suggests to avoid the apostrophe except in a few special cases:

Apostrophes and plural forms

The general rule is that you should not use an apostrophe to form the plurals of nouns, abbreviations, or dates made up of numbers: just add -s (or -es, if the noun in question forms its plural with - es). For example:

...
MP MPs (e.g. Local MPs are divided on this issue.)

1990 1990s (e.g. The situation was different in the 1990s.)

It's very important to remember this grammatical rule.

There are one or two cases in which it is acceptable to use an apostrophe to form a plural, purely for the sake of clarity:

• you can use an apostrophe to show the plurals of single letters:
  • I've dotted the i's and crossed the t's.
  • Find all the p's in appear.
• you can use an apostrophe to show the plurals of single numbers:
  • Find all the number 7’s.

These are the only cases in which it is generally considered acceptable to use an apostrophe to form plurals: remember that an apostrophe should never be used to form the plural of ordinary nouns, names, abbreviations, or numerical dates.