I've decided to homebrew some more Instruments of the Bards, representing a few instruments that are not currently included. The only main thing that's different between each one are the unique spells, which influences the rarity of each individual instrument.
Here's my attempt at creating a couple of new instruments (names taken from the list of "Elf, Child" names from XGtE, in case anyone's interested):
\begin{array}{}
\text{Instrument} & \text{Rarity} & \text{Spells} \\
\hline
\text{Koeth Flute} & \text{Uncommon} & \text{Feather fall, gust of wind, protection from energy (cold only)} \\
\text{Naeris Viol} & \text{Rare} & \text{Enemies abound, heroism, pyrotechnics, thunder step}
\end{array}
The main issue I'm concerned about with regards to balance (i.e. does an Instrument of the Bards with the above spell, plus what all Instruments of the Bards have, match up with the rarities I've given them) is how to determine the "strength" of those spells for the purposes of determining the item's rarity. I'm kind of flying blind with regards to what "guidelines" I should follow, since the spells chosen for the existing instruments seem completely arbitrary to me.
I've tried to avoid big damage dealing spells like ice knife or fireball, since those seem overpowered compared to most of the other choices (mostly looking at the less rare end of the spectrum, since the legendary instrument has fire storm), although one of the instruments has wall of fire and is only considered rare, so that "guideline" I've come up with might be a bit of a red herring.
So my question is two fold; firstly, are these balanced (meaning, should they be the rarities I've listed them as), and if not, then secondly, what guidelines should I follow for "draft #2" with regards to what spells are appropriate for an uncommon/rare Instrument of the Bards?
Best Answer
By comparison to the pre-existing Instruments of the Bards, the first is balanced, the second may be slightly too strong but is probably alright
Let's compare your proposed spell lists to the pre-existing sets from the Instruments of the Bards of equal rarity:
Mac-Fuirmidh Cittern (Uncommon):
Fochlucan Bandore (Uncommon):
Doss Lute (Uncommon):
Koeth Flute (Uncommon):
None of the pre-existing instruments have any damaging spells at all, which is good for your proposed instrument as it also has none.
We can assume that the two protection from energy spells are roughly equivalent, so let's look at feather fall and gust of wind.
The first is a situational spell, in fact, it can only be cast as a reaction anyways. I would say this spell is situationally useful similar to fog cloud, faerie fire, and protection from poison, and isn't particularly strong compared to completely ending a current poison.
The second is a somewhat useful spell as it allows you to push enemies and cost them some extra movement, but when compared to entangle which has a much stronger effect (restraining), it seems balanced. I would say that entangle and animal friendship are both good "shutdown" spells, as they deter or limit their target much like gust of wind.
Conclusion:
The proposed instrument, aligns well with the pre-existing instruments and follows the slight pattern there as well. If anything, I would say it is weaker than the others, though not by much as the usefulness of the situational spells is very campaign-specific.
Cli Lyre (Rare):
Canaith Mandolin (Rare):
Naeris Viol (Rare):
We can immediately see that your instruments provides more spells than the others, though this is not exceptional as the Fochlucan Bandore mentioned earlier also did this.
Healing:
I feel a good comparison to make is between cure wounds and heroism. The first heals for 3d8+MOD while the second gives MOD temporary hit points each turn that concentration is maintained. Assuming a modifier of +4 it takes 5 turns (not too long for a combat) for heroism to become better, and with a modifier of +5 it takes only 4 turns. Thus, I feel alright saying this spells are about equal, if not having heroism being worse as it takes longer, is not guaranteed due to concentration, and cannot restore a creature to consciousness.
Damage:
Your instrument then has a good damaging spell (thunderstep), though it does an average of 16.5 damage compared to wall of fire's 22.5 damage. Though fire damage is often resisted, wall of fire will often hit more enemies as you can define its shape better, and it does not require enemies to be near the caster. While thunder step also allows you to teleport, and get out of a tight spot, wall of fire persists (barring concentration) and can deal damage multiple times, detering (shutting down) enemies as well. This puts them about on equal footing, if not with thunder step being worse.
Shutdown:
Your instrument then gives two "shutdown" spells: enemies abound, and pyrotechnics. These can either turn an enemy against its allies, or blind an entire group. Though these spells are not particularly strong, and I often see a caster picking up dispel magic as it is the spell to shutdown persisting spells, which are often quite strong themselves. The choice is whether you want to shutdown an enemy or shutdown a spell. These seem about equal to me, though dispel magic has some more use outside of combat, the two spells provide versatility simply by having more options.
A problem:
This would have us comparing thunder step with protection from energy, where thunder step is quite often the superior choice. This points out what I feel the Viol does too well; it covers too many bases. Your instrument provides healing, shutdown, and damage (with a bonus escape tied in). Though it isn't particularly great at any of the three things, it does provide a lot of versatility for the user.
The Lyre provides shutdown (the walls) and damage (wall of fire) but not any sort of healing, compared to the Mandolin which provides healing (cure wounds), and shutdown (dispel magic). Your instrument provides all three.
Other Benefits
Note that both of the other instrument also provide a situational benefit (stone shape and protection from energy), while yours, at best, provides heavy obscurement (pyrotechnics).
Conclusion
Your instrument doesn't excel in any of these categories, so it is likely balanced, in fact, it is outdone in each category by one of the other instruments. What may make it slightly overpowered is simply the amount of versatility it provides with its spells.