Modifying your weapon is a bit of a convoluted process.
First, you need both a modifiable weapon, and a weapon mod. For instance, you can only stick an SMG mod on an SMG - no SMG means you can't use the mod at all.
Second, you need to find a weapon bench, or go to the armory on the Normandy and use that.
From here, simply click the appropriate weapon class, and then the red "Modify" button.
Next, you want to select one of the two mod slots. Note the counter here. If the number is 0/0, that means you have no available weapon mods (at least for this weapon). Pressing the left/right arrow will let you browse through available mods, and the up/down arrow will move between the two mod slots.
Simply manipulate these two arrows until you've got what you want, and then press "Confirm" on the lower left.
Ta da! Brand swanky new gun!
A lot of this is dependent on class. As a Vanguard for example, you'll probably never want to use an Assault Rifle - they're just too heavy, and as a Soldier, you'll usually give SMG's a pass - everything they can do an AR does better. Meanwhile, an Infiltrator gets lots of bonuses when using their trusty Sniper Rifle, evening if irresponsibly shooting it from the hip.
The second consideration is range: at very long ranges, a Sniper Rifle offers top notch accuracy. AR's and Pistols Can be pretty good. SMG's can be flaky though, and shotguns are nigh useless. By contrast, Shotgubs ate devastating at short range, but the loss of peripheral vision while looking down that sniper scope can get you killed.
The next consideration is the defenses of your opponents. Slow firing, high caliber weapons like Sniper Rifles and Pistols chew through Armor, but are weak against Biotic Barriers and Shields. Rapid fire weapons with a lot of projectiles, by contrast, like SMG's and Shotguns chew up Shields and Barriers quickly, but have trouble with armor. Assault Rufles are a bit of an odd duck as they're 'pretty good' at handling everything.
Additionally there are mods availiable for several weapon types that alter their capabilities. You can add Armor-penetration to an SMG or a shotgun, or a melee upgrade to a pistol or shotgun. Maybe a scope on your SMG to improve its mid-long range performance?
Finally, you may want to pair some weapons up with certain specialty ammo. For instance, SMG's are devestating in combination with Cryo or Incendiary Ammo as their high rate of fire gives them more opportunities to proc the special effects provided by these ammo types than, say, a slow firing shotgun.
Best Answer
They work for all non-DLC weapons at the least. I don't know about DLC/special weapons like the Collector SMG however. They may be special, but any normal gun purchased/unlocked in the main story is compatible with all weapon mods of that class.