Best Bets
According to the official Prima game guide (Engaging Vehicles, p. 16), your best option is the M320 grenade launcher, which you'll unlock very early. And that's not at all an ideal option since you won't be doing nearly as much damage as a rocket, mine, or tank would do. My feeling is that it's like throwing rocks at a rhino. Sure, eventually you'll do damage, but long before you do enough damage it'll figure out where you are. Bad scene. And as for helicopters, good luck getting them to stay still long enough for a grenade launcher takedown.
Note that if a chopper or heavily armored vehicle is very badly damaged (spewing smoke), it might be worth a grenade launcher or even a grenade attack in the hope of shaving off those last few hit points. But don't expect miracles.
Small Arms Fire
Small arms fire won't damage heavily armored vehicles. You can damage some light vehicles with small arms fire, but don't count on fast results.
As a better strategy you can damage people in the least-armored vehicles, buggies and jeeps. If you pull it off, as a bonus you get an intact vehicle to jack.
M320 Example
As an example of probably the best case scenario, grenade launcher vs. lightly armored vehicle: Earlier tonight I disabled a couple of APCs with four grenade launcher rounds each.
Though I can't complain about the result, in one case the APC had no gunner and the driver didn't jump out, drive away, or hop in the gunner's seat. In the other case the gunner... uh... had suboptimal situational awareness. And that's just disabling, not outright destroying; an engineer could have repaired them.
Against slightly more skilled opponents, or on a fully populated server, I doubt I would have have survived either attempt. To pump out four grenades takes a good 10-15 seconds including reload time.
What you can do
So as a rule, leave it to engineers with their rockets and mines, support with their C4, and people in vehicles.
Perhaps the most useful thing you can do as light infantry is to spot enemy vehicles. Just because you're not blowing it up yourself doesn't mean you're "completely useless". Spotting is actually very valuable, and it's disappointing how infrequently most players do that.
Best Answer
The bullets from a tank's secondary position will inflict damage on most vehicles and all infantry.
Damage to aerial vehicles is fair, it's definitely worth shooting them to at least annoy them, and possibly disable them if they stick around long enough.
Damage to enemy armour is non-existant, and you're better off looking for enemy engineers repairing the tank, or scanning around for any sneaky engineers/supports who are looking for an easy kill on a distracted tank.