The simple answer would be 'GM's discretion'.
There has been discussion on the Paizo forums about possible things you could grab, effectively having 'Quick Draw' for none weapons. Some are suggesting light weapons could be pulled with it, but I believe this traits is more designed for very small light objects less than one pound (which would be potions, scrolls, many magic items, coinage, etc) as explained in more detail in the Grasping Tail feat.
Your world, your rules
Ultimately, when it comes down to it, the DM is the one deciding which home-brew rules, if any, the group is going to use. The DM always has final say. So if you feel that you cannot properly handle his home-brew classes, you need to tell him so. Especially if you are new at this. Tell him he can't do it.
Which brings me to my next point
Talk to your players
You can't solve any problem with your players if you don't communicate. Talk to this guy, tell him you're not going to allow him to play home-brewed classes. And tell him why. Let him know that you're new at this and you don't have time to research all of these strange classes. Tell him that you'd rather play vanilla classes so you can get the hang of DMing. Limit him to the PHB, it's got everything he needs. He should be able to find something fun out of the original classes.
Finally, he seems like the type of player that likes to be creative in his builds. This is fine, but you need to let him know where the line is and both of you need to understand that there are going to be some disagreements on some of the rules. The best thing for him to do is to just do as you say. Sessions are always 10 times more fun if people aren't arguing all the time, even if someone has to bite their tongue. The best thing for you to do is remain firm, but fair. Remember, this player isn't doing this to spite you (hopefully), he is just trying to have fun. Remind him that you're a player too, and you're trying to have fun creating and running a world for the players and you can't do that with all these extra rules. I would also let him know that maybe down the road, you might be experienced enough to deal with home-brew classes, but right now you'd prefer things to be vanilla.
Best Answer
Tiefling tails look pretty flexible, but they probably don't have much fine control over them
The books published so far (for 5e, at least) seem pretty thin on description of tieflings in the detail you want. By searching, the most I could find that referenced the tiefling's tail is the default description given in the Player's Handbook (pg 42):
And this part of the Variant Tiefling option in the Sword Coast Adventurer's Guide (page 118):
And there's also the official art. With those depictions in mind, here's my take on the tiefling's tail:
Flexibility
The official art of tieflings from the manuals gives them quite chunky tails indeed - the character illustration for tieflings in the PHB depicts a character whose tail looks to be thicker than her neck at the base and is at least as thick as her arm for most of that 4-5ft length. It is displaying an appreciable curvature, though, which suggests a decent degree of flexibility. The flavour given that a tiefling's tail may lash about or coil around their legs also suggests to us that it has a reasonable range of motion. It would seem reasonable, based on the art and the description, to describe the tail as being about as a flexible as a snake with similar dimensions.
Control
The question of how much control the tiefling has over their tail is a bit trickier. The only example of tail behaviour we're given is based on the tiefling's emotional state; a tail that lashes or coils, sounding a lot like a cat's does, without deliberate control. We can also extrapolate from what is not stated - the tiefling has no special racial trait which explicitly lets them make use of their tail for a functional purpose. That suggests that whatever control they do have over the appendage, it is not fine enough that they can interact with their environment using the tail anywhere near as adeptly as their other limbs.
Based on that, I'd judge that the best real-world animal to use as an example would actually be a cat. Cats' tails are flexible and can lash and coil like the tiefling's is described as doing - and a cat may deliberately flick their tail about sometimes, and tail behaviour is a helpful indicator as to a cat's mood - but they aren't capable of using their tail to pick things up, or manipulate their environment beyond hitting things (relatively gently).
A tiefling is more intelligent than your average cat, so you could imagine they could be a bit more creative in the use of their tail. They could probably use it to pick up reasonably sized objects that are both sturdy enough to withstand a bit of rough handling but not too heavy; potions and scrolls seem too small for the tail to be able to coil around enough to grasp, but I'd judge a spear/polearm or a bucket could be lifted (but not manipulated effectively) if the tiefling concentrates on what they're doing.
Mechanical ruling
Ultimately I would probably rule that tiefling could deliberately flick/lash their tail about, or coil it around something reasonably sized and stationary, using the free object/environment interaction they get on their turn. That means they could do stuff like flick levers, behave inappropriately in bars, and knock your drink off the table with relative ease. I'd also suggest that a tiefling spending their action, and therefore concentrating their effort, on controlling their tail could use it to pick up reasonably sized objects, but could not manipulate them adroitly enough to do anything effective with them beyond carry them (no making attacks with a weapon held in the tail, etc.)