Two things are required to bully a city state: high military power relative to the other main civs, and nearby military power. This begs two questions:
How high do you need to be in military power? top 50 percent of civs? within 25 percent of leading civ? I assume its based on the soldiers demographic.
How close does a unit have to be in order to influence a civ toward giving tribute?
Best Answer
The diplomatic effects of military influence work very much the same way on city-states as they do with other civilizations. You don't necessarily need to be in any "top percentage" in order to bully a city-state, they simply have to feel threatened by you. Just as opposing civs will feel threatened by a warmongering player, city-states will likely give in to your demands easier when they know you have a large army AND you don't hesitate to use it.
Also like other civs, city-states will notice if you start building up units close to their borders. I couldn't tell you an exact distance, but I would assume that as long as they have visibility enough to see the tiles where your units are, it'll increase the likelihood of them giving into your demands.
There really is no understanding the AI in Civilization, this is just what I have picked up throughout my countless lost hours of playing. Hope it helps =)