No, it doesn't
Firstly it's against the spirit of the rule, which is to allow creatures to become invisible with all their gear, so they don't give themselves away by a floating sword.
Secondly, it's against the letter of the rule.
- anything on the target's person - people cannot be 'on someone's person', and horses are not persons, except for Mr Ed.
A simple test for borderline cases: Is the item under your control?
I think this test is a little easier to adjudicate and is a little tighter than the "possession" and "50% support" tests. Simply enough, does the character have the ability to drop, throw, or manipulate the item at will? (When applying this test, we ignore any conditions that the person is under, such as Restrained or Petrified.)
In the case of the man holding the chain constraining the other man, the answer would be, Only the links he controls turn invisible.
Suppose that it is the man in the chains that is Invisible. He is restrained from actually manipulating the chains, but the chains (or the portion on his body) should be invisible - if he wasn't restrained they would be under his control just as his clothes are. But if he was in a set of stocks, the stocks would not be invisible, as he would not control the stocks even if he wasn't stuck in them.
Suppose the fellow is chained up, invisible, and Petrified. The chains would be invisible, since he would be in control if only the conditions didn't apply. Suppose another person removed the chains from him. The moment the second person has control over the chains, they become visible.
It's a simple, intuitive test. It is consistent with how many other spells work. It's hard to abuse the rule. And I believe it matches our general expectations.
Best Answer
No touching or stealing does not end the spell, as there is no attack roll involved (thus no attack), it is not a grapple or shove, which are special attacks requiring the Attack action, and you have not cast a spell.
The conditions for breaking invisibility are succinct. 5e attempts to have spells do exactly what they say they do, no more, no less, and with no hidden presumptions. You can tap someone on the shoulder or pick their pocket without fear of losing your invisibility.
This is supported in many posts of the lead designer, Jeremy Crawford and succinctly stated here:
and here: