Learn English – “neither … nor ” alternative

No, "neither" must always come first in this construction.

Also it generally comes after the verb in the example you give, not before it:

I like neither thing A nor thing B.

As @Richard points out, putting "neither" before the verb implies that the verb is part of what the "neither" is applying to:

I neither like nor detest thing A.

You already have a negative in not, so a second one isn’t necessary. That means you can say I don’t ride a bike or a horse. For emphasis, you could say I ride neither a bike nor a horse. In practice, it all depends on the context. In speech, for example, a native speaker would be more likely to say something like I don’t ride a bike and I don’t ride a horse either.