Learn English – Preposition “over” vs Adverb “over”

As is well-recognised by linguists, dictionaries are not a good place to start when trying to establish parts of speech. A good reference grammar is. Of the three great grammars of the English language from the last hundred years, the most recent and up-to-date is The Cambridge Grammar of the English Language, Huddleston and Pullum, 2002. It says the following:

Locative there is an intransitive preposition contrasting with here: it has deictic and anaphoric uses ... (p. 1319)

(For a fuller account, consult pages 598-691). Part of the reason given for there and here being prepositions is that, exactly as OP has commented, they are modifiable by straight and right. Furthermore, these words function in the same way as other prepositional phrases: they are able to function as locative complements of the verb BE, and also as spacial and temporal adjuncts.

Many phrase types have the latter function, including adverbs and adverb phrases, however adverbs don't generally function as complements of the verb BE. Furthermore adverbs are not modifiable by either right or straight:

Complements of BE

• The comments were erroneously. * (adverb complement)- wrong
• The comments were locally. * (adverb complement)- wrong
• The comments were out of order.
• The comments were at the bottom.
• The comments were below.
• The comments were here.
• The comments were there.

Straight and right as modifiers.

• He flew straight beautifully. * (wrong)
• He flew right directly. * (wrong)
• He flew straight to Paris.
• He flew right over the house.
• He flew straight in.
• He flew straight there.

This all goes to show that the OP is indeed correct: there does indeed seem to be a preposition.

Hope this is helpful

The to in the full infinitive is considered a grammatical particle, i.e., a word that does not serve a part-of-speech function by itself, but only in combination with another word in the sentence.