One problem is that the entire concept of "part of speech" is very old. How we use it in English, especially in dictionaries, goes back to the study of Latin and Greek. In this view of English grammar "adverb" is the catch-all category where everything that doesn't fit into one of the other traditional categories ends up. (The others being noun, verb, adjective, pronoun, preposition, conjunction, and interjection.)
Now there is no one, true description of any language (except perhaps constructed languages such as yours). There are merely alternative or competing descriptions which appear over time as more independent analyses of the language are undertaken. Such descriptions or analyses may be called "grammars".
Most (but not all) grammars include a concept of word class under one name or another. So one problem is that "part of speech" has two meanings. One is the specific set of eight categories from the classical languages, the other is as a synonym for word class, which is a lot looser.
So all your example words are adverbs under this older stricter view of parts-of-speech, but their qualities and quirks can be much more thoroughly investigated in newer ways. And various new ways will have various new terms for the classes they put these various words into.
Unless you are inventing a new language specifically to embrace the classical parts of speech you don't have to worry in which they belong, but if you are inventing a new language to learn more about how language works then it will be worth your time reading up on the many newer grammars and language descriptions and analyses.
The correct sentence would be:
Didn't find what you were looking for?
Why? Because you did not find something. That happens when you were looking for something, in the past. So, when you did not find something, you can't use a construction which indicates you are still looking (in present tense), ie: you are looking.
To address your other question:
To me "you are" or "you're" is the best fit, because the idea is
you've scrolled through the page, and not found what you're after, so
you're still looking.
True, that you have scrolled through the page and did not find what you were looking for, so you're still searching. To address this, you need to use the present form of "didn't" ie: "can't", and the correct question would be:
Can't find what you are/you're looking for?
OR
Still can't find what you're looking for?
Best Answer
Just between you and me acts as a disjunct, that is, it affects the interpretation of the whole sentence. That may be easier to see if you replace it with an adverb like confidentially.