Think of how B perceives it:
B: ... think think think think think think think—Sorry, what did you say?
A: [blah blah]
B looks back and thinks of A's utterance as an event which is completely done, finished. Linguists call this view of an event the perfective aspect. Perfective aspect in the past is expressed using the past form of a verb, You said or You did say.
That event, A’s utterance, occurred at a time when B was in a state of thought which started before A's utterance and ended sometime after it. Linguists call this view of a state imperfective aspect. Imperfective aspect is expressed using a progressive construction; in this case, since the state lies in the past, B uses the past progressive construction I was thinking.
Perfect constructions like I have thought and I have been thinking express a state which arises from a previous event and is current at the point in time which you are talking about. Linguists call that time you are talking about reference time (RT). A present perfect construction has the present as its RT: it expresses a state which is current now, at the time of speaking. Since B is talking about a past event, his† RT is the past; in that context a present perfect cannot be used.
A past perfect construction expresses a state with a past RT, a state which was current in the past, so you might think that I had thought or I had been thinking would be appropriate here. But the perfect construction does not express a state denoted by the lexical verb (think) in the construction, it expresses a state which arises from the state or event denoted by the lexical verb. B is not talking about the result of his thinking, he is talking about the thinking itself—so a past perfect construction doesn’t work either.
There is more about aspect here, and entirely too much about perfect constructions here. Be careful not to confuse perfective aspect with perfect constructions—they are entirely different things.
† I make the appallingly sexist assumption that A is B’s wife, only because that’s how this conversation always plays out in my own household.
The first answer is correct because the first speaker, let's call him Bob, wants to know something about the second speaker's life experience, whom we shall call Alice. For example,
Bob: Have you ever lived in Madrid?
Alice: No, never!
Bob wants to know whether Alice at some point in her life lived in Madrid. Maybe she has Spanish relatives, maybe she is a bohemian and has lived in several different cities in her lifetime, whatever the pretext, Bob wants to know about Madrid. At this stage, he is not interested in knowing "when" she lived there, only if she did. He must, therefore, be fairly certain that Alice is not currently living in Madrid.
Now, Bob asks Alice a second question, using the PRESENT PERFECT again.
Bob: Have you ever worked in a factory?
Alice: No, never!
The structure of the question is the same, Bob is asking about a specific situation that Alice may or may not have experienced at any time up to ‘now’. Alice replies that she has never worked in a factory, it is an experience which she didn't have in the PAST. Had she replied differently
Alice: Yes, I have! (= I have worked in a factory)
This would mean that Alice, at a specific point in her life, worked in a factory. But Bob wouldn't know ‘when’, so he might inquire further with any one of the following questions, using the SIMPLE PAST:
Bob: When was that?
– How long did you work there?
– Where did you work?
– What was the name of the factory? etc.
On the other hand, if Alice is currently working in a factory, she might reply:
– Actually yes, I do work in a factory
– Actually, I am a factory worker.
– Didn't you already know? I work in a factory.
– Funny you ask me that, I'm actually working in a factory at this moment.
In which case, Bob could dig deeper, using the PRESENT PERFECT CONTINOUS construction, and ask Alice
How long have you been working there?
The PPC is often used in questions asking about the duration of an action or situation that is perceived to be temporary in nature, it is preferred when the speaker wants to emphasize the continuation of an activity, but the Present Perfect can also be used if the speaker interprets the situation or act as being more permanent.
Bob: “How long have you worked there?”
Alice: ”Ever since I left school.”
– “Since I was 16.”
– “For about twenty years.”
Best Answer
As a native US English speaker, I sense no difference in meaning between the two sentences if you were to speak them. However, if you were writing, you should probably have used the second sentence to be grammatically correct.