Phonemic /l/ that occurs in words like laugh and full is indeed two different sounds in many speakers, but these are just different allophones of the same underlying phoneme.
In fact, phonemic /l/ can be realized as any of [l], [ɫ], [ɤ], [w], [o], or [ʊ] — see here.
For most speakers of English, the allophone in laugh is phonetic [l], whereas the one in full is [ɫ]. Some speakers do have [ɫ] for both, however.
This [ɫ] sound is sometimes called a dark l. Both the bright l and the dark one are alveolar lateral approximants, but the dark version is also velarized.
There are several varieties of English which use a [d] to represent the voiced th sound and a [t] to represent the voiceless one, as well as several others which use [v] and [f] respectively instead.
However, I think the Original Poster is asking about variation in the pronunciation of [ð] in accents such as British RP, or General American, where a canonical /ð/ is thought of as being a voiced dental fricative.
If you ask speakers of such types of English what sound they use at the beginning of the word the, they would make a clearly articulated [ð]. This sound has two parts to it. The first is voicing caused by the vibration of the vocal chords. This is what gives the sound its pitch. The second aspect is the audible friction caused by turbulence in the air as it is forced over the surface of the tongue and out through the stricture formed by the tongue resting against the back of the upper incisors.
The description of this sound is a voiced dental fricative. The term voiced reflects the vocal fold vibration, the term dental reflects the place of articulation and the involvement of the front teeth and the term fricative indicates the type of consonant and the fact that it uses turbulence caused by air being forced through a narrow aperture. This involves an increase in the air pressure in the mouth behind the stricture.
This, however, is just how we think about this sound in an idealised kind of way. In real life the realisation of this phoneme is very often not actually a fricative at all. It is actually very often realised as an approximant. What this means is that the tongue still makes contact with the teeth but there is no friction involved. The air pressure in the mouth does not increase and there is no audible friction caused by air being forced out of a narrow stricture at high pressure.
The reason that the Original Poster might perceive the /ð/ in the string as they went as a [d] is because there's very likely to be a complete absence of frication - which will make the sound quite different to our idealised canonical version of /ð/.
It is probably also worth mentioning that this phoneme is subject to many types of assimilatory processes as well as, in the case of consonant clusters, complete elision (in other words not being pronounced at all).
So, for example in the sequence in the, the th sound will often be realised as a voiced dental nasal, in other words as an [n] made on the back of the front teeth instead of on the alveolar ridge. So the th sound there will have adopted the manner of the consonant from the preceding /n/. This type of assimilation happens in many other sequences. So in fast speech the th sound in is there any ... will often transorm into a /z/.
The Original Poster isn't deaf! Far from it, they have razor-sharp ears.
Best Answer
The question is not should it always be pronounced completely but is it always. The answer to the latter is no.
Many native speakers of English have trouble producing that sound reliably. Variations of this symptom range from a "w" sound ("Don't be siwwy!") to a kind of swallowed consonant which is produced without the tongue ever touching the palate. Not only is this unremarkable, but one of America's most famous network television news anchors, Tom Brokaw, made it part of his onscreen personality. Listen to how he says "British Co[l]umbia" in this YouTube clip.
If you get close to the sound, people will understand what you're saying.