Because every attempt to change it makes the law more complex and more expensive. The archaic terms, like 'plaintiff' and 'writ' had clear definitions, partly because they had been hammered out over generations. A well-intentioned attempt to make it easier for the layman to understand (which every new Government tries to bring in) replacing them with 'claimant' and 'claim form' ends up causing confusion at the margins: if one partner in a marriage starts the divorce process but makes no financial claim while the other does, which one is the claimant? A writ was different in important respects from an originating summons; if both are replaced by a Claim Form, does the distinction survive? And, of course, confusion at the margins is where lawyers are needed, and where it gets expensive.
This is not pure guesswork on my part; in 2000, the British courts made a huge effort to simplify the civil courts' language and procedure. Everybody agreed that it was overdue, as the Rules of the Supreme Court were over a thousand pages long, and had to be re-issued every two years. Several of the brightest people in the country spent years rewriting the rules, and I was one of the people who had to try to make it work. It failed miserably; it turned out that words mean different things in different contexts (as any contributor here knows), so to replace one archaic term needed several near-synonyms, and each occurrence had to be scrutinised by the whole committee to see which applied. In practice, of course, this was impossible.
(As an analogy, try to debug a thousand-page program where each occurrence of each variable has to be replaced with one of six alternatives, but nobody knows which; and, of course, regular updates change both the variables and the structure while you're working. Large parts of it are safety-critical, since it bears on the liberty of the subject; and it has to be compatible with other systems across the world, in a variety of languages and with different philosophies.)
The current Civil Procedure Rule book is heading towards two thousand pages, and is replaced every year, with quarterly updates.
And, of course, the expense and confusion were not blamed on the politicians who ordered the change (and moved on before it happened), but on the lawyers and civil servants.
TL;DR: Because archaic English is clear and logical, while modern replacements are more confusing than helpful.
The short answer is that you may employ the simple past perfect to express a continuing action only when the expression is atelic or bears in context a reasonably natural atelic interpretation.
A telic expression is one which has a goal or ending point "built in" to its sense—finish, for instance. Employing the test suggested in the article linked above, it makes perfect sense to say He finished in an hour, but not (normally) He finished for an hour.
Expressions which do not have such a goal are atelic. In your first example, work is an atelic expression: using the same test, He worked for an hour is acceptable, but not (normally) He worked in an hour. Atelic expressions are, so to speak, inherently continuous. Consequently, a simple past perfect construction use supports a continuous sense; this is why the two are "interchangeable".
Your other examples, however, are telic. Eating dinner and cleaning a room are not (normally) protracted indefinitely, they come to an end when the dinner is consumed and the room is clean. Consequently, using simple perfect constructions implies completion, and if you want to convey that the action continues you must employ a progressive construction.
Note, however, that "telicity" is a very subtle matter in practice. As the linked article tells you, grammarians are in some disagreement over just how it works; and I have been careful to qualify all my analyses with the (normally) tag.
Note, too, that there is an alternative to the two constructions you illustrate. The past progressive ("I was eating dinner when ... " and "I was cleaning my space when ... ") is more natural to my ear than the past perfect progressive. You want the past perfect progressive only if you employ a qualifier like since dawn, which removes the focus from the present-in-the-past to the past-in-the-past, the stretch of time which preceded the present-in-the-past.
Best Answer
It's acceptable legalese. Legal writing uses many conventions not commonly used in day-to-day writing. This same usage of at is applied in legal descriptions of real estate as well, often describing a property as being at a city or at a county rather than in it. Legal jargon has evolved expressly for legal purposes, so ex curia, it is often not standard fare. However, to be fair, such linguistic oddities are not exclusive to the field of law and its jargon. It is quite usual for an industry or a field to have special words and grammar that is understood by its members and a few others in the know but not by lay persons. Such is the nature of jargon: to create or bend language in order to widely meet specialized needs.