Here, fast doesn't mean "(capable of) moving quickly". Much rather, it is being used in the sense "firmly fixed" (see fasten your seatbelts or fast friends). The Phrase Finder says that "This is a nautical term. A ship that was hard and fast was simply one that was firmly beached on land." It adds that the term was used in figurative sense by the early 19th century.
Personally, I don't think it's being used more commonly in the negative. Indeed, a quick COCA search returns 40 occurences of "hard and fast rule" or "hard-and-fast rule", but only 22 of them are using it in a negative context — and I am being as generous as possible there, counting not only "no hard and fast rule" and "not a hard-and-fast rule", but also "don't have any hard and fast rule", "rather than any hard and fast rule", "was never a hard-and-fast rule" and the like.
One thing stands out to me, though: out of 16 occurrences of "hard-and-fast rule", with hyphens, 12 appear in a negative context, or 75%. For the non-hyphenated version, it's almost the other way round: 60% positive, 40% negative. (Again, counting "negative" very generously.)
The figures from the BNC are too small to be statistically meaningful. But anyway, here's an overview:
COCA BNC
total negative total negative
hard and fast rule 24 10 8 7
hard-and-fast rule 16 12 1 1
The specifics are proving rather elusive. However, yourdictionary.com notes:
A related expression is how the land lies, as in Let's be cautious
till we know how the land lies. This usage originated in Britain about
1700 as the lie of the land and is still so used there.
The modern definitions for 'lay' (or 'lie' for British English) do not include the meaning needed for this idiom, but must have at some point. That, or some other factor is needed to explain from whence the phrase evolved. The word which does fit the expression—layout—didn't turn up until 1910, according to etymonline.com. 'Layout' also implies a root definition for 'lay' that has been lost (well, transferred, anyway).
Best Answer
This was show-biz parlance in the 1920s, and it referred to a show that closed quickly after opening. The most famous usage was by Variety (a show-biz newspaper) in 1929 after the stock market crashed: "Wall Street Lays An Egg"
Some sources I have seen say that the original meaning came from the number 0, which is what is put up on a scoreboard when a team fails to score. It resembles an egg, and is still today called a "goose egg," so by extension when a team scores zero it "lays an egg."