When I hear people speak of "this ungodly hour" they are usually complaining about being awake (or especially working) earlier than they are accustomed.
But why is this called ungodly? It would seem that the diligence to arise early and begin work would be a mark of godliness. It seems that an "ungodly hour" would be a time, perhaps after midnight, when being awake is often associated with ungodly behavior, such as drunkenness and debauchery.
What is the history of this phrase?
Edit: 'ungodly hour', does carry the connotation of early, not just different than expected. Otherwise, ungodly means immoral. http://www.google.com/search?q=ungodly+definition
ungodlier, comparative; ungodliest, superlative
Irreligious or immoral - ungodly lives of self-obsession, lust, and pleasure Unreasonably early or inconvenient - I've been troubled by telephone calls at ungodly hours