Switch – Dimmer switch remains powered with switch off!

You are right HandyMan - I called Lutron and it turns out there is a micro processor in this model which allows it to work well with led bulbs which causes this delay. It is not defective nor the wiring. They said they have a newer version of a similar model called the Diva, which has no delay when turning on the light. They said that there is a newer model of the Skylark coming out soon too which will eliminate the delay. Thank you all for your help with this question! :)

Well big ups to @ThreePhaseEel for the suggestion to call Lutron customer support. Did not expect them to have 24/7 support, and stateside to boot. I first verified that one of the switches was a 4-way switch (visible even with the panel plate still on as the switch is beefier than a normal light switch) and that I had not messed with that, so was good there.

Lutron support was indeed very helpful - I explained the scenario with what I was experiencing, and the guy said it is probably a "current limiter" on the fan. Apparently all ceiling fans manufactured after Jan 1, 2007 are required to have a light bulb wattage limiter per the Energy Policy Act of 2005 which caps the total wattage allowed to be used by all bulbs at 190 watts. I was using three candelabra-style 60w incandescent bulbs, and as mentioned they did work normally with the dimmer switch after the light and/or switch was on and presumably warmed up after 10 minutes or so. The tech. support guy with Lutron explained that the way dimmer switches work is that they basically cycle the power on and off at about 120 times per second, which matches the line cycle rate or something (I'm not an electrician, obviously). But, he thought the limiter might be interpreting that as some sort of power spike, and I guess when the limiter / light wasn't "warmed up". This made sense as I noticed there was a light but audible click whenever the lights would turn off during an episode of this annoying behavior.

So, to test this theory, I tried turning the light off for a half hour to cool it, and then removed one bulb to reduce the wattage used (now 2 bulbs), and when I turned the light on and moved the dimmer toggle slide up to full blast, the lights did not turn off. This told me that the regular incandescent bulbs in concert with the dimmer and the stock wattage limiter were causing the light to switch off as it had. I ran out and bought 3 60w equivalent dimmable LED candelabra style lights which use something like 6.5 watts each total, and I installed those, and sure enough, they worked perfectly when turning the light on. Good to go.