I've started DMing a new group. We've only had two short, 2hour sessions, and most players are new to DnD. I've been a DM before (though just a dozen or so sessions) but in those cases it was with a group who had years of experience, so I only have a little experience leading a game.
Right now the players seem to play the game in a way that is best described as 'everything is a simple linear story and everything that happens is there to progress the story'.
For example, they go to a castle that was used by mages as a school of magic. My idea was for the castle to be abandoned – after investigation they would find that it was the site of a large battle, and that would lead them to the 'main' quest line I had in mind.
When they got to the castle, they spend about half an hour (real time, not in-game time) banging the door waiting for it to be opened. When that didn't happen (after I told them maybe they should start considering alternative ways in), they found a cave off the path leading towards the castle, and went in. In the cave, they found a large cavernous opening, 40-50 feet tall, 150 feet across, with some light trickling in from the sky through a small opening. When one of them asked me where they were with respect to the castle and then rolled a natural 20 on their survival check, I said that this would be under the castle somewhere.
As a result, instead of looking for ways out of this cavern where I planned some interesting encounters with a group of goblins, they spent almost all of the remainder of the session trying to get a grappling-hook style thing up through the little hole and then climb out.
I hope that somewhat paints the issue I'm having right now – the PCs never seem to look for clues and just expect everything to be there because they are supposed to 'use' it (there is a hole with light, it must be the way out). Even after repeatedly failing to do what they think they should do, they just keep going, until I tell them to try something else, reminding them that not everything in the world is there just to be in their story.
I don't think I am railroading, though reading back over my question it does appear that way. I didn't plan anything ahead of time. Simply, I had in mind that the castle would be empty, and abandoned after a fight. Then after a while of banging on the door one of the PCs went 'I want to see if there are any caves or tunnels around' and I thought hey, I guess that would make sense! And so on.
Best Answer
Your players are new to the game (and new to your game). Unless you've told them what to expect, how could they know what your game is like? Getting stuck investigating small details or trying to get their way in through the door could plausibly be what your game is about. I think you should be more active in leading the game towards the direction you'd like to see it play out in.
The solution to this problem is simple in principle: don't let them do this. You are not obliged to let the players try banging on the door indefinitely long in real time. Consider the following example:
This can go on for arbitrarily long unless you or the players stop it. Do something like this instead:
If you want to be really explicit about it --- which I tend to recommend especially to new players and GMs, add something like:
The important trick is that you don't let your players spend time on anything boring, and that you pass the long, boring time their characters spend on knocking the door with a sentence or two. Repeated attempts at mundane tasks is something you don't have to focus on, and indeed shouldn't --- time spent in-universe doesn't have to be mirrored in real time.
Similarly, your example about the hole in ceiling shouldn't take an hour under any circumstances: the first time a player attempts to grapple themselves there, you can come up with a reason why they can't go there:
Or, to take an alternate path:
Whatever you do, don't make your players try something and roll for minutes. If the task is something where failure wouldn't be interesting and they could just try again, let them succeed without a roll. If it's something that can't work out, tell them it's impossible after the first try.
A few words on railroading and linearity
You say your players seem to think of the game as a simple, linear story, but to be honest, that sounds like a pretty good assessment. You have a castle where you've planned a lot of content that's accessible only through the caverns where you've also planned a lot of content, and you're upset because the players aren't engaging with your content. You are running a railroaded campaign.
A common misconception is that railroading is bad, period. It's not. Railroading is fine, as its own genre, but you need to be open to your players about what kind of a game this is and need to have clear cues for your players to follow in order for it to work. Without clear directions, your players are still as railroaded as before, but will spend time stumbling, feeling their way in through the invisible railroad instead of having fun advancing along it.
A few words about storytelling
What you're saying is contrary to the usual principles of RPGs and storytelling. You're making a story with the players: everything in the world exists for that story and the players' (including you, the GM) enjoyment, and not for anything else. Isn't that why the whole world you're playing in exists?
Different genres of fiction handle different narrative tropes differently, but one fairly major principle that works its way across genres is that of Chekhov's Gun: if there's a gun hanging on the wall in a play, it is there to be fired. More generally, if a work of fiction describes something, the reader/viewer/listener/player will assume it is going to be important and is usually absolutely right.
This is not necessarily true, nor necessarily realistic (for groups who prefer that sort of thing). However, it is so prevalent in normal storytelling that it's essential that you make sure your players and you are on the same page regarding how it is to be applied, because your players are going to assume it holds, consciously or not.
Think about it: you went through the trouble describing an appealing hole in the ceiling and told them they were under the castle they were supposed to infiltrate --- it seems obvious to anyone who hasn't heard your side of the story that the hole is of vital importance!
The Same Page Tool & Talking with your players
Finally, check out the handy Same Page Tool which gives you some things to discuss with your players regarding the type of game you want to play. Note that it's not a survey, nor a list of things to vote on, it's a discussion aid. It might help you clarify what kind of a game you want to play, overall.
You should also never hesitate to talk with your players, off-game, if you're not having fun with the way the game is playing out, and encourage them to be open with you in a similar manner. Table time is too valuable to spend on rolling for futile grappling hook attempts or other fumbling.