No to both questions. The only one which might come close is Broken Steel, which expands on the core story a bit. However, it isn't necessary for any core understanding of the main storyline.
The other advantage of Broken Steel is that it does (SPOILER ALERT) allow you to continue playing after the final quest.
According to the Fallout Wiki, knowledge of Fallout 3 is not needed for Fallout: New Vegas, and there are no storyline connections, although like in previous games, there will be the occasional reference to the prior ones in the series.
(I should note that I've not played the original two, and had no issues with 3 and its references to them, so I'm guessing any references from New Vegas to prior ones will be similarly subtle. I do, however, plan on playing the original two!)
Finally, in terms of the DLC, I did purchase, play and enjoy all of them. I was not super-thrilled with Operation: Anchorage, but I thought the others were all exceptionally well-done (even if Mothership Zeta had really nothing to do with the Fallout universe). So naturally your mileage may vary, but it's worth noting that if you want some great additional (and different) stories, the add-ons are, to me, worth the money. Broken Steel is the only one which didn't offer a completely new environment: The others all offered large new lands to explore, with stories to go with them.
Either choice you make in the rotunda will "kill" you. It will show you the black and white film and apparently "end" the game even with the expansion. However, after the "end" finishes, it should allow you to continue (unlike without the expansion, where I believe it just takes you back to the main menu).
Best Answer
I'm not sure how it works in the GOTY edition, but if you purchase the expansion packs separately then the missions will appear in your quest log automatically (after a text box pop-up describing some radio signal or something similar).
The only expansion which ties into the main story is Broken Steel - it is an extension of the main storyline beyond the regular ending, and allows you to continue playing after completing the main quest line (unlike the original).
The others are all totally standalone, all taking place in alternate locations to the Capital Wasteland, and so it makes no difference to the main story when you start them.
In terms of toughness - Point Lookout definitely has the hardest enemies to fight, so I would tackle that one later on. The others are fairly playable at any decent level I think.