Anything that dilutes the cream is going to make it less firm/stable - whipped heavy cream can last for hours, while whipped light cream (such as Reddi-Whip) won't last more than five-ten minutes. I've never had any whipped cream last for days, however, unless it contained a stabilizer such as gelatin.
1 tablespoon brandy per cup of cream does sound a bit much - just for comparison, you'd use maybe a third that much vanilla extract for the same amount of cream. (You were probably using much less Amaretto, and Bailey's contains cream so it doesn't dilute things as much.) If you can't get the flavor you want with a reduced quantity of brandy, you could try making a hard sauce (butter + alcohol) to serve with your dessert, in addition to or instead of plain whipped cream. Or you could try a whipped cream stabilizer - either the stuff you can buy specifically made for the purpose, or unflavored gelatin carefully dissolved in a part of the cream.
Note that for best results, you should add the sugar and flavorings to the cream before starting to whip. It sounds like you already know to keep everything as cold as possible.
It sounds to me like you didn't get the gelatin dispersed properly.
Since the recipe you linked to doesn't explicitly mention it, and neither does your question, I'm going to assume that you didn't bloom the gelatin first. You must do this if you want proper dispersion of any gelatin product.
To bloom powdered gelatin, just sprinkle it over cold liquid and leave it (cold) until it's visibly swelled. This usually takes about 5 minutes. Then simply heat and stir (thoroughly) to disperse.
To bloom sheet gelatin, you do almost the same thing - soak them for a little longer (up to 10 minutes) in cold water a separate vessel, then squeeze them to wring out any excess water, and then add to your cooking liquid (in this case cream).
You might also consider use a stick blender/immersion blender to make sure that it is completely dispersed after melting. This is usually unnecessary for gelatin, but if you're blooming it and still can't get uniform distribution, that will help.
I'm certain that if you follow the proper preparation, your panna cotta will turn out fine next time.
P.S. Do be careful not to add too much lime or other acid; gelatin will not set properly at pH levels around or below 4.
Best Answer
I have a 4.5l kitchen aid, the most cream I have ever whipped in it was 1l. The limitation is the wire whisk, you don't want so much that the level gets above the level at which the whisk can contact it, as then it won't whip effectively.
Too much also will get messy, getting all over the rotating parts.