Learn English – What does the phrase “Follow that!” mean

That's as may be means that what the speaker says may be true, but it doesn't change your argument or opinion as it is not strictly relevant, or other facts need to be taken into consideration; I can't find any reference for it online but a phrase with a similar meaning is be that as it may.

It's usually meant literally, but "tea" doesn't just refer to the drink, but refers to the meal known as tea. There are quote a few different types of meal called tea, depending on which country or which part of the UK you come from.

For example, there's "low tea", or "afternoon tea", usually a light snack such as sandwiches or eaten between 2pm and 5pm. Nowadays, this can also be a treat in a cafe or hotel with cakes, pastries, or scones and jam and cream. (See this book and this book.)

There's also "high tea", an early evening meal, eaten between 5pm and 7pm. This would be followed by a larger meal later on. Low and high refer to the height of the table where tea was eaten.

Tea can also refer to the main evening meal in the north of England, Scotland, Wales and Northern Ireland.

"Taking tea" is a common phrase for having this meal or snack or just sitting down to brew up, pour and drink tea; it is somewhat of a ceremony. For what it's worth, here's an Ngrams chart (and there's not much difference in shape between UK and US English):

"Children to compete for chance to take tea with the Mad Hatter."

This clearly refers to the Lewis Carroll's Alice's Adventures in Wonderland where in chapter 7 the Mad Hatter is having a tea party. It's always six o'clock, and they're always drinking tea. There's bread and butter laid out as well:

'Take some more tea,' the March Hare said to Alice, very earnestly.

'I've had nothing yet,' Alice replied in an offended tone, 'so I can't take more.'

'You mean you can't take less,' said the Hatter: 'it's very easy to take more than nothing.'

'Nobody asked your opinion,' said Alice.

'Who's making personal remarks now?' the Hatter asked triumphantly.

Alice did not quite know what to say to this: so she helped herself to some tea and bread-and-butter, and then turned to the Dormouse, and repeated her question. 'Why did they live at the bottom of a well?'

But as to the original question, the winning children would join the Hatter's tea party. There might be actual drinks and snacks for the children, or it could be meant that the winners would get to spend time with the Hatter.