[SalesForce] Updating custom page layout in a Managed Package

Unfortunately, you have your answer right there, beta packages can't be upgraded.

You will have to uninstall to get to the latest version. SFDC should help you restore your managed package data (in the installed package page) to help you restore your data after the re-install process has completed.

However, if your data structure has changed very much between versions this might not be helpful, and you should review the data you get BEFORE you uninstall.

Error codes of that nature indicate than an exception was thrown in the underlying application server that Salesforce runs on and was not trapped by any exception handlers. You'll need Salesforce (probably R&D) to investigate this for you as only they have access to the underlying logs and code. I've gained access to the log snippets before through some of my SFDC contacts, but it doesn't help without being able to see the code, and even if by some miracle I could spot the problem there's nothing I can do to fix it.

I'd be inclined to ask support if they have confirmed that the error that you are seeing is definitely caused by the known issue - otherwise its a pretty drastic change to have to make and I'd expect a few gotchas with downgrading to API v16. The last time I hit this type of issue with a managed package it was a bug in the packaging code that required a release to fix.