[SalesForce] “SerialBatchApexRangeChunkHandler Internal Salesforce.com Error” from a batch execution

We are an ISV and we have a managed package. In the managed package we have an implementation of Database.Batchable that is also implementing Database.Stateful. It is global, so is part of our exposed API for use outside the package. The implementations of the start, execute and finish methods are only public and provide the basic implementation for the batch itself.

The class exposes some similar global abstract methods, onStart, onExecute and onFinish, to allow our class to be extended and processing implemented against these steps in the batch process flow. (There are good reasons why we have done this that I'm not covering in this explanation.)

Note that we use an abstract base class to provide our "secret sauce" behaviour rather than using an interface and some form of delegation arrangement because we want the implementation class name to appear in the Async Apex Jobs table, thereby allowing admins to know which batch is which.

All works fine for us when we have an extension of this class within our managed package; we can schedule the batch and have it execute fine.

To give an idea, part of the API looks like:

global abstract without sharing class AdaptiveBatch implements Database.Batchable<SObject>,
        Database.Stateful {
    public System.Iterable<SObject> start(Database.BatchableContext context) {
        ...
    }
    ...

    global abstract void onStart(Database.BatchableContext context);
}

The (customer-specific) implementation then might look like the following (adjusted with namespaces appropriately, and made global, when the implementation is outside the package):

public without sharing class TestAdaptiveBatch extends AdaptiveBatch {
    public override void onStart(Database.BatchableContext context) {
        ...
    }
    ...
}

However, when we have a customer-specific implementation extending this API outside the package we get the above error whenever the batch is executed.

We know that it isn't the typical problem of unserializable state in the batch – we have one SObject, a Date and a List<Set<Id>> in the customer-specific case and we know we can successfully JSON serialize and deserialize an instance of this batch implementation. We also know we can successfully queue the batch for execution, with this state defined, using Database.executeBatch.

When Salesforce tries to execute the batch, we get the error cited above. There is nothing useful in the debug logs, even when collected via subscriber login; just entry to the custom class, some heap allocation statements then the error itself before any methods in the custom code get invoked.

I have a suspicion that this may relate to cross-namespace issues; we have a custom class on the subscriber org (and in the org's "no namespace" namespace) that is a Database.Batchable, because it inherits that relationship from our API, but where the methods for the batchable are actually in our package namespace. This is the only conceptual difference between our in-package extension of our API and the customer-specific extension of our API.

Has anyone come across this issue and, if so, do you have a solution? Do we need to make our implementations of the start/execute/finish methods global to sort this out? Any other suggestions?

I am asking first because we will have to create a new (beta, with luck) release of our package just to test these possible changes, installing the updated package and testing the changes against a custom class on a temporary org. It would be nice to avoid a lot of iteration on this!

Thanks in advance