Build large single XML Document over multiple batches

apexbatchgovernorlimitsxml

I would like to understand if it's possible to generate large XML files using XmlNode and Document classes in Batch apex?

Ideally, I would like the Dom.Document class variable to be stateful. However, this is not possible. As an attempted workaround, I have serialised the document into a string. This works in execute anonymous however it does not work in batch (System.SerializationException: Not Serializable: dom.XmlNode).

I have tried creating the file synchronously and also using @Future however both failed with CPU timeout beyond a small number of records. That is why I would like to create the XML document in small chunks within Salesforce governor limits.

Note that the code below has been simplified for illustration purposes.

Thanks in advance

public with sharing class ExampleBatchJob  implements Database.Batchable<sObject>, Database.Stateful {

Dom.Document doc = new DOM.Document();
Dom.XmlNode root = doc.createRootElement('AccountPerformanceSalesData', null, null);
String theDocAsString = doc.toXmlString();

public Database.QueryLocator start(Database.BatchableContext BC) {
    String query = 'SELECT Id FROM Account';
    return Database.getQueryLocator(query);
}

public void execute(Database.BatchableContext BC, List<sObject> scope) {

    Dom.Document doc2 = new DOM.Document();
    doc2.load(theDocAsString);
    List<Account> accountList = scope;
    for (Account acc : accountList) {
        doc2.getRootElement().addChildElement('test', 'test', 'test');
    }
    theDocAsString = doc2.toXmlString();
}

public void finish(Database.BatchableContext BC) {
    System.debug(theDocAsString);
}

Best Answer

You can't serialize doc or root. Remove them from the class, and initialize theDocAsString in your start(Database.BatchableContext context) method.

public Database.QueryLocator start(Database.BatchableContext BC) {
  Dom.Document doc = new Dom.Document();
  doc.createRootElement(...);
  theDocAsString = doc.toXmlString();
  return Database.getQueryLocator([SELECT Id FROM Account]);
}

Related Solutions

[SalesForce] Build Custom Batch Queue to centrally manage Batches & circumvent the Max. 5 Batches in parallel

I've done this a few times. It works well. It can take a while to get the thing running end-to-end, since debugging can be arduous.

A few suggestions:

in your command table, add a "log" long text field and write back relevant status/debugging info there.
catch all exceptions from your batch manager and write them out to the log rather than blowing up.
you can use the Apex dynamic instantiation (reflection) to allow your command table to hold actual class names if you want to maximize flexibility
I use a number of common base classes for my batch jobs, that do certain common stuff like logging, recording start/stop times and record counts, self-rescheduling (via command table) for recurring jobs, etc.

Hope that helps.

[SalesForce] Transaction Control for multiple batches

When you perform an update, and that update performs additional updates via a trigger, you can partially fail the transaction to perform an automatic rollback and partial retry.

Consider the following trigger:

trigger cascadePhone on Account (after update) {
    // List of numbers that were changed from
    Set<String> oldPhones = new Set<String>();

    // Find all phones that changed and record those
    for(Account record: Trigger.old) {
        if(record.Phone != Trigger.newMap.get(record.Id).Phone) {
            oldPhones.add(record.Phone);
        }
    }

    // Find all contacts that might match
    Contact[] contacts = [SELECT    Id, AccountId, Phone 
                          FROM      Contact 
                          WHERE     AccountId IN :Trigger.new AND 
                                    Phone IN :oldPhones];

    // Remove contacts that have the right phone but wrong account
    // We count backwards because we're removing records in place in the list
    // and if we counted forwards we'd skip rows
    for(Integer index = contacts.size()-1; index >= 0; index--) {
        if(contacts[index].Phone == Trigger.oldMap.get(contacts[index].AccountId).Phone) {
            contacts[index].Phone = Trigger.newMap.get(contacts[index].AccountId).Phone;
        } else {
            contacts.remove(index);
        }
    }

    // Partial update
    Database.SaveResult[] results = Database.update(contacts, false);

    // For each record that failed, mark the account as failed as well
    for(Integer index = 0; index < contacts.size(); index++) {
        if(!results[index].isSuccess()) {
            Trigger.newMap.get(contacts[index].AccountId).Phone.addError('Failed to update some phone numbers');
        }
    }

}

If you perform an update of accounts in an execute function, like this:

public static void execute(Database.BatchableContext bc, Account[] scope) {
    // Fix accounts in scope
    // Then update them
    Database.update(scope, false);

}

What's not obvious here is that the system automatically performs a rollback on all records that error out, and attempt to retry the accounts that are left. This is an implied partial callback. Nowhere am I using a SavePoint to rollback the errors; the system handles the logic for me. The child records that failed also roll back, so the contacts that were on the accounts that failed would also not update as a result of this code.

Best Answer

Related Solutions

[SalesForce] Build Custom Batch Queue to centrally manage Batches & circumvent the Max. 5 Batches in parallel

[SalesForce] Transaction Control for multiple batches

Related Topic