I have a requirement to publish articles to an external system for translation. This service is used when articles are created and when the articles are modified as well. Also, I would like to receive these articles from the external system after translation. I am thinking of two approaches – Batch Apex or Scheduled Job.
Batch Apex:
I would like to have a batch job to send one article(since an article could be of size up to 10MB) at a time for each execution. And in finish send a summary email about the successes and failures. Likewise, another batch job to receive one article at a time.
Scheduled Apex:
Upon second thought, since we are sending only one article per execution in batch. Can I chain scheduled job itself? By invoking the same job again after 5 minutes, I am thinking that I can achieve the same result as using a Batch job. By doing this, I believe I can save one of the 5 batch jobs limit.
Since I cannot send arguments to a scheduled job, I might have to use some custom object to pass the arguments between jobs. I feel like that this approach might warrant maintenance since I am using Custom Object. Likewise, scheduling another scheduled job for receiving articles.
I would like to know which is the optimal design and why. Please advise
Best Answer
I would recommend you use a pattern discussed in the 3rd Edition of Dan Appleman's Book Advanced Apex Programming. You mention using a custom object. I'm presuming you're saving articles to the custom object when you want to send them out for translation. That would work especially well for this pattern.
The pattern would have you use a trigger on the custom object. The trigger would save a record to a 2nd custom object with the recordId, and a few other details you'd want to have for the purposes of error handling as files are processed by your class, plus sent and retrieved from your web services. Let's call the 2nd object
DataMonitor.When the record is saved to the DataMonitor object. An After Insert trigger would fire that will call a queueable provided there are sufficient limits available for it to call it (queued at time <5 & no more than 100 jobs on hold in flex queue, plus under org limits for async apex). If there aren't, it will check to see if it can request a schedulable job instead (limit of 100 in org at a time plus under org limits for async apex). If it can call the queueable, the queueable will query for records in DataMonitor. If it finds any, it will retrieve what it can process (only one if that's what you say it can handle, more if that's possible). Similar would happen with the schedulable.
From there it will make the web services call out. If it's successful at initiating the callouts, it will write-back a success message to the records it sent; marking them as having been sent. If you wish, it can also send you an email with the results (check email limits first). Finally, if there are more records remaining and limits available, it can call another queuable to process them. If there isn't room in the queue, it could alternatively schedule another Schedulable to do them at a later time depending on the kind of limits available.
This is something of an "industrial strength" pattern in that it provides the ability to retry sending records if the first attempt isn't successful. If, after a couple attempt, or after receiving a recognizable error type, you can choose to write back the error to the
DataMonitorrecord and also send the email to the Admin with some kind of special notation on it, so they'll know they need to take action. I believe this pattern is discussed in Chapter 7 of Dan's book.