[SalesForce] Schedule apex class to run every minute as there can only be 5 batch jobs

I have a need to execute the Apex class every minute.

I have a Sites VF page and in that I have link called "Export to Excel" which exports bulk data (say 50000 records of my wrapper class type) through Batch Apex.

The application is used by about 100 users When more than 5 users click on that link simultaneously (at the same time), or within a minute batch jobs fails.

Am planning to have a custom object and make an entry in it whenever the users click on that link and populate whatever necessary details needed for executing that job and will also have a status column. If the status is yet to process and if the number of running process is less than 5 (or) 3, planning to call an apex batch job and this caller class will be a schedulable type and thought of scheduling it every minute. Unfortunately am unable to schedule it for every minute.

Any ideas for doing a custom implementation for scheduling my job every minute without installing any external apps? Or is there an alternative solution for my problem?

Best Answer

You could also create you own custom apex batch queue. I did this myself and created the GitHub project SObject Work Queue which had the following design goals:

  • Must prevent Max 5 batch in parallel limit - We should never run into this limit with work that is processed over the queue.
  • The queue needs to be so generic that "work" on any type of database object needs to be enqueued.
  • Any type of modification of database objects need to be possible. This should be transparently handled by the queue.
  • Provides better error diagnostics like Batch. Knows last successful Id, full stacktrace and sends email to developers.
  • Secures data integrity like Batch or better. Failures should not leave data in inconsistent state or user of the infrastructure should be able to handle them.
  • Optimistic locking : Instead of locking all many records we do not to process work on Ids that have other work already scheduled.
  • Work that can be run synchronously, should not be queued and processed asynch.

Maybe you want to check if that could be used in you case or collaborate on GitHub to extend it for your purpose.

Here is an overview sequence diagram that shows how work is defined, enqueued and processed in such a custom Apex Queue:

enter image description here

Related Topic