I'm writing an integration that requires polling Salesforce to pull down changed data. I'm worried about concurrent updates to the records that would cause me to 'miss' updates.
Concrete example:
Service A starts updating records at 5:15:10
Service B starts updating records at 5:15:20
Service B finishes updating records at 5:15:25 (5 seconds)
Service C starts its request at 5:15:30 and requests all records modified on or after 5:15:00 (Service A's job is still running)
Service C receives a response at 5:15:35 (5 seconds)
Service A finally finishes at 5:16:00 (50 seconds)
I'd expect Service C to see all the records modified by Service B since they were committed before Service C started its request — but none of the ones modified by Service A
I have a few choices about how I can ask for records next time I poll. I can use the highest SystemModstamp of the records returned in the first request as the 'high water mark'. Or I can return a DateTime.now() timestamp as from the server as part of the payload. But in either case, I would get a date that is higher than when Service A's transaction started — so if SystemModstamp is the 'beginning of transaction', I'd miss all of those updates when Service C polls again
The three questions are:
Is SystemModstamp the beginning or end of the transaction?
Or, even worse, is SystemModstamp different for each record updated in a transaction?
Does anyone have a better solution out of this mess?
I clawed through all of the Salesforce docs I could find and there's not a shred of info about this.
The maximum duration of a transaction is documented to be 10 minutes so the brute force solution is to always ask for stuff that's 10 minutes older than my high water mark — but that adds network overhead and extra client code to deal with the possible "repeats" in the stream of updates
Edit: @superfell's answer below is definitely exactly what I wanted — but I didn't put in my initial question that my endpoints are Apex REST so I don't have access to getUpdated from Apex. I'm not going to mark that as the solution for this question.
Edit again: There is Database.getUpdated() available to Apex — this question is answered thanks to @superfell
Best Answer
generally sysModStamp is sent when the row gets written to the db, but with triggers, workflow etc, there's no real way to generalize when it gets set vs the lifetime of the transaction. Its definitely never going to be transaction commit time.
If you use getUpdated() to do this, part of the response data is the timestamp of the start of the oldest inflight transaction, you can use this as the start time of your next poll. Otherwise, as you say, you can get holes due to the difference between when the sysModStamp is calculated vs when the transaction actually commits.
If you're doing anything remotely expensive on the receiving side, you'll want to have something that double checks the last sysModstamp you processed for each row to help cut down duplicate processing.