I had to write a trigger to manage a lead deduplication process designed by someone else.
In a nutshell:
- A lead gets inserted
- Checks on name and email of existing leads and contacts (person accounts)
- If no duplicate is found, lead must get associated to both a Talent campaign and a Inbound campaign; this happens through lead fields TalentSource__c and InboundCampaign__c
, which always have the name of a campaign as a value. So a campaign member is created for both campaigns and they get inserted together with the lead. - If a duplicate is found (either in another lead or a person account), a check on existing campaign members is done; if something gets found, the campaign member fieds get updated with some info from the incoming lead; if nothing is found, the same procedure as above happens, with the difference that the existing lead/person account gets updated with some new info from the incoming lead and the new lead gets deleted.
Now, I have gone through the trigger many times and basically had to rewrite it a couple times, since new requirements kept coming in from the guy who designed the process (who happens to be a less Salesforce expert than me).
I've done my best in order to write a good trigger, trying to avoid governor limits. Trigger works fine until I try to bulk insert new leads. It works until I try with 400 leads; if I go up to 500 I start getting the error.
I think it is mainly due to for loops running too many times, but I don't know how I could do otherwise.
Here's the trigger code (it's a bit of a long one):
public with sharing class LeadTriggerUtil {
public static void duplicateLeadManagement_setup(Lead[] leadsList) {
System.debug('isBefore');
System.debug('DEBUG LEADSLIST BEFORE: ' + leadsList.size());
for (Lead l : leadsList) {
HLP_duplicateLeadManagement.talentCampaignsMAP.put(l.TalentSource__c, null);
HLP_duplicateLeadManagement.inboundCampaignsMAP.put(l.InboundCampaign__c, null);
// storing new lead names and emails for duplicate check
HLP_duplicateLeadManagement.newNameSet.add(l.LastName);
HLP_duplicateLeadManagement.newEmailSet.add(l.Email);
}
HLP_duplicateLeadManagement.talentCampaignsMAP2.putAll([SELECT Id, Name FROM Campaign WHERE Name IN :HLP_duplicateLeadManagement.talentCampaignsMAP.keySet()]);
HLP_duplicateLeadManagement.inboundCampaignsMAP2.putAll([SELECT Id, Name FROM Campaign WHERE Name IN :HLP_duplicateLeadManagement.inboundCampaignsMAP.keySet()]);
HLP_duplicateLeadManagement.talentCampaignsMAP.clear();
HLP_duplicateLeadManagement.inboundCampaignsMAP.clear();
// IF B2C????
String B2CID = Schema.SObjectType.Lead.getRecordTypeInfosByName().get('B2C').getRecordTypeId();
HLP_duplicateLeadManagement.dbLeads = [SELECT id, LastName, Email FROM Lead WHERE RecordTypeId=:B2CID AND (Email IN: HLP_duplicateLeadManagement.newEmailSet AND LastName IN: HLP_duplicateLeadManagement.newNameSet)];
HLP_duplicateLeadManagement.dbPersonAccounts = [SELECT Id, Email, LastName FROM Contact WHERE isPersonAccount=true AND (Email IN: HLP_duplicateLeadManagement.newEmailSet AND LastName IN: HLP_duplicateLeadManagement.newNameSet)];
System.debug('newNameSet: ' + HLP_duplicateLeadManagement.newNameSet);
System.debug('newEmailSet: ' + HLP_duplicateLeadManagement.newEmailSet);
System.debug('PERSON ACCOUNTS: ' + HLP_duplicateLeadManagement.dbPersonAccounts);
}
public static void duplicateLeadManagement_process(Lead[] leadsList) {
List<Campaign> talentCampaignsForDup = [SELECT Id FROM Campaign WHERE RecordType.Name = 'Talent'];
List<Lead> leadsToDelete = new List<Lead>();
List<CampaignMember> membersToUpdate = new List<CampaignMember>();
List<CampaignMember> membersToInsert = new List<CampaignMember>();
List<Lead> leadsToUpdate = new List<Lead>();
List<Contact> contactsToUpdate = new List<Contact>();
List<CampaignMember> cms = new List<CampaignMember>();
List<CampaignMember> cmsPA = new List<CampaignMember>();
Map<Id, String> leadType = new Map<Id, String>();
System.debug('isAfter');
System.debug('DEBUG LEADSLIST AFTER: ' + leadsList.size());
// System.debug(HLP_duplicateLeadManagement.leadType);
if (HLP_duplicateLeadManagement.dbLeads.size() > 0) {
// RETRIEVING CAMPAIGN MEMBERS OF DUPLICATE LEADS
cms = [SELECT Id, LeadId FROM CampaignMember WHERE LeadId IN :HLP_duplicateLeadManagement.dbLeads AND CampaignId IN : talentCampaignsForDup];
}
//// RETRIEVING CAMPAIGN MEMBERS OF DUPLICATE PERSON ACCOUNT
if (HLP_duplicateLeadManagement.dbPersonAccounts.size() > 0) {
cms = [SELECT Id, ContactId FROM CampaignMember WHERE ContactId IN :HLP_duplicateLeadManagement.dbPersonAccounts AND CampaignId IN : talentCampaignsForDup];
}
for (Lead newLead : leadsList) {
// setting value to update to hasemailoptout
Boolean hasOptOut;
if (newLead.TalentConsent__c == true) {
hasOptOut = false;
} else {
hasOptOut = true;
}
// setting lead type
if (newLead.TalentSource__c != 'Realize Media') {
if (newLead.RecordTypeId == Schema.SObjectType.Lead.getRecordTypeInfosByName().get('B2C').getRecordTypeId()) {
leadType.put(newLead.Id, 'B2C');
} else {
leadType.put(newLead.Id, 'B2B');
}
} else {
leadType.put(newLead.Id, 'B2B');
}
// for each campaign ID in talentCampaignsMap2 keys check if the corresponding campaign name value equals the new lead TalentSource fields;
// if so, have the other map (which is empty) populated with the new lead TalentSource as key and the corresponding campaign as value
for (Id idCampaign : HLP_duplicateLeadManagement.talentCampaignsMAP2.keySet()) {
if (HLP_duplicateLeadManagement.talentCampaignsMAP2.get(idCampaign).Name == newLead.TalentSource__c) {
HLP_duplicateLeadManagement.talentCampaignsMAP.put(newLead.TalentSource__c, HLP_duplicateLeadManagement.talentCampaignsMAP2.get(idCampaign));
}
}
// same as above but for inbound campaigns
for (Id idCampaign : HLP_duplicateLeadManagement.inboundCampaignsMAP2.keySet()) {
if (HLP_duplicateLeadManagement.inboundCampaignsMAP2.get(idCampaign).Name == newLead.InboundCampaign__c) {
HLP_duplicateLeadManagement.inboundCampaignsMAP.put(newLead.InboundCampaign__c, HLP_duplicateLeadManagement.inboundCampaignsMAP2.get(idCampaign));
}
}
system.debug('FINAL SOURCE-CAMPAIGN MAP: ' + HLP_duplicateLeadManagement.talentCampaignsMAP);
system.debug('FINAL SOURCE-INBOUND CAMPAIGN MAP: ' + HLP_duplicateLeadManagement.inboundCampaignsMAP);
// System.debug('LEADTYPE MAP: ' + HLP_duplicateLeadManagement.leadType);
// System.debug('LEADTYPE FOR ENTERING LEAD: ' + HLP_duplicateLeadManagement.leadType.get(newLead.Id));
// IF LEAD IS B2C
if (leadType.get(newLead.Id) != 'B2B' && !String.isBlank(newLead.LastName) && !String.isBlank(newLead.Email)) {
if (HLP_duplicateLeadManagement.dbLeads.size() > 0) {
// duplicate lead found - running lead deduplication
System.debug('DUPLICATE LEAD FOUND');
// DEDUP LEAD
dedupLead(newLead, cms, leadsToDelete, membersToUpdate, membersToInsert, leadsToUpdate, hasOptOut);
} else if (HLP_duplicateLeadManagement.dbPersonAccounts.size() > 0) { /// PERSON ACCOUNT CHECK
System.debug('DUPLICATE PERSON ACCOUNT FOUND');
// duplicate contact (person account) found - running PA deduplication
dedupPersonAccount(newLead, cms, leadsToDelete, membersToUpdate, membersToInsert, contactsToUpdate, hasOptOut);
} else { // duplicate not found - creating campaign member
System.debug('NO DUPLICATE');
System.debug(HLP_duplicateLeadManagement.talentCampaignsMAP);
if (HLP_duplicateLeadManagement.talentCampaignsMAP.size() > 0) {
CampaignMember cm = new CampaignMember();
Lead newLeadCopy = new Lead();
newLeadCopy.Id = newLead.Id;
cm.LeadId = newLead.Id;
cm.CampaignId = HLP_duplicateLeadManagement.talentCampaignsMAP.get(newLead.TalentSource__c).Id;
cm.HasOptOutOfTalent__c = hasOptOut;
cm.LastUpdateHasOptOutOfTalent__c = newlead.DateTalentConsent__c;
membersToInsert.add(cm);
if (HLP_duplicateLeadManagement.inboundCampaignsMAP2.size() > 0) {
CampaignMember cm2 = new CampaignMember();
cm2.LeadId = newLead.Id;
cm2.CampaignId = HLP_duplicateLeadManagement.inboundCampaignsMAP.get(newLead.InboundCampaign__c).Id;
membersToInsert.add(cm2);
}
newLeadCopy.LastDateHasOptOutOfEmail__c = newLead.DateTalentConsent__c;
newLeadCopy.HasOptedOutOfEmail = hasOptOut;
newLeadCopy.SyncToMarketingCloud__c = true;
newLeadCopy.RecordTypeId = Schema.SObjectType.Lead.getRecordTypeInfosByName().get('B2C').getRecordTypeId();
leadsToUpdate.add(newLeadCopy);
} else {
system.debug('Talent campaign not found - cannot execute code');
}
}
} else {
// LEAD IS B2B - WILL BE PROCESSED WITH STANDARD DUPLICATE RULE
System.debug('Lead is either B2B or doesn\'t respect entry criterias for processing');
}
}
System.debug('LEADS TO UPDATE: ' + leadsToUpdate );
if (membersToUpdate.size() > 0) {
Database.update(membersToUpdate, false);
System.debug('members updated');
}
if (membersToInsert.size() > 0) {
Database.insert(membersToInsert, false);
System.debug('members inserted');
}
if (leadsToUpdate.size() > 0) {
Database.update(leadsToUpdate, false);
System.debug('leads updated');
// leadsToUpdate.clear();
}
if (contactsToUpdate.size() > 0) {
Database.update(contactsToUpdate, false);
System.debug('person accounts(contacts) updated');
// leadsToUpdate.clear();
}
if (leadsToDelete.size() > 0) {
Database.delete(leadsToDelete, false);
System.debug('leads deleted');
}
}
public static void dedupPersonAccount(Lead newLead, List<CampaignMember> cmsPA, List<Lead> leadsToDelete, List<CampaignMember> membersToUpdate, List<CampaignMember> membersToInsert, List<Contact> contactsToUpdate, Boolean hasOptOut) {
for (Contact c : HLP_duplicateLeadManagement.dbPersonAccounts) {
if (c.LastName == newLead.LastName && c.Email == newLead.Email) {
System.debug('LEAD MATCHED WITH PERSON ACCOUNT!');
if (cmsPA.size() > 0) {
for (CampaignMember cm : cmsPA) {
if (cm.ContactId == c.Id) {
cm.HasOptOutOfTalent__c = hasOptOut;
cm.LastUpdateHasOptOutOfTalent__c = newLead.DateTalentConsent__c;
membersToUpdate.add(cm);
}
}
} else {
// creazione campaign member e associazione campagna
System.debug('lead duplicato con Person Account - creazione campaign member e associazione campagna');
CampaignMember cm = new CampaignMember();
cm.ContactId = c.Id;
cm.CampaignId = HLP_duplicateLeadManagement.talentCampaignsMAP.get(newLead.TalentSource__c).Id;
cm.HasOptOutOfTalent__c = hasOptOut;
cm.LastUpdateHasOptOutOfTalent__c = newlead.DateTalentConsent__c;
membersToInsert.add(cm);
if (HLP_duplicateLeadManagement.inboundCampaignsMAP2.size() > 0) {
CampaignMember cm2 = new CampaignMember();
cm2.LeadId = c.Id;
cm2.CampaignId = HLP_duplicateLeadManagement.inboundCampaignsMAP.get(newLead.InboundCampaign__c).Id;
membersToInsert.add(cm2);
}
}
// updating lead in DB with new lead data
c.HasOptedOutOfEmail = hasOptOut;
system.debug('CONTACTS TO UPDATE BEFORE ADD: ' + contactsToUpdate);
contactsToUpdate.add(c);
system.debug('CONTACTS TO UPDATE AFTER ADD: ' + contactsToUpdate);
Lead leadToDel = new Lead(Id=newLead.Id);
leadsToDelete.add(leadToDel);
}
}
}
public static void dedupLead(Lead newLead, List<CampaignMember> cms, List<Lead> leadsToDelete, List<CampaignMember> membersToUpdate,
List<CampaignMember> membersToInsert, List<Lead> leadsToUpdate, Boolean hasOptOut) {
for (Lead l : HLP_duplicateLeadManagement.dbLeads) {
if (l.LastName == newLead.LastName && l.Email == newLead.Email) {
System.debug('LEAD MATCHED!');
if (cms.size() > 0) {
for (CampaignMember cm : cms) {
if (cm.LeadId == l.Id) {
cm.HasOptOutOfTalent__c = hasOptOut;
cm.LastUpdateHasOptOutOfTalent__c = newLead.DateTalentConsent__c;
membersToUpdate.add(cm);
}
}
} else {
// creazione campaign member e associazione campagna
System.debug('lead duplicato - creazione campaign member e associazione campagna');
CampaignMember cm = new CampaignMember();
cm.LeadId = l.Id;
cm.CampaignId = HLP_duplicateLeadManagement.talentCampaignsMAP.get(newLead.TalentSource__c).Id;
cm.HasOptOutOfTalent__c = hasOptOut;
cm.LastUpdateHasOptOutOfTalent__c = newlead.DateTalentConsent__c;
membersToInsert.add(cm);
if (HLP_duplicateLeadManagement.inboundCampaignsMAP2.size() > 0) {
CampaignMember cm2 = new CampaignMember();
cm2.LeadId = l.Id;
cm2.CampaignId = HLP_duplicateLeadManagement.inboundCampaignsMAP.get(newLead.InboundCampaign__c).Id;
membersToInsert.add(cm2);
}
}
// updating lead in DB with new lead data
l.InboundCampaign__c = newLead.InboundCampaign__c;
l.LastDateHasOptOutOfEmail__c = newLead.DateTalentConsent__c;
l.HasOptedOutOfEmail = hasOptOut;
l.SyncToMarketingCloud__c = true;
system.debug('LEADS TO UPDATE BEFORE ADD: ' + leadsToUpdate);
leadsToUpdate.add(l);
system.debug('LEADS TO UPDATE AFTER ADD: ' + leadsToUpdate);
Lead leadToDel = new Lead(Id=newLead.Id);
leadsToDelete.add(leadToDel);
}
}
}
}
Here's the code of the test method I am using:
@isTest
public static void massiveLeadInsert() {
List<Campaign> campaigns = new List<Campaign>();
List<Lead> leads = new List<Lead>();
Campaign c1 = new Campaign(Name='TalentTest', RecordTypeId=Schema.SObjectType.Campaign.getRecordTypeInfosByName().get('Talent').getRecordTypeId());
Campaign c2 = new Campaign(Name='InboundTest', RecordTypeId=Schema.SObjectType.Campaign.getRecordTypeInfosByName().get('Inbound').getRecordTypeId());
// List<Lead> leadsToupd = new List<Lead>();
campaigns.add(c1);
campaigns.add(c2);
insert campaigns;
for (Integer i = 0; i<500; i++) {
Lead l = new Lead();
l.LastName = 'Test LName' + i;
l.Email = i + '[email protected]';
l.Company = 'WR';
l.Status = 'Open';
l.TalentSource__c = 'TalentTest';
l.InboundCampaign__c = 'InboundTest';
l.DateTalentConsent__c = Date.newInstance(2022, 06, 7);
l.RecordTypeId = Schema.SObjectType.Lead.getRecordTypeInfosByName().get('B2C').getRecordTypeId();
leads.add(l);
}
insert leads;
}
Class HLP_duplicateLeadManagement is basically a static variables container which I am using in order to have access to these variables in both before and after context. This might be something unnecessary as I was using this before switching to a proper trigger framework with handler classes, dispatcher and interface. But still, I don't think that's the reason why I'm hitting governor limits.
public with sharing class HLP_duplicateLeadManagement {
public static Set<String> newNameSet = new Set<String>();
public static Set<String> newEmailSet = new Set<String>();
public static List<Lead> dbLeads = new List<Lead>();
public static List<Contact> dbPersonAccounts = new List<Contact>();
public static Map<String, Campaign> talentCampaignsMAP = new Map<String, Campaign>();
public static Map<Id, Campaign> talentCampaignsMAP2 = new Map<Id, Campaign>();
public static Map<String, Campaign> inboundCampaignsMAP = new Map<String, Campaign>();
public static Map<String, Campaign> inboundCampaignsMAP2 = new Map<String, Campaign>();
}
There might be some old stuff in the comments which isn't actually in the code any longer (like old variable names); just ignore it.
How can I improve this code in order to avoid the "Apex CPU time limit exceeded" error?
Any help is much appreciated!
Best Answer
There's a lot going on in this code, but I'll start with some of the basics.
First, use Test.startTest() before you insert the leads. This gives you more CPU time for you to test with. However, it's not going to do too much, so we'll have to focus elsewhere.
Next, stop leaving debug statements in your code. Every time you debug, it costs CPU time, even if logging is disabled. This is because Salesforce doesn't optimize away the "toString" process that has to occur when you write something like:
This is costing you a lot of CPU time. Even worse, the trigger has to run three times (triggers are max 200 records per transaction), so you're getting hit with these heavy debug statements 3 times.
You can save a ton of time by changing to an sObject constructor:
Takes approximately 2x longer than:
You have further optimizations you can do as well. Instead of:
You can first assign the value ahead of time:
And then assign that:
Even though the platform caches describes, you are still using far more CPU time than you would if you used variables like this to cache results.
This is a good candidate for a map. You're potentially calling the inner loop 40,000 times for every 200 contacts. If I had to guess, this is what's eating up copious amounts of CPU time.
...
The above also applies to how you're scanning every contact for each lead. Use a map:
Pass that in to your dedupPersonAccount method, and it now starts off with:
In addition, you have places where you're checking if a list is empty unnecessarily.
Is precisely identical to:
Same for DML statements:
Is the same as:
No governor limits are held against you if the list is empty.
You have may have other potential problems with this code, but this answer already waxes verbose. You should be able to get a very nice boost in performance simply with replacing the nested loops with maps.