Determining if a Data Value Change is a “real” change, in a few different languages

As explored recently, Marketo uses exact string comparison (case-sensitive and accent-sensitive) to determine if fields have changed:

And this is is, overall, A Good Thing. Case changes to first and last names, or accent adjustments like “Rege-Jean” to “Regé-Jean,” are significant[1] and obviously must be saved to the database.

It’s only unnecessary bulk changes of this sort that are a problem because they increase sync latency: one of our clients had an enrichment service that was upper-casing thousands of values every day, only for SFDC to switch back to the original values due to field permissions. What a waste!

Surveying your “real” vs. “less real” changes using the Marketo REST API

The Get Lead Activities and Get Lead Changes endpoints provide the newValue and oldValue for any change. Extracting changes and using a Unicode base character collator will tell you how many “less real” changes are happening in your instance.

Decided to flex my cross-language skills, such as they are, and show how to create & use that collator in JavaScript, Java, PHP, and Python. Each example sets the Boolean isEquiv to true if the values only differ by case or accent.

You can see the code is very similar. Interestingly, JS (my favorite language) uses sensitivity, while the others call it strength; base sensitivity and primary strength mean the same thing.

JavaScript

let baseLetterComparator = Intl.Collator(undefined, { sensitivity: "base" });
let isEquiv = baseLetterComparator.compare(newValue, oldValue) == 0;

Java

import java.text.*;

public class Main {
  public static void main(String[] args) {
    Collator baseLetterComparator = Collator.getInstance();
    baseLetterComparator.setStrength(Collator.PRIMARY);
    boolean $isEquiv = baseLetterComparator.compare(newValue, oldValue) == 0;
  }

}

PHP

$baseLetterComparator = new Collator("");
$baseLetterComparator->setStrength(Collator::PRIMARY);
$isEquiv = $baseLetterComparator->compare(newValue, oldValue) == 0;

Python

(My least favorite language, but I know some of you like it. Note Python is the only one that requires a 3rd-party library, PyICU, to do such a simple thing. Not exactly beating the allegations.😜)

from PyICU import Collator

baseLetterComparator = Collator.createInstance()
baseLetterComparator.setStrength(Collator.PRIMARY)
isEquiv = baseLetterComparator.compare(newValue, oldValue) == 0

Notes

[1] I’m strongly against apps that claim to “fix” proper nouns, since “de la Vega” and “De La Vega” are not the same name.