Determining if a lead is brand new at form submit time

💡

This post? This post right here? This post right here? Right here. This post is probably the strongest ever on the blog.

When someone submits a Marketo form, there’s no built-in way to determine if they’re net new right then, in the browser. This means you lose your chance at directing some leads to a “welcome, new person” site journey and others to a “welcome back” path.

While it’s always been possible to add Net New Detection (NND), the technical recipe hasn’t been written down previously, left in a haze of “I’ll explain it someday” by yours truly.

It has overlaps with my SimpleDTO library used by a lot of folks for cross-domain pre-fill, but doesn’t depend on the SimpleDTO JS and is much simpler overall.

Note that while SimpleDTO works across domains (i.e. with embedded forms), net new detection requires a Marketo LP. In the future, it could be expanded to work cross-domain, but you’ve waited long enough as it is!

Today’s recipe

The recipe has 3 not-so-simple ingredients:

1. Adding an Original Form Submission UUID field to the forms you want to NND-enable and creating the accompanying Smart Campaign, as detailed in this earlier post.

2. Ensuring the form’s Thank You page is a Marketo LP with a special invisible <datalist> element.

3. Polling the Thank You document under the hood until it’s associated with the submitting lead. Once it’s associated, check if this submit’s UUID was just set as the original UUID. If it was, presto! You know the person is net new.

For Flowchart Fans, a.k.a. Trapezoid Noids

1. Add the UUID field

Follow the directions in the earlier post if you haven’t already, including the Smart Campaigns and this JS on the front end:

MktoForms2.whenReady(function(readyForm){
   readyForm.onSubmit(function(submittingForm){
      submittingForm.addHiddenFields({ 
         originalFormSubmissionUUID : self.crypto.randomUUID() 
      })
   });
});

2. Create a Thank You LP

I recommend testing the setup with a new, otherwise blank LP template + LP. Later, you can copy the <datalist> into the production Thank You page. Here’s the simplest possible HTML:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="robots" content="noindex">
  <title>DTP</title>
</head>
<body translate="no">
<datalist id="marketo-tokens">
  <option label="lead.Email Address"><!--email_off-->{{lead.Email Address}}<!--/email_off--></option>
  <option label="lead.Original Source Type">{{lead.Original Source Type}}</option>
  <option label="lead.Original Form Submission UUID">{{lead.Original Form Submission UUID}}</option>
</datalist>
</body>
</html>

As you can see, we’re just embedding three necessary {{lead.tokens}} in an HTML-safe way[1] so they can be reliably read from our code.

3. Get the code

The code has two parts:

a remote <script src> containing the general-purpose polling function pollResponseDocument
a local <script> that calls pollResponseDocument in a Marketo-aware way

Host the Response Document Poller JS

Download Response Document Poller from here:

teknkl-response-document-poller-v2.js

Upload that file to Design Studio and include the <script> in the <head> of your LP.

What Response Document Poller offers

Response Document Poller exposes the function pollResponseDocument, implementing simple logic:

fetch a remote HTML document and run a callback function matcher against the doc
if matcher returns false, fetch the document again after the interval delayMs (default 500ms)
continues fetching until matcher returns trueor the max tries is reached, (default 20)

Marketo domain knowledge + `pollResponseDocument` = Net New Detection

The code below implements NND using pollResponseDocument plus Marketo-aware JS. After form submit, it logs the new/existing result to the console and replaces the button with either...

or...

Customize the new vs. existing behavior any way you want. Go play!

<!-- replace pages.example.com with your LP domain -->
<script src="https://pages.example.com/teknkl-response-document-poller-v2.js"></script>
<script>
  MktoForms2.whenReady(function (readyForm){
   const formEl = readyForm.getFormElem()[0],
              buttonRow = formEl.querySelector(".mktoButtonRow");

   readyForm.onSuccess(function (submittedValues, thankYouHref) {
      const thankYouURL = new URL(thankYouHref);
      thankYouURL.protocol = "https:";
      thankYouURL.searchParams.delete("aliId");       
      thankYouURL.searchParams.set("_mkt_trk", submittedValues._mkt_trk);

      pollResponseDocument({
         log: true,
         url: thankYouURL.href,
         matcher: function (responseDocument) {
            const tokenList = responseDocument.querySelector("datalist#marketo-tokens");

            if (tokenList) {
               const associatedEmailAddress = tokenList.querySelector("option[label='lead.Email Address']").textContent;     
               const originalSourceType = responseDocument.querySelector("option[label='lead.Original Source Type']").textContent;
               const originalUUID = responseDocument.querySelector("option[label='lead.Original Form Submission UUID']").textContent;

               if (associatedEmailAddress === submittedValues["Email"]) {      
                  console.log("Form submission committed to Marketo.")
                  if(originalSourceType == "Web form fillout" && originalUUID == submittedValues["originalFormSubmissionUUID"]){
                    // lead is NEW
                    console.log("Person was created by this form fill (net new).");
                    buttonRow.textContent = "New lead!";
                  } else {
                    // lead was EXISTING
                    console.log("Person already existed.")
                    buttonRow.textContent = "Existing lead!";
                  }
                  return true;
               }
               return false;
            }

            return false;
         },
         onExpiry: function(responseDocument){
            // couldn’t determine for some reason, do fallback stuff
            console.log("Max retries exceeded.");
         }
      });
      return false;
   });
});
</script>

Notes

[1] The “HTML-safe” part has long needed its own post but I haven’t gotten to it.