Be aware of the *reserved words* in Marketo emails (Part 1 of 2)

Sanford Whiteman -

Betcha didn’t know: Marketo has a small set of reserved words that can’t be used as-is in link URLs, nor in email content in general.

The reserved word list doesn't appear explicitly in the docs,[1] but it can be derived — well, at least the subset I’ll cover in this first of 2 posts — once you understand the environment.

Their very existence is quite obscure (even I don’t tell clients about them right away, not wanting to be pessimistic!). But the danger lingering worry is real.

Here are the 13 essential words for this first round:

#if          #else         #elseif        #foreach      #end
#set         #define       #macro         #include      #parse
#break       #stop         #evaluate

For reasons I’ll get into in Part 2, you’re more likely to have a problem with this initial set than with the other set of words (though the other one can bite you as well).

Who/what makes them reserved?

It pains me to introduce the word “Velocity” here because you might think, “I don’t use Email Script {{my.tokens}}, doesn’t apply to me” and move on.

That would be a mistake.

Yep, the reserved words do come from Velocity. But as I’ve noted before, every Marketo email is assembled, under the hood, using Velocity even if you never write a single line of “userland” VTL code.

So whether you’re a developer or a regular Marketo user, you have an equal need to know about the reserved words. Please keep paying attention, as this does affect you! You should only skip this topic if:

  • you’re a non-developer and
  • you never, ever send URLs with a #hash part, a.k.a. “URL fragment” or “anchor link,” via Marketo (could you ever outright refuse to use a URL, from your web team or a partner?) and
  • you never, ever use the # symbol at all in your emails, even in the body content.

So let’s face it, you have to read on. :)

The reserved #words collide with #hash values

You’ve probably put 1-and-1 together already, realizing that the reserved words all start with # — like how the hash starts with the (first) # in a URL. Yep, that’s the problem.

Take this simple, real-world-possible URL:

https://www.example.com/legal/#end-user-privacy-policy

That URL (sometimes called a “jump link”) would of course open the page https://www.example.com/legal/ and then instantly scroll to the element with id="end-user-privacy-policy".

Try inserting that link in an email, though:[2]

And you’ll get an abrupt validation error on saving:

What’s happened here is clear:

  • the email is parsed as one big Velocity template
  • #end is a special word in Velocity (e.g. the end of a condition block) and it’s expected to be preceded by an #if or #foreach
  • you don’t have a matching start expression (you didn’t even know you were implicitly writing VTL, so this is understandable!)
  • you haven’t done anything to signal that the word #end should not have its usual Velocity-land meaning

Normally (if you weren’t in Marketo, but a more “vanilla” Velocity environment) you would use EscapeTool’s ${esc.h} to easily solve the problem:

https://www.example.com/legal/${esc.h}end-user-privacy-policy

${esc.h} tells the Velocity engine to ignore the usual special meaning of the # character (which is the first character in all Velocity expressions) and print a literal #. Thus the word #end is treated as… just any old word.

But! This method won’t cut it in Marketo. The reason is that Marketo makes multiple (recursive) passes over the email content, each time parsing it as Velocity. Hence the first pass doesn’t see ${esc.h}end as #end, and doesn’t throw the error. But the second pass sees just #end, then tries to evaluate it, and there’s your error. I have found no level of double-, triple-, or something-le-escaping that, er, escapes from the error.

So! Time to open our other bag of tricks.

Think about basic URL structure. We’re used to seeing URLs that are a mixture of:

  • unreserved characters (characters A-Z, a-z, 0-9 and non-special ASCII symbols)
  • well-known symbols (:, /, ?, &, =, #, etc. that are special in specific positions in the URL)
  • percent-encoded sequences (all Unicode codepoints beyond Latin-1, whitespace and control characters, and special-meaning characters when we want to remove their special meaning).

For example, check out this simple URL:

https://www.example.com/my%20page.html?rating=%E2%98%BA

%20 is a percent-encoded space, and %E2%98%BA is a percent-encoded smiley. On a web page or in the browser’s Location bar, the URL might be displayed as https://www.example.com/my page.html?rating=☺️, but the underlying percent-encoding is required for it to be valid.

What you might not realize is that percent-encoding can be used for any unreserved character, even characters that don’t strictly require encoding. So this is a perfectly valid (if exquisitely hard to read) version of the same URL:

https://www.example.com/%6D%79%20%70%61%67%65%2E%68%74%6D%6C?%72%61%74%69%6E%67=%E2%98%BA

Here, the entirety of my page.html – not just the space – is percent-encoded as %6D%79%20%70%61%67%65%2E%68%74%6D%6C. Same with the query string, where rating is encoded even though it could be used as-is. Again, nothing’s wrong with the URL – if the page name were all in a non-Latin-1 language it would have to look like this!

Note that the special characters ? (to start the query string) and = (to separate the query param name from the value) are not encoded, because we need to maintain their special meaning.

We can use optional percent-encoding to solve the problem with #end. Instead of e, use %65. That’s enough to avoid the validation error — we don’t have to percent-encode the whole thing. (And remember from just above, we would never encode the # itself as it needs to stay special in the URL.)

So our final URL is:

https://www.example.com/legal/#%65nd-user-privacy-policy

In the Marketo editor:

This allows the email to be approved, because it doesn’t contain a recognized keyword! Easyish-peasyish.

When the link is clicked, all browsers read the percent-encoded #%65nd correctly, treating it as identical to #end (as mandated by RFC 3986) and scrolling to the corresponding id.

The only difference is that some browsers display the percent-encoded form in the Location bar, while other browsers will decode it back to #end for friendliness. As of December 2019, latest Firefox and Chrome show the friendly #end, while latest Safari, Edge, and IE keep the original #%65nd, as did some older Chromes (~Chrome 50).

Percent-encoded reserved words save the day

Here are the percent-encoded forms of the 13 reserved words, in the same order as above, so you don’t have to figure them out yourself (though that’d be a nice, very minor learning moment, so don’t be afraid to try):

#%69f        #%65lse       #%65lseif      #%66oreach    #%65nd
#%73et       #%64efine     #%6dacro       #%69nclude    #%70arse    
#%62reak     #%73top       #%65valuate

Sounds good for URLs, but didn’t you say, “… nor in email content in general” up top?

Right.

Percent-encoding only fixes the conflict between a reserved #word and an identical-looking URL #hash part.

It doesn’t solve the problem of using a reserved #word in the body content of an email. For example, this will throw the same fatal validation error:

The good news: it’s easier to solve in displayed HTML content than in a URL. HTML supports an invisible Unicode character you can put just after the #: the Word Joiner character, ⁠ or ⁠.[3]

Just pop a Word Joiner in there and the sequence won’t be confused for a keyword anymore, but will display as usual:

In case it’s not clear from the screenshot, that final text is:

all the way to the #⁠end.

That is, the # to be displayed, then the (decimal) entity representing the Word Joiner, ⁠, then the rest of the displayed content end.

Note: In a somewhat squirm-inducing fashion, Marketo no longer uses the Word Joiner entity code after you save the HTML – I mean in the ostensibly “raw” HTML view. The Word Joiner character is injected instead: you can tell by the extra stop when you left- and right-arrow through the code. And if you remove the whole HTML you’ll be back to the error.

What about Text content?

If you have a multipart HTML/Text email (the Marketo default) and “Automatically copy from HTML” is checked, the Word Joiner character should be correctly transferred to the Text section. Let me know if you find exceptions.

I plan to discuss Text-Only and custom multipart Text more in in Part 2.

Stop reading here if you’re not a developer

After this, it’s developer-only.

What about Velocimacros? Don’t they start with #?

Yes. In truth, there aren’t just 13 reserved words – there are unlimited reserved words, if you’re writing custom code (Velocity/Email Script {{my.tokens}}). But I was trying not to be too terrifying, c’mon. :)

For example, merely writing this macro…

#macro( myLittleFunction )
## do something in here
#end

… creates the new reserved word #myLittleFunction, since that’s how you’d call the macro! So if you’re writing macros (and you should, they’re extreme timesavers) make sure you’re ready to use percent-encoding and/or the Word Joiner as necessary.

What about $references?

Yep. There are a set of system-reserved $references too, in addition to any custom Velocity variables you #set. Sigh… that’s what Part 2 is slated to be about.

Notes

[1] Marketo probably errs on the side of probability here, which I wouldn’t have done. To each their own.

[2] Same goes for manually building the <a> in HTML.

[3] Using the decimal entity &#8288; was a deliberate choice. Can you sense why I didn’t use the hex entity?