Your choice: Scylla, Charybdis, or returning 𝑛𝑢𝑙𝑙 in Velocity

Velocity’s quirky treatment of null could drive a mortal mad.

As a language aimed at web developers rather than “real” developers, VTL attempts to smooth over Java’s strict typing: it allows comparison across types[1] and blends false and null into a form of falsiness. We can work with or around those features.

But it gets weirder: in versions including Marketo’s install[2], Velocity also prohibits creating new null references at runtime. There’s no real workaround for this one; ironically, to avoid related bugs, you must follow poor programming practices!

Take a person with a single Opportunity:

(Whitespace around the LodgingType and City is deliberate. And null is really null, not an empty string.)

Here’s the raw output of ${OpportunityList}:

{LodgingType=commercial hotel, Host=  Zaan Hotel  , PointsProgram=null, City= Amsterdam , Meals=false}

Knowing your CRM data is messy — that’s how the whitespace crept in — you decide to trim all Strings using the regex from my recent post on trimming:

#foreach( $oppty in $OpportunityList )
  #foreach( $entry in $oppty.entrySet() )
    #set( $key = $entry.getKey() )
    #set( $val = $entry.getValue() )
    #if( $val.class.name.equals("java.lang.String") )
      #set( $oppty[$key] = $val.replaceAll("^[\h\v]+|[\h\v]+${esc.d}","") )
    #end
  #end
#end

Before clicking View Output below, try to guess what the new output of ${OpportunityList} will be. Got an idea? OK, now click:

{LodgingType=commercial hotel, Host=Zaan Hotel, PointsProgram=Zaan Hotel, City=Amsterdam, Meals=false}

Wha??? Trimming worked, but PointsProgram is now "Zaan Hotel" — same as Host. That ain’t right and will vastly screw up any further logic.

The culprit: being too professional

Look at line 4 above:

#set( $val = $entry.getValue() )

A seemingly innocent assignment of the current Map value to an intermediate variable $val. If the current value is null, then $val will be null, right? Nope! You cannot assign (nor reassign) null to a reference. The attempt is silently ignored and $val keeps its previous value: the last non-null value in the loop, " Zaan Hotel "! (Which then gets trimmed.)

In sum, while Velocity honors existing nulls[3], it hates new nulls. This makes it harder to write readable, efficient code. If you do the unprofessional thing for readability and resource utilization[4] and skip the intermediate variable $val:

#foreach( $oppty in $OpportunityList )
  #foreach( $entry in $oppty.entrySet() )
    #set( $key = $entry.getKey() )
    #if( $entry.getValue().class.name.equals("java.lang.String") )
      #set( $oppty[$key] = $entry.getValue().replaceAll("^[\h\v]+|[\h\v]+${esc.d}","") )
    #end
  #end
#end

You get the correct result:

{LodgingType=commercial hotel, Host=Zaan Hotel, PointsProgram=null, City=Amsterdam, Meals=false}

Pretty crazy, right?

There’s a tiny bit more

Sufficiently artificially-stimulated readers will notice there’s still a bug. While you wouldn’t naturally[5] encounter this in a Marketo context, Java allows LinkedHashMaps to have a single null key, not just null values![6] Velocity will faithfully preserve this existing null, but will bug out if you try to assign $key to it.

Java-safe VTL thus avoids both #set directives:

#foreach( $oppty in $OpportunityList )
  #foreach( $entry in $oppty.entrySet() )
    #if( $entry.getValue().class.name.equals("java.lang.String") )
      #set( $oppty[$entry.getKey()] = $entry.getValue().replaceAll("^[\h\v]+|[\h\v]+${esc.d}","") )
    #end
  #end
#end

Ugh. If you didn’t know better, you’d think this was junior-authored code. But it’s only junior-style to cooperate with Velocity’s quirks.

Notes

[1] That is, types lacking an explicit compareTo. As I’ve noted on many community threads, the automatic type coercion with == creates hard-to-find errors and it’s better to use equals() even if the engine gives you something seemingly easier.

[2] This behavior was removed in the next Velocity release (2.0) after Marketo’s current version (1.7) because it did more harm than good. Unfortunately, if Marketo were to upgrade, any tokens that purposely or coincidentally rely on this behavior would break! Hard to upgrade with so many userland Velocity “cooks,” most of whom — just being honest — wouldn’t even know how to audit their code to find such dependencies.

[3] The Velocity context is agnostic about objects & datatypes provided by the hosting Java VM. This is good! It means you can work with complex data structures, custom/child classes, etc. from VTL and null is totally allowed. But prior to Velocity 2.0, they really, really didn’t want you to create new null $references.

[4] Wouldn’t be noticeable until you did it thousands of times, but dereferencing $entry.getValue() over and over is less efficient than deref’ing once and storing the result.

[5] In Marketo, neither Person field names, Opportunity field names, nor CO field names can be null because the app doesn’t allow it. You can, however, purposely put() a null key into a LinkedHashMap (even including the $lead object, since it’s mutable). put() is different from #set-ing a reference!

[6] N.B. a JavaScript Map also allows a single null key, and a single undefined key too.