February’s Brainteaser unteased: Here’s why that bit of Velocity code works instead of failing

Sanford Whiteman -

In the most recent Velociteaser I challenged readers to explain why certain Velocity code works instead of — as a datatype mismatch suggests — throwing a fatal error.

To recap, we had a String field in Marketo with a semicolon-delimited set of values...

... and we  split the String, then checked if it contains an interesting value...

#set( $SportsList = $lead.SportsInterests.split(";") )
#if( $SportsList.contains("beach volleyball") )
If you don't know Karch Kiraly, you don’t know Beach!
#end

... which works fine. But upon further inspection, we shouldn’t have been able to call contains. The result of split is a simple Array of Strings, and Arrays don’t support contains searches, only direct index lookup (array[0], [array[99], etc.)

So why does contains work?

I mentioned (in a note in the comments) that the correct answer would cite a specific Java class — a class that wasn’t mentioned anywhere in the first post.

The answer

The class that holds the answer is ArrayListWrapper, a class that’s used under the hood by Velocity. Using a screenshot of the Velocity Engine source as it’s easier to highlight:

So an ArrayListWrapper is a merely a wrapper object that adds List-y stuff on top of a simple Array (the private variable called lower-case-a  array is the original Array).

As you can see, above, ArrayListWrapper directly implements the methods get, set, size, and length — none of these methods exist on Arrays themselves.

It doesn’t implement the method we’re curious about, contains, but it does inherit from the parent AbstractList. AbstractList inherits from AbstractCollection, and AbstractCollection does have a built-in version of  contains. So that’s why an ArrayListWrapper supports it.

When/why does Velocity decide to put an an Array inside an ArrayListWrapper? It’s simple (don’t get me wrong, this whole thing isn’t simple, but this part is!).

The Velocity tokenizer (an utterly different concept from “tokenizing” which some people use to refer to Marketo {{my.tokens}}, by the way!) splits a template into all of its constituent parts before evaluating it (a.k.a. “compiling it” or “running it”).

For instance, it’ll recognize this sequence of 2 tokens: $reference.method().

Based on the dollar $, dot ., and parentheses (), that’s an attempt to call the method called  method of an object called  $reference.

Here’s what the engine does next, rather than immediately trying to call method():

  • checks to see if the $reference is an Array.
    • if it isn’t an Array, proceeds to call method() (whether that call succeeds or not is off-topic!)
    • if it is an Array, continues with another check
  • checks to see if the ArrayListWrapper has a method called method
    • if ArrayListWrapper doesn’t have method, exits silently
    • if ArrayListWrapper does have method, wraps the Array $reference inside a temporary ArrayListWrapper, then calls the wrapper’s version of method()

Here’s that logic in a flowchart:

Now you know why contains, a method we’d expect to work only on Lists or other Collections, works even on an Array. Via temporary object wrappers, the Velocity engine tries — though it doesn’t always succeed, more on that another day! — to smooth over the differences between types of collection-like objects.

(It does so, by the way, at a relatively heavy price in terms of resource overhead. But you’d never notice the difference in an environment like Marketo.)

The other part

I also mentioned the right answer (i.e. ArrayListWrapper) will also account for another case, the fact that this code doesn’t work:

#set( $void = $SportsList.add("football") )

That’ll throw a fatal error:

org.apache.velocity.exception.MethodInvocationException: 
  Invocation of method 'add' in  class [Ljava.lang.String; 
  threw exception java.lang.UnsupportedOperationException

As you can see in the Velocity source code up top, ArrayListWrapper doesn’t implement the  add method. Like any AbstractList, it could have its own add. But it doesn’t, for good reason: the ArrayListWrapper is backed by the fixed-length Array ($SportsList). That is, the original Array is still the source of the data — it’s not copied, just peeked into. So it would never be possible to implement a method that adds new items, since the Array itself won’t support it.

Instead, AbstractList keeps the default version of add from AbstractCollection, about which the docs are very clear:

This implementation always throws an UnsupportedOperationException.

And that’s the lengthy answer to a remarkably complex question!