“Tested in Chrome” is still not enough (*sigh*)

If you've wondered why devs still ask you for “the OS, browser vendor, and browser version” after all these years, when browser features seem so converged: it's cases like this!

Was troubleshooting some fancy-schmancy Forms 2.0 JS using Cross-Browser Testing. I never reproduced the specific misbehavior that had been reported (turned out to be due to the usual form-post-to-database delay), but I did accidentally discover something else while testing across browser versions (CBT is great at providing a wide range of older releases on their boxes).

Observe, in the top row, Chrome 51 and Chrome 52, and in the bottom row, Chrome 53 and Chrome 54. Pictured OS happens to be Mac OSX 10.11, but same results on Windows 8.1 and 10:

Yep, in Chrome 53, which came out in early September 2016 and was for the most part auto-updated off the map by early October 2016, Marketo pods were not available over https://. This also means Marketo forms weren't working when loaded from https:// pages.

Even if you saw the warning screen as a Marketo user, you might not have realized there was a wider problem, since you could open up Firefox or Safari to work around it. But form operation depends on pod operation (the forms /save and /save2 endpoints are on your pod) so it was affecting leads and admins alike.

This was not Marketo's fault in any way. It was a bug in that particular Chrome build in the way it deep-verified SSL certificates (basically, they broke support for Marketo's certificate chain, even though they had it right in Chrome 52 and again in Chrome 54).

Marketo doesn't do anything crazy with SSL, either. It's just the perfect storm of a broken browser build and a major SaaS app. And nothing could've been done to fix it, unless you could get your leads to downgrade (which is pretty unlikely). I guess you could've sniffed the Chrome version and told them there was a browser bug and to use another browser. Too late now, though.

Anyway, that's why we still like to know the version! And also why you should test in prerelease browser versions (Chrome Canary, Firefox Aurora, etc.). Even if it seems impossible to imagine that core functionality would break in mature browsers, it happens.