JavaScript for Line of Business Applications

However, in the real world you’ll find that most lexers (particularly lexers for real programming languages) are not written that way. They are carefully hand-written to go over the input and dispatch to the appropriate token handling code depending on the first character encountered.

This technique makes more sense for complex languages because it’s much more flexible than regexes (for instance, representing nested comments with regexes is a big challenge). But I was curious also about its performance implications.

So I hacked together a hand-written lexer for exactly the same language used in the previous benchmark and also exercised it on the same large file to make sure the results are correct. Its runtime, however, surprised me. Here’s the runtime of lexing a large-ish document (smaller is better):

Regular expressions in JavaScript may not always be intuitive. I aim to provide some clarity by providing examples about things I have found myself getting stuck on. This post covers a few topics including state in regular expressions, regular expression performance and various things that have tripped me up.

I have had a love/hate relationship with regular expressions in the past. Reading or writing a regular expression typically made me feel like I was toying with a broken Rubiks Cube. However, after I would fiddle an expression into submission, almost by accident, and it did its job, I would become enamored with its brevity and power.