NodeJS Architecture II: V8, Ignition and TurboFan

In the previous post, we briefly discussed the challenges of using JavaScript on the backend and where tools like V8 and Libuv fit into this process. Initially, the browsers needed a common standard so that every JavaScript implementation would yield the same results. Thus, in 1997, Netscape submitted the language specifications to the ECMA International organization, which named the official standard as ECMAScript. This standard is maintained and evolved by TC39, a technical committee within ECMA comprising representatives from companies like Google, Apple, Microsoft, Mozilla, and others.

From then on, several JavaScript engines emerged for browsers, such as SpiderMonkey, JavaScriptCore and V8. Although they all implement the ECMAScript standard, each engine handles code execution in its own unique way, and it is precisely the internal architecture of V8 that we will explore right now.

With focus on optimization, Google created a powerful tool and a robust compilation process with V8, which was launched in the first Chrome version. Among the main optimizations implemented by V8, we can highlight JIT compilation, or Just-in-time compilation. This process implies that JavaScript code compilation takes place at runtime, and to support this process, a robust mechanism was created with V8.

The entry point of this process is the JavaScript source code written by developers. The first step consists of building a structure named AST, or Abstract Syntax Tree, in order to create a semantic structure to identify and highlight each Javascript resource used in the source code.

To dive deeper, we can explore the V8 repository on GitHub. The JavaScript ingestion starts with the initialization of the “Scanner” struct, which iteratively reads the code through the “Scan()” method (def: src/parsing/scanner-inl.h).

During this process, reserved words in the language, such as “function”, “const”, and “class”, are separated from the code, generating building blocks for the AST called tokens. The Scanner instance is created and managed by the “ParseProgram()” function of “Parser” struct (src/parsing/parser.cc) which then delegates all the heavy lifting (that we will see ahead) to the “DoParseProgram()” function. This generic function routes the parsing to specialized functions for each token type found in the code, such as ParseFunction, ParseModuleItemList, ParseImportDeclaration, and others. Within these specialized parsing functions, calls are made to build the AST using the AstNodeFactory class.

The AST structure created during parsing represents how V8 interpreted your JavaScript code, but in a structured and abstract way, ready to be compiled into machine code. To illustrate what this tree looks like, we can analyze a code sample and its corresponding tree. Consider the code below, where the system performs a simple addition:

This small line of code is converted into an AST with at least four components: a variable declarator of “const” type, a binary addition operator and two literal values for numbers 2 and 3 that make up the addition. The AST below represents this line of code:

In the structure we can notice keys present in every node, such as “type”, which represents the type of structure created, and “start” and “end”, which indicate the start and end indexes of a specific compiled block within the original code. Furthermore, we have keys like “value” and “raw”, which contain literal data values, and “name” to identify variables and named structures. There are other keys to define compilation features, but these can be explored by simulating code blocks on AST Explorer (https://astexplorer.net/) and analyzing the results, just as we did above.

Based on the AST, a component added to V8 in 2016 called Ignition was introduced to optimize the JavaScript compilation process, which until then had been an expensive task for the compiler. Unlike the previous approach, instead of compiling directly to machine code, Ignition is responsible for compiling JavaScript code into Bytecode, a type of code consisting of an operation code (opcode) and operands (the data to be processed), where each opcode is one byte in length, allowing the execution of hundreds of operations. In this way, the main impact of Ignition is the optimization of compilation and the reduction of the memory footprint occupied by the final code. Moreover, it provides portability, as Bytecodes act as a universal intermediate representation for different processors, making it possible to generate code for specific architectures such as AMD, ARM, Intel, and others.

From the generated Bytecode, we will have many different instructions, such as variable creation, mathematical operations, loops, and more. In turn, these operations are interpreted by Bytecode Handlers in real time, performing the transformation into executable machine code. Each instruction has its own specialized handler designed for the environment where the code will run, which may vary across browsers for client-side Javascript, or even across operating systems and processor architectures for NodeJS runtime.

Throughout a program’s execution, there are usually functions and loops that run multiple times. For these code blocks, Ignition inserts an “invisible counter” into the Bytecode to track which blocks are often executed. When this counter tracks the number of times a function has run, it is called an “invocation counter”, whereas for counting loop executions, it is called a “back-edge counter”. With this information, V8 can identify what are known as “Hot Functions”.

These frequently used functions go through a different JIT compilation process. After being identified as Hot Functions, they are directly compiled into machine code by another V8 component in parallel, without blocking the main thread. The component responsible for this optimization is called TurboFan.

Thus, the compilation process can be summarized in the following steps:

1. The JavaScript code is read by the Scanner and transformed into an AST by the Parser;
2. Ignition converts the AST into Bytecode instructions in real time (JIT);
3. Frequently used functions are compiled directly into machine code by TurboFan.

JavaScript is a dynamically typed language, which means that types are resolved at runtime. Thus, a variable declared with a numeric value can later become a string or boolean. The problem arises when we apply this concept to the compiler, because every time a parameter type changes, that entire code block must be compiled again. When we combine this concept with TurboFan’s optimizations, we reach a critical point: changing the data type renders the previously generated optimized code unusable. At that moment, the code generated by TurboFan is discarded, and execution falls back to Ignition using Bytecode, until the function is classified as a Hot Function once more and re-optimized by TurboFan. This process is far more costly for the compiler than the optimized path, so we should be careful to avoid this level of dynamism in our code to achieve better performance during execution. This internal compiler process is called “deoptimization” or “bailout”. Consider the code sample below, which demonstrates parameter type changes occurring after a Hot Function optimization by TurboFan:

To identify this type of unwanted event in the code, we can execute the command “node –trace-deopt <path>.js”, which will display the deoptimizations that occurred at runtime. See the example for the code above:

Understanding this optimization process is fundamental to understanding why the type consistency in NodeJS is extremely important. Besides that, it helps us to understand how tools like Typescript, although they can’t completely prevent deoptimization, assist us in maintaining best practices so that compilers can make the most of their resources.

In the next post, we will explore where arrays, objects and other data structures live in V8 memory, and how the Garbage Collector manages this footprint.