pdf/audit_results.json

[
  {
    "document": "mozilla_pdf_spec_excerpt.pdf",
    "target": "docx",
    "input_bytes": 1016315,
    "source_pages": 14,
    "source_chars": 82701,
    "source_words": 13452,
    "source_images": 90,
    "elapsed_s": 6.5383,
    "cpu_s": 8.2656,
    "cpu_to_wall": 1.264,
    "rss_start_mb": 83.75,
    "rss_peak_mb": 203.77,
    "output_bytes": 76891,
    "fidelity": "lossy",
    "quality": 0.93886875,
    "media": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
    "output": {
      "text": "Trace-based Just-in-Time Type Specialization for Dynamic\nLanguages\nAndreas Gal∗+, Brendan Eich∗, Mike Shaver∗, David Anderson∗, David Mandelin∗,\nMohammad R. Haghighat$, Blake Kaplan∗, Graydon Hoare∗, Boris Zbarsky∗, Jason Orendorff∗,\nJesse Ruderman∗, Edwin Smith#, Rick Reitmaier#, Michael Bebenita+, Mason Chang+#, Michael Franz+\nMozilla Corporation∗\n{gal,brendan,shaver,danderson,dmandelin,mrbkap,graydon,bz,jorendorff,jruderman}@mozilla.com\nAdobe Corporation#\n{edwsmith,rreitmai}@adobe.com\nIntel Corporation$ {mohammad.r.haghighat}@intel.com\nUniversity of California, Irvine+\n{mbebenit,changm,franz}@uci.edu\nAbstract\nDynamic languages such as JavaScript are more difficult to com pile than statically typed ones. Since no concrete type information is available, traditional compilers need to emit generic code that can handle all possible type combinations at runtime. We present an al ternative compilation technique for dynamically-typed languages that identifies frequently executed loop traces at run-time and then generates machine code on the fly that is specialized for the ac tual dynamic types occurring on each path through the loop. Our method provides cheap inter-procedural type specialization, and an elegant and efficient way of incrementally compiling lazily discov ered alternative paths through nested loops. We have implemented a dynamic compiler for JavaScript based on our technique and we have measured speedups of 10x and more for certain benchmark programs.\nand is used for the application logic of browser-based productivity applications such as Google Mail, Google Docs and Zimbra Col laboration Suite. In this domain, in order to provide a fluid user experience and enable a new generation of applications, virtual ma chines must provide a low startup time and high performance.\nCompilers for statically typed languages rely on type informa tion to generate efficient machine code. In a dynamically typed pro gramming language such as JavaScript, the types of expressions may vary at runtime. This means that the compiler can no longer easily transform operations into machine instructions that operate on one specific type. Without exact type information, the compiler must emit slower generalized machine code that can deal with all potential type combinations. While compile-time static type infer ence might be able to gather type information to generate opti mized machine code, traditional static analysis is very expensive and hence not well suited for the highly interactive environment of a web browser.\nCategoriesa nd SubjectD escriptors D.3.4 [ProgrammingL an guages]: Processors— Incremental compilers, code generation.\nWe present a trace-based compilation technique for dynamic languages that reconciles speed of compilation with excellent per formance of the generated machine code. Our system uses a mixed mode execution approach: the system starts running JavaScript in a fast-starting bytecode interpreter. As the program runs, the system identifies hot (frequently executed) bytecode sequences, records them, and compiles them to fast native code. We call such a se quence of instructions a trace.\nGeneral Terms Design, Experimentation, Measurement, Perfor mance.\nKeywords JavaScript,j ust-in-time compilation, trace trees.\n1. Introduction\nDynamic languages such as JavaScript, Python, and Ruby, are pop ular since they are expressive, accessible to non-experts, and make deployment as easy as distributing a source file. They are used for small scripts as well as for complex applications. JavaScript, for example, is the de facto standard for client-side web programming\nUnlike method-based dynamic compilers, our dynamic com piler operates at the granularity of individual loops. This design choice is based on the expectation that programs spend most of their time in hot loops. Even in dynamically typed languages, we expect hot loops to be mostly type-stable, meaning that the types of values are invariant. (12) For example, we would expect loop coun ters that start as integers to remain integers for all iterations. When both of these expectations hold, a trace-based compiler can cover the program execution with a small number of type-specialized, ef ficiently compiled traces.\nPermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.\nEach compiled trace covers one path through the program with one mapping of values to types. When the VM executes a compiled trace, it cannot guarantee that the same path will be followed or that the same types will occur in subsequent loop iterations.\nPLDI’09, June 15–20, 2009, Dublin, Ireland.\nCopyright c 2009 ACM 978-1-60558-392-1/09/06. . . $5.00\nHence, recording and compiling a trace speculates that the path and 1 for (var i = 2; i < 100; ++i) {\ntyping will be exactly as they were during recording for subsequent 2 if (!primes[i])\niterations of the loop. 3 continue;\nEvery compiled trace contains all the guards (checks) required 4 for (var k = i + i; i < 100; k += i)\nto validate the speculation. If one of the guards fails (if control 5 primes[k] = false;\nv0 := ld state[748]\nflow is different, or a value of a different type is generated), the trace exits. If an exit becomes hot, the VM can record a branch trace starting at the exit to cover the new path. In this way, the VM records a trace tree covering all the hot paths through the loop.\nFigure 1. Sample program: sieve of Eratosthenes. primes is initialized to an array of 100 false values on entry to this code snippet.\nN¨ested loops can be difficult to optimize for tracing VMs. In a naıve implementation, inner loops would become hot first, and the VM would start tracing there. When the inner loop exits, the VM would detect that a different branch was taken. The VM would try to record a branch trace, and find that the trace reaches not the inner loop header, but the outer loop header. At this point, the VM could continue tracing until it reaches the inner loop header again, thus tracing the outer loop inside a trace tree for the inner loop. But this requires tracing a copy of the outer loop for every side exit and type combination in the inner loop. In essence, this is a form of unintended tail duplication, which can easily overflow the code cache. Alternatively, the VM could simply stop tracing, and give up on ever tracing outer loops.\nWe solve the nested loop problem by recording n¨ested trace trees. Our system traces the inner loop exactly as the naıve version. The system stops extending the inner tree when it reaches an outer loop, but then it starts a new trace at the outer loop header. When the outer loop reaches the inner loop header, the system tries to call the trace tree for the inner loop. If the call succeeds, the VM records the call to the inner tree as part of the outer trace and finishes the outer trace as normal. In this way, our system can trace any number of loops nested to any depth without causing excessive tail duplication.\nSymbol Key BIntterpret\nOverhead yecodes\nelodop \nge\nInterpreting\ncoldlo/bolpa/cekxliistted Native nitor\nMo\nrecaobrodritn g\ncompreileadd ytrace \nloohpo/etxit\nmEilnte rT\nLRIRe cTorarcde\nCoped race loofipn isheh aadt er\nlosoapm eed gtyep wesith \nLCIRo mTrpaiclee\nComExielecu Tterac\np d e\nno esxidiset inegx ittr,ace\nexsiidstein egx titr atoc e mLeave\nCopiled Trace\nFigure 2. State machine describing the major activities of Trace Monkey and the conditions that cause transitions to a new activ ity. In the dark box, TM executes JS as compiled traces. In the light gray boxes, TM executes JS in the standard interpreter. White boxes are overhead. Thus, to maximize performance, we need to maximize time spent in the darkest box and minimize time spent in the white boxes. The best case is a loop where the types at the loop edge are the same as the types on entry–then TM can stay in native code until the loop is done.\nThese techniques allow a VM to dynamically translate a pro gram to nested, type-specialized trace trees. Because traces can cross function call boundaries, our techniques also achieve the ef fects of inlining. Because traces have no internal control-flowj oins, they can be optimized in linear time by a simple compiler (10). Thus, our tracing VM efficiently performs the same kind of op timizations that would require interprocedural analysis in a static optimization setting. This makes tracing an attractive and effective tool to type specialize even complex function call-rich code.\nWe implemented these techniques for an existing JavaScript in terpreter, SpiderMonkey. We call the resulting tracing VM Trace Monkey. TraceMonkey supports all the JavaScript features of Spi derMonkey, with a 2x-20x speedup for traceable programs.\na set of industry benchmarks. The paper ends with conclusions in Section 9 and an outlook on future work is presented in Section 10.\nThis paper makes the following contributions:\n2. Overview: Example Tracing Run\n• We explain an algorithm for dynamically forming trace trees to\nThis section provides an overview of our system by describing how TraceMonkey executes an example program. The example program, shown in Figure 1, computes the first 100 prime numbers with nested loops. The narrative should be read along with Figure 2, which describes the activities TraceMonkey performs and when it transitions between the loops.\ncover a program, representing nested loops as nested trace trees.\n• We explain how to speculatively generate efficient type-specialized\ncode for traces from dynamic language programs.\n• We validate our tracing techniques in an implementation based\non the SpiderMonkey JavaScript interpreter, achieving 2x-20x speedups on many programs.\nT\nraceMonkey always begins executing a program in the byte code interpreter. Every loop back edge is a potential trace point. When the interpreter crosses a loop edge, TraceMonkey invokes the trace monitor, which may decide to record or execute a native trace. At the start of execution, there are no compiled traces yet, so the trace monitor counts the number of times each loop back edge is executed until a loop becomes hot, currently after 2 crossings. Note that the way our loops are compiled, the loop edge is crossed before entering the loop, so the second crossing occurs immediately after the first iteration.\nThe remainder of this paper is organized as follows. Section 3 is a general overview of trace tree based compilation we use to cap ture and compile frequently executed code regions. In Section 4 we describe our approach of covering nested loops using a num ber of individual trace trees. In Section 5 we describe our trace compilation based speculative type specialization approach we use to generate efficient machine code from recorded bytecode traces. Our implementation of a dynamic type-specializing compiler for JavaScript is described in Section 6. Related work is discussed in Section 8. In Section 7 we evaluate our dynamic compiler based on\nHere is the sequence of events broken down by outer loop iteration:\n// load primes from the trace activation record // store primes to interpreter stack // load k from the trace activation record // convert k from int to double // store k to interpreter stack // store false to interpreter stack // load class word for primes\nst sp[0], v0 v1 := ld state[764]\nv2 := i2f(v1)\nst sp[8], v1 st sp[16], 0 v3 := ld v0[4]\nv4 := and v3, -4\n// m ask out object class tag for primes // test whether primes is an array // side exit if v5 is false\nv5 := eq v4, Array\nxf v5\nv6 : js_Array_set(v0, v2, false) // call function to set array element v7 := eq v6, 0\n // test return value from call\nxt v7\n // side exit if js_Array_set returns false.\nFigure 3. LIR snippet for sample program. This is the LIR recorded for line 5 of the sample program in Figure 1. The LIR encodes the semantics in SSA form using temporary variables. The LIR also encodes all the stores that the interpreter would do to its data stack. Sometimes these stores can be optimized away as the stack locations are live only on exits to the interpreter. Finally, the LIR records guards and side exits to verify the assumptions made in this recording: that primes is an array and that the call to set its element succeeds.\n// load primes from the trace activation record // (*) store primes to interpreter stack // load k from the trace activation record // (*) store k to interpreter stack // (*) store false to interpreter stack // (*) load object class word for primes // (*) m ask out object class tag for primes // (*) test whether primes is an array // (*) side exit if primes is not an array // bump stack for call alignment convention // push last argument for call\nmov edx, ebx(748)\nmov edi(0), edx\nmov esi, ebx(764)\nmov edi(8), esi\nmov edi(16), 0\nmov eax, edx(4)\nand eax, -4\ncmp eax, Array\njne side_exit_1\nsub esp, 8\npush false\n // push first argument for call\npush esi\n// call function to set array element // clean up extra stack space\ncall js_Array_set\nadd esp, 8\nmov ecx, ebx\n// (*) created by register allocator // (*) test return value of js_Array_set // (*) side exit if call failed\ntest eax, eax\nje side_exit_2\nside_exit_1:\n // restore ecx\nmov ecx, ebp(-4)\n // restore esp\nmov esp, ebp\n // jump to ret statement\njmp epilog\nFigure 4. x86 snippet for sample program. This is the x86 code compiled from the LIR snippet in Figure 3. Most LIR instructions compile to a single x86 instruction. Instructions marked with (*) would be omitted by an idealized compiler that knew that none of the side exits would ever be taken. The 17 instructions generated by the compiler compare favorably with the 100+ instructions that the interpreter would\nexecute for the same code snippet, including 4 indirectj umps.\ni=2. This is the first iteration of the outer loop. The loop on lines 4-5 becomes hot on its second iteration, so TraceMonkey en ters recording mode on line 4. In recording mode, TraceMonkey records the code along the trace in a low-level compiler intermedi ate representation we call LIR. The LIR trace encodes all the oper ations performed and the types of all operands. The LIR trace also encodes guards, which are checks that verify that the control flow and types are identical to those observed during trace recording. Thus, on later executions, if and only if all guards are passed, the trace has the required program semantics.\ninterpreter PC and the types of values match those observed when trace recording was started. The first trace in our example, T45, covers lines 4 and 5. This trace can be entered if the PC is at line 4, i and k are integers, and primes is an object. After compiling T45, TraceMonkey returns to the interpreter and loops back to line 1.\ni=3. Now the loop header at line 1 has become hot, so Trace Monkey starts recording. When recording reaches line 4, Trace Monkey observes that it has reached an inner loop header that al ready has a compiled trace, so TraceMonkey attempts to nest the inner loop inside the current trace. The first step is to call the inner trace as a subroutine. This executes the loop on line 4 to completion and then returns to the recorder. TraceMonkey verifies that the call was successful and then records the call to the inner trace as part of the current trace. Recording continues until execution reaches line 1, and at which point TraceMonkey finishes and compiles a trace for the outer loop, T16.\nTraceMonkey stops recording when execution returns to the loop header or exits the loop. In this case, execution returns to the loop header on line 4.\nAfter recording is finished, TraceMonkey compiles the trace to native code using the recorded type information for optimization. The result is a native code fragment that can be entered if the\nA trace records all its intermediate values in a small activation i=4. On this iteration, TraceMonkey calls T16. Because i=4, the if statement on line 2 is taken. This branch was not taken in the\nrecord area. To make variable accesses fast on trace, the trace also original trace, so this causes T16 to fail a guard and take a side exit.\nimports local and global variables by unboxing them and copying them to its activation record. Thus, the trace can read and write The exit is not yet hot, so TraceMonkey returns to the interpreter, which executes the continue statement.\nthese variables with simple loads and stores from a native activation i=5. TraceMonkey calls T16, which in turn calls the nested trace\nrecording, independently of the boxing mechanism used by the interpreter. When the trace exits, the VM boxes the values from T45. T16 loops back to its own header, starting the next iteration without ever returning to the monitor.\nthis native storage location and copies them back to the interpreter i=6. On this iteration, the side exit on line 2 is taken again. This\nstructures.\nF\nor every controlflow branch in the source program, the time, the side exit becomes hot, so a trace T23,1 is recorded that\nrecorder generates conditional exit LIR instructions. These instruc covers line 3 and returns to the loop header. Thus, the end of T23,1\ntions exit from the trace if required control flow is different from jumps directly to the start of T16. The side exit is patched so that\nwhat it was at trace recording, ensuring that the trace instructions on future iterations, itj umps directly to T23,1.\nare run only if they are supposed to. We call these instructions At this point, TraceMonkey has compiled enough traces to cover\nguard instructions.\nthe entire nested loop structure, so the rest of the program runs entirely as native code.\nMost of our traces represent loops and end with the special loop LIR instruction. This isj ust an unconditional branch to the top of the trace. Such traces return only via guards.\n3. Trace Trees\nNow, we describe the key optimizations that are performed as part of recording LIR. All of these optimizations reduce complex In this section, we describe traces, trace trees, and how they are\ndynamic language constructs to simple typed constructs by spe formed at run time. Although our techniques apply to any dynamic\ncializing for the current trace. Each optimization requires guard in language interpreter, we will describe them assuming a bytecode\nstructions to verify their assumptions about the state and exit the interpreter to keep the exposition simple.\ntrace if necessary.\nType specialization.\n3.1 Traces\nAll LIR primitives apply to operands of specific types. Thus, A trace is simply a program path, which may cross function call\nL\nIR traces are necessarily typespecialized, and a compiler can boundaries. TraceMonkey focuses on loop traces, that originate at\neasily produce a translation that requires no type dispatches. A a loop edge and represent a single iteration through the associated\ntypical bytecode interpreter carries tag bits along with each value, loop.\nand to perform any operation, must check the tag bits, dynamically Similar to an extended basic block, a trace is only entered at\ndispatch, mask out the tag bits to recover the untagged value, the top, but may have many exits. In contrast to an extended basic\nperform the operation, and then reapply tags. LIR omits everything block, a trace can containj oin nodes. Since a trace always only\nexcept the operation itself.\nfollows one single path through the original program, however,j oin\nA potential problem is that some operations can produce values nodes are not recognizable as such in a trace and have a single\nof unpredictable types. For example, reading a property from an predecessor node like regular nodes.\nobject could yield a value of any type, not necessarily the type A typed trace is a trace annotated with a type for every variable\nobserved during recording. The recorder emits guard instructions (including temporaries) on the trace. A typed trace also has an entry\nthat conditionally exit if the operation yields a value of a different type map giving the required types for variables used on the trace\ntype from that seen during recording. These guard instructions before they are defined. For example, a trace could have a type map\nguarantee that as long as execution is on trace, the types of values (x: int, b: boolean), meaning that the trace may be entered\nmatch those of the typed trace. When the VM observes a side exit\nonly if the value of the variable x is of type int and the value of b\nalong such a type guard, a new typed trace is recorded originating is of type boolean. The entry type map is much like the signature\nat the side exit location, capturing the new type of the operation in of a function.\nquestion.\nIn this paper, we only“ discu”ss typed loop traces, and we will\nRepresentation specialization: objects. In JavaScript, name refer to them simply as traces . The key property of typed loop\nlookup semantics are complex and potentially expensive because traces is that they can be compiled to efficient machine code using\nthey include features like object inheritance and eval. To evaluate the same techniques used for typed languages.\nan object property read expression like o.x, the interpreter must In TraceMonkey, traces are recorded in trace-flavored SSA LIR\nsearch the property map of o and all of its prototypes and parents. (low-level intermediate representation). In trace-flavored SSA (or\nProperty maps can be implemented with different data structures TSSA), phi nodes appear only at the entry point, which is reached\n(e.g., per-object hash tables or shared hash tables), so the search both on entry and via loop edges. The important LIR primitives\nprocess also must dispatch on the representation of each object are constant values, memory loads and stores (by address and\nfound during search. TraceMonkey can simply observe the result of offset), integer operators, floating-point operators, function calls,\nthe search process and record the simplest possible LIR to access and conditional exits. Type conversions, such as integer to double,\nthe property value. For example, the search might finds the value of are represented by function calls. This makes the LIR used by\no.x in the prototype of o, which uses a shared hash-table represen TraceMonkey independent of the concrete type system and type\ntation that places x in slot 2 of a property vector. Then the recorded conversion rules of the source language. The LIR operations are\ncan generate LIR that reads o.x withj ust two or three loads: one to generic enough that the backend compiler is language independent.\nget the prototype, possibly one to get the property value vector, and Figure 3 shows an example LIR trace.\none more to get slot 2 from the vector. This is a vast simplification Bytecode interpreters typically represent values in a various\nand speedup compared to the original interpreter code. Inheritance complex data structures (e.g., hash tables) in a boxed format (i.e.,\nrelationships and object representations can change during execu with attached type tag bits). Since a trace is intended to represent\ntion, so the simplified code requires guard instructions that en’sure efficient code that eliminates all that complexity, our traces oper\nthe object representation is the same. In TraceMonkey, objects rep- ate on unboxed values in simple variables and arrays as much as possible.\nStarting a tree. Tree trees always start at loop headers, because they are a natural place to look for hot paths. In TraceMonkey, loop headers are easy to detect–the bytecode compiler ensures that a bytecode is a loop header iff it is the target of a backward branch. TraceMonkey starts a tree when a given loop header has been exe cuted a certain number of times (2 in the current implementation). Starting a treej ust means starting recording a trace for the current point and type map and marking the trace as the root of a tree. Each tree is associated with a loop header and type map, so there may be several trees for a given loop header.\nresentations are assigned an integer key called the object shape. Thus, the guard is a simple equality check on the object shape.\nRepresentation specialization: numbers. JavaScript has no integer type, only a Number typ“e that is” the set of 64-bit IEEE 754 floating-pointer numbers ( doubles ). But many JavaScript operators, in particular array accesses and bitwise operators, really operate on integers, so they first convert the number to an integer, and then convert any integer result back to a double.1 Clearly, a JavaScript VM that wants to be fast must find a way to operate on integers directly and avoid these conversions.\nClosing the loop. Trace recording can end in several ways. Ideally, the trace reaches the loop header where it started with the same type map as on entry. This is called a type-stable loop iteration. In this case, the end of the trace canj ump right to the beginning, as all the value representations are exactly as needed to enter the trace. Thej ump can even skip the usual code that would copy out the state at the end of the trace and copy it back in to the trace activation record to enter a trace.\nIn TraceMonkey, we support two representations for numbers: integers and doubles. The interpreter uses integer representations as much as it can, switching for results that can only be represented as doubles. When a trace is started, some values may be imported and represented as integers. Some operations on integers require guards. For example, adding two integers can produce a value too large for the integer representation.\nFunction inlining. LIR traces can cross function boundaries in either direction, achieving function inlining. Move instructions need to be recorded for function entry and exit to copy arguments in and return values out. These move statements are then optimized away by the compiler using copy propagation. In order to be able to return to the interpreter, the trace must also generate LIR to record that a call frame has been entered and exited. The frame entry and exit LIR savesj ust enough information to allow the intepreter call’ stack to be restored later and is much simpler than the interpreters standard call code. If the function being entered is not constant (which in JavaScript includes any call by function name), the recorder must also emit LIR to guard that the function is the same.\nIn certain cases the trace might reach the loop header with a different type map. This scenario is sometime observed for the first iteration of a loop. Some variables inside the loop might initially be undefined, before they are set to a concrete type during the first loop iteration. When recording such an iteration, the recorder cannot link the trace back to its own loop header since it is type-unstable. Instead, the iteration is terminated with a side exit that will always fail and return to the interpreter. At the same time a new trace is recorded with the new type map. Every time an additional type unstable trace is added to a region, its exit type map is compared to the entry map of all existing traces in case they complement each other. With this approach we are able to cover type-unstable loop iterations as long they eventually form a stable equilibrium.\nGuards and side exits. Each optimization described above requires one or more guards to verify the assumptions made in doing the optimization. A guard isj ust a group of LIR instructions that performs a test and conditional exit. The exit branches to a side exit, a small off-trace piece of LIR that returns a pointer to a structure that describes the reason for the exit along with the interpreter PC’ at the exit point and any other data needed to restore the interpreters state structures.\nFinally, the trace might exit the loop before reaching the loop header, for example because execution reaches a break or return statement. In this case, the VM simply ends the trace with an exit to the trace monitor.\nAs mentioned previously, we may speculatively chose to rep resent certain Number-typed values as integers on trace. We do so when we observe that Number-typed variables contain an integer value at trace entry. If during trace recording the variable is unex pectedly assigned a non-integer value, we have to widen the type of the variable to a double. As a result, the recorded trace becomes inherently type-unstable since it starts with an integer value but ends with a double value. This represents a mis-speculation, since at trace entry we specialized the Number-typed value to an integer, assuming that at the loop edge we would again find an integer value in the variable, allowing us to close the loop. To avoid future spec ulative failures involving this variable, and to obtain a type-stable trace we note the fact that the variable in question as been observed to sometimes hold“ non-in”teger values in an advisory data structure which we call the oracle .\nAborts. Some constructs are difficult to record in LIR traces. For example, eval or calls to external functions can change the program state in unpredictable ways, making it difficult for the tracer to know the current type map in order to continue tracing. A tracing implementation can also have any number of other limi tations, e.g.,a small-memory device may limit the length of traces. When any situation occurs that prevents the implementation from continuing trace recording, the implementation aborts trace record ing and returns to the trace monitor.\n3.2 Trace Trees\nEspecially simple loops, namely those where control flow, value types, value representations, and inlined functions are all invariant, can be represented by a single trace. But most loops have at least some variation, and so the program will take side exits from the main trace. When a side exit becomes hot, TraceMonkey starts a new branch trace from that point and patches the side exit toj ump directly to that trace. In this way, a single trace expands on demand to a single-entry, multiple-exit trace tree.\nWhen compiling loops, we consult the oracle before specializ ing values to integers. Speculation towards integers is performed only if no adverse information is known to the oracle about that particular variable. Whenever we accidentally compile a loop that is type-unstable due to mis-speculation of a Number-typed vari able, we immediately trigger the recording of a new trace, which based on the now updated oracle information will start with a dou ble value and thus become type stable.\nThis section explains how trace trees are formed during execu tion. The goal is to form trace trees during execution that cover all the hot paths of the program.\nExtending a tree. Side exits lead to different paths through the loop, or paths with different types or representations. Thus, to completely cover the loop, the VM must record traces starting at all side exits. These traces are recorded much like root traces: there is a counter for each side exit, and when the counter reaches a hotness threshold, recording starts. Recording stops exactly as for the root trace, using the loop header of the root trace as the target to reach.\n1 Arrays are actually worse than this: if the index value is a number, it must\nbe converted from a double to a string for the property access operator, and then to an integer internally to the array implementation.\nOur implementation does not extend at all side exits. It extends only if the side exit is for a control-flow branch, and only if the side exit does not leave the loop. In particular we do not want to extend a trace tree along a path that leads to an outer loop, because we want to cover such paths in an outer tree through tree nesting.\n3.3 Blacklisting\nSometimes, a program follows a path that cannot be compiled into a trace, usually because of limitations in the implementation. TraceMonkey does not currently support recording throwing and catching of arbitrary exceptions. This design trade off was chosen, because exceptions are usually rare in JavaScript. However, if a program opts to use exceptions intensively, we would suddenly incur a punishing runtime overhead if we repeatedly try to record a trace for this path and repeatedly fail to do so, since we abort tracing every time we observe an exception being thrown.\nT\nTree Anchor\nTrunk Trace\nTrace Anchor\nBranch Trace\nGuard\nSide Exit\nFigure 5. A tree with two traces, a trunk trace and one branch trace. The trunk trace contains a guard to which a branch trace was attached. The branch trace contain a guard that may fail and trigger a side exit. Both the trunk and the branch trace loop back to the tree anchor, which is the beginning of the trace tree.\nAs a result, if a hot loop contains traces that always fail, the VM could potentially run much more slowly than the base interpreter: the VM repeatedly spends time trying to record traces, but is never able to run any. To avoid this problem, whenever the VM is about to start tracing, it must try to predict whether it will finish the trace.\nOur prediction algorithm is based on blacklisting traces that have been tried and failed. When the VM fails to finish a trace start ing at a given point, the VM records that a failure has occurred. The VM also sets a counter so that it will not try to record a trace starting at that point until it is passed a few more times (32 in our imple mentation). This backoff counter gives temporary conditions that prevent tracing a chance to end. For example, a loop may behave differently during startup than during its steady-state execution. Af ter a given number of failures (2 in our implementation), the VM marks the fragment as blacklisted, which means the VM will never again start recording at that point.\nTrace 1\nTrace 2\nTrace 1 Trace 2\nBoolean \nNumber\nNumber\nBoolean\nAfter implementing this basic strategy, we observed that for small loops that get blacklisted, the system can spend a noticeable amount of timej ust finding the loop fragment and determining that it has been blacklisted. We now avoid that problem by patching the bytecode. We define an extra no-op bytecode that indicates a loop header. The VM calls into the trace monitor every time the inter preter executes a loop header no-op. To blacklist a fragment, we simply replace the loop header no-op with a regular no-op. Thus, the interpreter will never again even call into the trace monitor.\nThere is a related problem we have not yet solved, which occurs when a loop meets all of these conditions:\nNumber \nBoolean \nNumber\nNumber\nLinked \nLinked Linked Closed \nTrace 1\nTrace 2\nTrace 3\nNumber\nBoolean String\nString Number String  String\nLinked\nClosed Linked\nFigure 6. We handle type-unstable loops by allowing traces to compile that cannot loop back to themselves due to a type mis match. As such traces accumulate, we attempt to connect their loop edges to form groups of trace trees that can execute without having to side-exit to the interpreter to cover odd type cases. This is par ticularly important for nested trace trees where an outer tree tries to call an inner tree (or in this case a forest of inner trees), since inner loops frequently have initially undefined values which change type to a concrete value after the first iteration.\n• The VM can form at least one root trace for the loop.\n• There is at least one hot side exit for which the VM cannot\ncomplete a trace.\n• The loop body is short.\nIn this case, the VM will repeatedly pass the loop header, search for a trace, find it, execute it, and fall back to the interpreter. With a short loop body, the overhead of finding and calling the trace is high, and causes performance to be even slower than the basic interpreter. So far, in this situation we have improved the implementation so that the VM can complete the branch trace. But it is hard to guarantee that this situation will never happen. As future work, this situation could be avoided by detecting and blacklisting loops for which the average trace call executes few bytecodes before returning to the interpreter.\nthrough the inner loop, {i2, i3, i5, α}. The α symbol is used to indicate that the trace loops back the tree anchor.\nWhen execution leaves the inner loop, the basic design has two choices. First, the system can stop tracing and give up on compiling the outer loop, clearly an undesirable solution. The other choice is to continue’ tracing, compiling traces for the outer loop inside the inner loops trace tree.\nFor example, the program might exit at i5 and record a branch trace that incorporates the outer loop: {i5, i7, i1, i6, i7, i1, α}. Later, the program might take the other branch at i2 and then exit, recording another branch trace incorporating the outer loop: {i2, i4, i5, i7, i1, i6, i7, i1, α}. Thus, the outer loop is recorded and compiled twice, and both copies must be retained in the trace cache.\n4. Nested Trace Tree Formation\nFigure 7 shows basic trace tree compilation (11) applied to a nested loop where the inner loop contains two paths. Usually, the inner loop (with header at i2) becomes hot first, and a trace tree is rooted at that point. For example, the first recorded trace may be a cycle\nt1 Outer Tree\ni1\ni1 t1 Nested Tree\nTree Call\nt2\ni2\ni2\nNested Tree\ni3\nt2\ni6 i3 i4\nExit Guard\ni4\nt4\ni5\ni5\nExit Guard\ni7\ni6\nFigure 8. Control flow graph of a loop with two nested loops (left) and its nested trace tree configuration (right). The outer tree calls  with an if statem\nent\nFigure 7. Control flow graph of a nested loop\nthe two inner nested trace trees and places guards at their side exit inside the inner most loop (a). An inner tre“e cap”tures the inner\nlocations.\nloop, and is nested inside an outer tree which calls the inner tree.\nThe inner tree returns to the outer tree once it exits along its loop condition guard (b).\nloop is entered with m different type maps (on geometric average), then we compile O(mk) copies of the innermost loop. As long as m is close to 1, the resulting trace trees will be tractable.\nIn general, if loops are nested to d¨epth k, and each loop has n paths\nAn important detail is that the call to the inner trace tree must act (on geometric average), this naıve strategy yields O(nk) traces,\nlike a function call site: it must return to the same point every time. which can easily fill the trace cache.\nThe goal of nesting is to make inner and outer loops independent; In order to execute programs with nested loops efficiently, a\nthus when the inner tree is called, it must exit to the same point tracing system needs a technique for covering the nested loops with\nin the outer tree every time with the same type map. Because we native code without exponential trace duplication.\ncannot actually guarantee this property, we must guard on it after the call, and side exit if the property does not hold. A common\n4.1 Nesting Algorithm\nreason for the inner tree not to return to the same point would The key insight is that if each loop is represented by its own trace\nbe if the inner tree took a new side exit for which it had never\ntree, the code for each loop can be contained only in its own tree,\ncompiled a trace. At this point, the interpreter PC is in the inner and outer loop paths will not be duplicated. Another key fact is that\ntree, so we cannot continue recording or executing the outer tree. we are not tracing arbitrary bytecodes that might have irreduceable\nIf this happens during recording, we abort the outer trace, to give control flow graphs, but rather bytecodes produced by a compiler\nthe inner tree a chance to finish growing. A future execution of the for a language with structured control flow. Thus, given two loop\nouter tree would then be able to properly finish and record a call to edges, the system can easily determine whether they are nested\nthe inner tree. If an inner tree side exit happens during execution of and which is the inner loop. Using this knowledge, the system ca’n\na compiled trace for the outer tree, we simply exit the outer trace compile inner and outer lo’ops separately, and make the outer loops\nand start recording a new branch in the inner tree.\ntraces call the inner loops trace tree.\nThe algorithm for building nested trace trees is as follows. We\n4.2 Blacklisting with Nesting\nstart tracing at loop headers exactly as in the basic tracing system.\nThe blacklisting algorithm needs modification to work well with When we exit a loop (detected by comparing the interpreter PC\nnesting. The problem is that outer loop traces often abort during with the range given by the loop edge), we stop the trace. The\nstartup (because the inner tree is not available or takes a side exit), key step of the algorithm occurs when we are recording a trace\nwhich would lead to their being quickly blacklisted by the basic for loop LR (R for loop being recorded) and we reach the header\nalgorithm.\nof a different loop LO (O for other loop). Note that LO must be an\nThe key observation is that when an outer trace aborts because inner loop of LR because we stop the trace when we exit a loop.\nthe inner tree is not ready, this is probably a temporary condition.\n• If LO has a type-matching compiled trace tree, we call LO as\nThus, we should not count such aborts toward blacklisting as long a nested trace tree. If the call succeeds, then we record the call\nas we are able to build up more traces for the inner tree.\nin the trace for LR. On future executions, the trace for LR will\nIn our implementation, when a’n outer tree aborts on the inner call the inner trace directly.\ntree, we increment the outer trees blacklist counter as usual and\n• If LO does not have a type-matching compiled trace tree yet,\nback off on compiling it. When the inner tree finish“es a trace”, we decrement the blacklist counter on the outer loop, forgiving the we have to obtain it before we are able to proceed. In order\nouter loop for aborting previously. We also undo the backoff so that to do this, we simply abort recording the first trace. The trace\nthe outer tree can start immediately trying to compile the next time monitor will see the inner loop header, and will immediately\nwe reach it.\nstart recording the inner loop. 2\nIf all the loops in a nest are type-stable, then loop nesting creates no duplication. Otherwise, if loops are nested to a depth k, and each\n5. Trace Tree Optimization\nThis section explains how a recorded trace is translated to an\n2Instead of aborting the outer recording, we could principally merely sus\noptimized machine code trace. The trace compilation subsystem,\npend the recording, but that would require the implementation to be able\nNANOJIT, is separate from the VM and can be used for other\nto record several traces simultaneously, complicating the implementation, while saving only a few iterations in the interpreter.\napplications.\nDescription\nTag JS Type \n5.1 Optimizations\nxx1 number \n31-bit integer representation\nBecause traces are in SSA form and have noj oin points or φ\npointer to JSObject handle\n000 object \nnodes, certain optimizations are easy to implement. In order to\n010 number \npointer to double handle\nget good startup performance, the optimizations must run quickly,\npointer to JSString handle\n100 string \nso we chose a small set of optimizations. We implemented the\n110 boolean \nenumeration for null, undefined, true, false\noptimizations as pipelined filters so that they can be turned on and\nnull, or\noff independently, and yet all run inj ust two loop passes over the\nundefined\ntrace: one forward and one backward.\nEvery time the trace recorder emits a LIR instruction, the in\nFigure 9. Tagged values in the SpiderMonkey JS interpreter. struction is immediately passed to the first filter in the forward\nTesting tags, unboxing (extracting the untagged value) and boxing pipeline. Thus, forward filter optimizations are performed as the\n(creating tagged values) are significant costs. Avoiding these costs trace is recorded. Each filter may pass each instruction to the next\nis a key benefit of tracing.\nfilter unchanged, write a different instruction to the next filter, or write no instruction at all. For example, the constant folding filter can replace a multiply instruction like v13 := mul3, 1000 with a constant instruction v13 = 3000.\nheuristic selects v with minimum vm. The motivation is that this We currently apply four forward filters:\nfrees up a register for as long as possible given a single spill.\nIf we need to spill a value vs at this point, we generate the\n• On ISAs without floating-point instructions, a soft-float filter\nrestore codej ust after the code for the current instruction. The converts floating-point LIR instructions to sequences of integer\ncorresponding spill code is generatedj ust after the last point where instructions.\nvs was used. The register that was assigned to vs is marked free for the preceding code, because that register can now be used freely\n• CSE (constant subexpression elimination),\nwithout affecting the following code\n• expression simplification, including constant folding and a few\nalgebraic identities (e.g., a − a = 0), and\n6. Implementation\n• source language semantic-specific expression simplification,\nTo demonstrate the effectiveness of our approach, we have im primarily algebraic identities that allow DOUBLE to be replaced\nplemented a trace-based dynamic compiler for the SpiderMonkey\nwith INT. For example, LIR that converts an INT to a DOUBLE\nJavaScript Virtual Machin’e (4). SpiderMonkey is the JavaScript and then back again would be removed by this filter.\nVM embedded in Mozillas Firefox open-source web browser (2), When trace recording is completed, nanojit runs the backward\nwhich is used by more than 200 million users world-wide. The core optimization filters. These are used for optimizations that require\nof SpiderMonkey is a bytecode interpreter implemented in C++. backward program analysis. When running the backward filters,\nIn SpiderMonkey, all JavaScript values are represented by the nanojit reads one LIR instruction at a time, and the reads are passed\ntype jsval. A jsval is machine word in which up to the 3 of the\nthrough the pipeline.\nleast significant bits are a type tag, and the remaining bits are data.\nWe currently apply three backward filters:\nSee Figure 6 for details. All pointers contained in jsvals point to GC-controlled blocks aligned on 8-byte boundaries.\n• Dead data-stack store elimination. The LIR trace encodes many\nJavaScript object values are mappings of string-valued property stores to locations in the interpreter stack. But these values are\nnames to arbitrary values. They are represented in one of two ways never read back before exiting the trace (by the interpreter or\nin SpiderMonkey. Most objects are represented by a shared struc another trace). Thus, stores to the stack that are overwritten\ntural description, called the object shape, that maps property names before the next exit are dead. Stores to locations that are off\nto array indexes using a hash table. The object stores a pointer to the top of the interpreter stack at future exits are also dead.\nthe shape and the array of its own property values. Objects with\n• Dead call-stack store elimination. This is th’e same optimization\nlarge, unique sets of property names store their properties directly as above, except applied to the interpreters call stack used for\nin a hash table.\nfunction call inlining.\nThe garbage collector is an exact, non-generational, stop-the world mark-and-sweep collector.\n• Dead code elimination. This eliminates any operation that\nIn the rest of this section we discuss key areas of the TraceMon stores to a value that is never used.\nkey implementation.\nAfter a LIR instruction is succe’ssfully read (“pulled”) from\n6.1 Calling Compiled Traces\nthe backward filter pipeline, nanojits code generator emits native machine instruction(s) for it.\nCompiled traces are stored in a trace cache, indexed by intepreter PC and type map. Traces are compiled so that they may be\n5.2 Register Allocation\ncalled as functions using standard native calling conventions (e.g., FASTCALL on x86).\nWe use a simple greedy register allocator that makes a single\nThe interpreter must hit a loop edge and enter the monitor in backward pass over the trace (it is integrated with the code gen\norder to call a native trace for the first time. The monitor computes erator). By the time the allocator has reached an instruction like\nthe current type map, checks the trace cache for a trace for the v3 = add v1, v2, it has already assigned a register to v3. If v1 and\ncurrent PC and type map, and if it finds one, executes the trace. v2 have not yet been assigned registers, the allocator assigns a free\nTo execute a trace, the monitor must build a trace activation register to each. If there are no free registers, a val“ue is se”lected for spilling. We use a class heuristic that selects the oldest register\nrecord containing imported local and global variables, temporary stack space, and space for arguments to native calls. The local and carried value (6).\nThe heuristic considers the set R of values v in registers imme\nglobal values are then copied from the interpreter state to the trace activation record. Then, the trace is called like a normal C function diately after the current instruction for spilling. Let vm be the last instruction before the current where each v is referred to. Then the\npointer.\nWhen a trace call returns, the monitor restores the interpreter state. First, the monitor checks the reason for the trace exit and\nRe’cording is activated by a pointer“ swap tha”t sets the inter preters dispatch table to call a single interrupt routine for ev\nery bytecode. The interrupt routine first calls a bytecode-specific\napplies blacklisting if needed. Then, it pops or synthesizes inter preter JavaScript call stack frames as needed. Finally, it copies the imported variables back from the trace activation record to the in\nrecording routine. Then, it turns off recording if necessary (e.g., the trace ended). Finally, itj umps to the standard interpreter byte code implementation. Some bytecodes have effects on the type map that cannot be predicted before executing the bytecode (e.g., call ing String.charCodeAt, which returns an integer or NaN if the index argument is out of range). For these, we arrange for the inter\nterpreter state.\nAt least in the current implementation, these steps have a non\nnegligible runtime cost, so minimizing the number of interpreter to-trace and trace-to-interpreter transitions is essential for perfor\npreter to call into the recorder again after executing the bytecode. Since such hooks are relatively rare, we embed them directly into the interpreter, with an additional runtime check to see whether a\nmance. (see also Section 3.3). Our experiments (see Figure 12) show that for programs we can trace well such transitions hap\npen infrequently and hence do not contribute significantly to total runtime. In a few programs, where the system is prevented from recording branch traces for hot side exits by aborts, this cost can rise to up to 10% of total execution time.\nrecorder is currently active.\nWhile separating the interpreter from the recorder reduces indi vidual code complexity, it also requires careful implementation and extensive testing to achieve semantic equivalence.\n6.2 Trace Stitching\nIn some cases achieving this equivalence is difficult since Spi derMonkey follows a fat-bytecode design, which was found to be Transitions from a trace to a branch trace at a side exit avoid the\nbeneficial to pure interpreter performance.\ncosts of calling traces from the monitor, in a feature called trace\nIn fat-bytecode designs, individual bytecodes can implement\nstitching. At a side exit, the exiting trace only needs to write live register-carried values back to its trace activation record. In our im\ncomplex processing (e.g., the getprop bytecode, which imple ments full JavaScript property value access, including special cases for cached and dense array access).\nplementation, identical type maps yield identical activation record layouts, so the trace activation record can be reused immediately\nFat bytecodes have two advantages: fewer bytecodes means by the branch trace.\nlower dispatch cost, and bigger bytecode implementations give the compiler more opportunities to optimize the interpreter.\nIn programs with branchy trace trees with small traces, trace stitching has a noticeable cost. Although writing to memory and then soon reading back would be expected to have a high L1 cache hit rate, for small traces the increased instruction count has\nFat bytecodes are a problem for TraceMonkey because they require the recorder to reimplement the same special case logic\nin the same way. Also, the advantages are reduced because (a)\na noticeable cost. Also, if the writes and reads are very close in the dynamic instruction stream, we have found that current\ndispatch costs are eliminated entirely in compiled traces,’ (b) the traces contain only one special case, not the interpreters large\nx86 processors often incur penalties of 6 cycles or more (e.g., if the instructions use different base registers with equal values, the processor may not be able to detect that the addresses are the same\nchunk of code, and (c) TraceMonkey spends less time running the base interpreter.\nOne way we have mitigated these problems is by implementing certain complex bytecodes in the recorder as sequences of simple bytecodes. Expressing the original semantics this way is not too dif ficult, and recording simple bytecodes is much easier. This enables us to retain the advantages of fat bytecodes while avoiding some of\nright away).\nThe alternate solution is to recompile an entire trace tree, thus\nachieving inter-trace register allocation (10). The disadvantage is that tree recompilation takes time quadratic in the number of traces.\nWe believe that the cost of recompiling a trace tree every time a branch is added would be prohibitive. That problem might be\ntheir problems for trace recording. This is particularly effective for fat bytecodes that recurse back into the interpreter, for example to convert an object into a primitive value by invoking a well-known method on the object, since it lets us inline this function call.\nmitigated by recompiling only at certain points, or only for very hot, stable trees.\nIn the future, multicore hardware is expected to be common,\nIt is important to note that we split fat opcodes into thinner op making background tree recompilation attractive. In a closely re\ncodes only during recording. When running purely interpretatively (i.e. code that has been blacklisted), the interpreter directly and ef\nlated project (13) background recompilation yielded speedups of up to 1.25x on benchmarks with many branch traces. We plan to\nficiently executes the fat opcodes.\napply this technique to TraceMonkey as future work.\n6.3 Trace Recording\n6.4 Preemption\nThej ob of the trace recorder is to emit LIR with identical semantics\nto the currently running interpreter bytecode trace. A good imple mentation should have low impact on non-tracing interpreter per formance and a convenient way for implementers to maintain se\nSpiderMonkey, like many VMs, needs to preempt the user program periodically. The main reasons are to prevent infinitely looping\nscripts from locking up the host system and to schedule GC. “ mantic equivalence.\nIn the i”nterpreter, this had been implemented by setting a pre empt now flag that was checked on every backwardj ump. This strategy carried over into TraceMonkey: the VM inserts a guard on\nIn our implementation, the only direct modification to the inter preter is a call to the trace monitor at loop edges. In our benchmark\nresults (see Figure 12) the total time spent in the monitor (for all activities) is usually less than 5%, so we consider the interpreter impact requirement met. Incrementing the loop hit counter is ex\nthe preemption flag at every loop edge. We measured less than a 1% increase in runtime on most benchmarks for this extra guard.\nIn practice, the cost is detectable only for programs with very short\npensive because it requires us to look up the loop in the trace cache, but we have tuned our loops to become hot and trace very quickly (on the second iteration). The hit counter implementation could be\nloops.\nWe tested and rejected a solution that avoided the guards by\ncompiling the loop edge as an unconditionalj ump, and patching thej ump target to an exit routine when preemption is required. This solution can make the normal case slightly faster, but then\nimproved, which might give us a small increase in overall perfor mance, as well as more flexibility with tuning hotness thresholds. Once a loop is blacklisted we never call into the trace monitor for\npreemption becomes very slow. The implementation was also very that loop (see Section 3.3).\ncomplex, especially trying to restart execution after the preemption.\n6.5 Calling External Functions\n?>9@AJ.D<F@-<>2.@A:0>#3$4,56#\n?>9@A?J>.90@AAJ:.<></JC.//F880--2##33$$44%$5566##\nLike most interpreters, SpiderMonkey has a foreign function inter face (FFI) that allows it to call C builtins and host system functions\n?>9?@>A9J@.A1J<.B?<2?)>'<##33%$44((5566##\n(e.g., web browser control and DOM access). The FFI has a stan\n92J25:.-A<#3'4%56#\ndard signature for JS-callable functions, the key argument of which\n77<><;>.;?.::2</9>I9<<FF..A?08797?##33(*44,$5566##\nis an array of boxed values. External functions called through the\n7<>;./89-@/#3'4,56#\nFFI interact with the program state through an interpreter API (e.g.,\n--<<>>22.B.8B89977<<>.>5.>:8<H921##33$$44!$5566##\nto read a property from an argument). There are also certain inter\n/9=:>8.?;<$#3(4,56#\npreter builtins that do not use the FFI, but interact with the program\n/9/=9=::>8>8.7.<-2(?##33%$44&)5566##\nstate in the same way, such as the CallIteratorNext function /8A>98FG8E.92/09?@D2#3$4!56# used with iterator objects. TraceMonkey must support this FFI in\n1@>18@>:8?.:1?@.>AE?@@?22D.2<.A1-@>#?3#%3(%44*%5566##\norder to speed up code that interacts with the host system inside hot\n1@>8:?.1@>?.@A.1=>2#3+4*56#\nloops.\n1@>8:?.&1@><.1/@/>2?.?@?A..A1?=@2>2D#23#%3&(44!(5566##\nCalling external functions from TraceMonkey is potentially dif\n<//2??.A18-=#3'4%56#\n< .BAAC ;#%4%5#\nficult because traces do not update the interpreter state until exit\n<//2/?/?2.1??@A<<9=.>902/2?#33!4,566#\ning. In particular, external functions may need the call stack or the\n&-&.-9.<7=>89<9:/;2##33%$44,%5566##\nglobal variables, but they may be out of date.\n&-./012#3%4%56#\nFor the out-of-date call stack problem, we refactored some of\n!\"# $!\"# %!\"# &!\"# '!\"# (!\"# )!\"# *!\"# +!\"# ,!\"# $!!\"#\nthe interpreter API implementation functions to re-materialize the interpreter call stack on demand.\nKA>29:92># L<ID2#\nWe developed a C++ static analysis and annotated some inter preter functions in order to verify that the call stack is refreshed at any point it needs to be used. In order to access the call stack,\nFigure 11. Fraction of dynamic bytecodes executed by inter a function must be annotated as either FOR CESSTACK or RE\npreter and on native traces. The speedup vs. interpreter is shown\nQUIRESSTACK. These annotations are also required in order to call\nin parentheses next to each test. The fraction of bytecodes exe REQUIRESSTACK functions, which are presumed to access the call\ncuted while recording is too small to see in this figure, except stack transitively. FOR CESSTACK is a trusted annotation, applied\nfor crypto-md5, where fully 3% of bytecodes are executed while to only 5 functions, that means the function refreshes the call stack.\nrecording. In most of the tests, almost all the bytecodes are exe REQUIRESSTACK is an untrusted annotation that means the func\ncuted by compiled traces. Three of the benchmarks are not traced tion may only be called if the call stack has already been refreshed.\nat all and run in the interpreter.\nSimilarly, we detect when host functions attempt to directly read or write global variables, and force the currently running trace to side exit. This is necessary since we cache and unbox global\nloops and heavily branching code, and a specialized fuzz tester in variables into the activation record during trace execution.\ndeed revealed several regressions which we subsequently corrected.\nSince both call-stack access and global variable access are rarely performed by host functions, performance is not significantly\n7. Evaluation\naffected by these safety mechanisms.\nAnother problem is that external functions can reenter the inter\nWe evaluated our JavaScript tracing implementation using Sun preter by calling scripts, which in turn again might want to access\nSpider, the industry standard JavaScript benchmark suite. SunSpi the call stack or global variables. To address this problem, we made\nder consists of 26 short-running (less than 250ms, average 26ms) the VM set a flag whenever the interpreter is reentered while a com\nJavaScript programs. This is in stark contrast to benchmark suites piled trace is running.\nsuch as SpecJVM98 (3) used to evaluate desktop and server Java Every call to an external function then checks this flag and exits\nVMs. Many programs in those benchmarks use large data sets and the trace immediately after returning from the external function call\nexecute for minutes. The SunSpider programs carry out a variety of if it is set. There are many external functions that seldom or never\ntasks, primarily 3d rendering, bit-bashing, cryptographic encoding, reenter, and they can be called without problem, and will cause\nmath kernels, and string processing.\ntrace exit on’ly if necessary.\nAll experiments were performed on a MacBook Pro with 2.2 The FFIs boxed value array requirement has a performance\nGHz Core 2 processor and 2 GB RAM running MacOS 10.5. cost, so we defined a new FFI that allows C functions to be an\nBenchmark results. The main question is whether programs notated with their argument types so that the tracer can call them\nrun faster with tracing. For this, we ran the standard SunSpider test directly, without unnecessary argument conversions.\ndriver, which starts a JavaScript interpreter, loads and runs each Currently, we do not support calling native property get and set\nprogram once for warmup, then loads and runs each program 10 override functions or DOM functions directly from trace. Support\ntimes and reports the average time taken by each. We ran 4 differ is planned future work.\nent configurations for comparison: (a) SpiderMonkey, the baseline interpreter, (b) TraceMonkey, (d) SquirrelFish Extre’me (SFX), the\n6.6 Correctness\ncall-threaded JavaScript interpreter used in Apples WebKit, and During development, we had access to existing JavaScript test\n(e) V8, the method-compiling JavaScript VM from Google.\nsuites, but most of them were not designed with tracing VMs in\nFigure 10 shows the relative speedups achieved by tracing, SFX, mind and contained few loops.\nand V8 against the baseline (SpiderMonkey). Tracing achieves the ’\nOne tool that helped us greatly was Mozillas JavaScript fuzz\nbest speedups in integer-heavy benchmarks, up to the 25x speedup on bitops-bitwise-and.\ntester, JSFUNFUZZ, which generates random JavaScript programs\nTraceMonkey is the fastest VM on 9 of the 26 benchmarks by nesting random language elements. We modified JSFUNFUZZ\n(3d-morph, bitops-3bit-bits-in-byte, bitops-bitwise to generate loops, and also to test more heavily certain constructs\nand, crypto-sha1, math-cordic, math-partial-sums, math we suspected would reveal flaws in our’ implementation. For exam\nspectral-norm, string-base64, string-validate-input). ple, we suspected bugs in TraceMonkeys handling of type-unstable\nFigure 10. Speedup vs. a baseline Jav’aScript interpreter (SpiderMonkey) for our trace-based JIT compiler, Apple’s SquirrelFish Extreme inline threading interpreter and Googles V8 JS compiler. Our system generates particularly efficient code for programs that benefit most from type specialization, which includes SunSpider Benchmark programs that perform bit manipulation. We type-specialize the code in question to use integer arithmetic, which substantially improves performance. For one of the benchmark programs we execute 25 times faster than the SpiderMonkey interpreter, and almost 5 times faster than V8 and SFX. For a large number of benchmarks all three VMs produce similar results. We perform worst on benchmark programs that we do not trace and instead fall back onto th’e interpreter. This includes the recursive\nbenchmarks access-binary-trees and control-flow-recursive\n, for which we currently dont generate any native code.\n• Two programs trace well, but have a long compilation time.\nIn particular, the bitops benchmarks are short programs that per form many bitwise operations, so TraceMonkey can cover the en tire program with 1 or 2 traces that operate on integers. TraceMon key runs all the other programs in this set almost entirely as native code.\naccess-nbody forms a large number of traces (81). crypto-md5 forms one very long trace. We expect to improve performance on this programs by improving the compilation speed of nano jit.\nregexp-dna is dominated by regular expression matching, which is implemented in all 3 VMs by a special regular expression compiler. Thus, performance on this benchmark has little relation to the trace compi’lation approach discussed in this paper.\n• Some programs trace very well, and speed up compared to\nthe interpreter, but are not as fast as SFX and/or V8, namely bitops-bits-in-byte, bitops-nsieve-bits, access fannkuch, access-nsieve, and crypto-aes. The reason is not clear, but all of these programs have nested loops with small bodies, so we suspect that the implementation has a rela tively high cost for calling nested traces. string-fasta traces well, but its run time is dominated by string processing builtins, which are unaffected by tracing and seem to be less efficient in SpiderMonkey than in the two other VMs.\nTraceMonkeys smaller speedups on the other benchmarks can be attributed to a few specific causes:\n• The implementation does not currently trace recursion, so\nTraceMonkey achieves a small speedup or no speedup on benchmarks that use recursion extensively: 3d-cube, 3d raytrace, access-binary-trees, string-tagcloud, and controlflow-recursive.\nDetailed performance metrics. In Figure 11 we show the frac tion of instructions interpreted and the fraction of instructions exe cuted as native code. This figure shows that for many programs, we are able to execute almost all the code natively.\n• The implementation does not currently trace eval and some\nother functions implemented in C. Because date-format tofte and date-format-xparb use such functions in their main loops, we do not trace them.\nFigure 12 breaks down the total execution time into four activ ities: interpreting bytecodes while not recording, recording traces (including time taken to interpret the recorded trace), compiling traces to native code, and executing native code traces.\n• The implementation does not currently trace through regular\nexpression replace operations. The replace function can be passed a function object used to compute the replacement text. Our implementation currently does not trace functions called as replace functions. The run time of string-unpack-code is dominated by such a replace call.\nThese detailed metrics allow us to estimate parameters for a simple model of tracing performance. These estimates should be considered very rough, as the values observed on the individual benchmarks have large standard deviations (on the order of the\nFigure 13. Detailed trace recording statistics for the SunSpider benchmark set.\nmean). We exclude regexp-dna from the following calculations, because most of its time is spent in the regular expression matcher, which has much different performance characteristics from the other programs. (Note that this only makes a difference of about 10% in the results.) Dividing the total execution time in processor clock cycles by the number of bytecodes executed in the base interpreter shows that on average, a bytecode executes in about 35 cycles. Native traces take about 9 cycles per bytecode, a 3.9x speedup over the interpreter.\nfastest available JavaScript inline threaded interpreter (SFX) on 9 of 26 benchmarks.\n8. Related Work\nTrace optimization for dynamic languages. The closest area of related work is on applying trace optimization to type-specialize dynamic languages. Existing work shares the idea of generating type-specialized code speculatively with guards along interpreter traces.\nUsing similar computations, we find that trace recording takes about 3800 cycles per bytecode, and compilation 3150 cycles per bytecode. Hence, during recording and compiling the VM runs at 1/200 the speed of the interpreter. Because it costs 6950 cycles to compile a bytecode, and we save 26 cycles each time that code is run natively, we break even after running a trace 270 times.\nTo our knowledge, Rigos Psyco (16) is the only published type-specializing trace compiler for a dynamic language (Python). Psyco does not attempt to identify hot loops or inline function calls. Instead, Psyco transforms loops to mutual recursion before running and trac’es all operations.\nThe other VMs we compared with achieve an overall speedup of 3.0x relative to our baseline interpreter. Our estimated native code speedup of 3.9x is significantly better. This suggests that our compilation techniques can generate more efficient native code than any other current JavaScript VM.\nPalls LuaJIT is a Lua VM in development that uses trace com pilation ideas. (1). There are no publications on LuaJIT but the cre ator has told us that LuaJIT has a similar design to our system, but will use a less aggressive type speculation (e.g., using a floating point representation for all number values) and does not generate nested traces for nested loops.\nThese estimates also indicate that our startup performance could be substantially better if we improved the speed of trace recording and compilation. The estimated 200x slowdown for recording and compilation is very rough, and may be influenced by startup factors in the interpreter (e.g., caches that have not warmed up yet during recording). One observation supporting this conjecture is that in the tracer, interpreted bytecodes take about 180 cycles to run. Still, recording and compilation are clearly both expensive, and a better implementation, possibly including redesign of the LIR abstract syntax or encoding, would improve startup performance.\nGeneral trace optimization. General trace optimization has a longer history that has treated mostly native code and typed languages like Java. Thus, these systems have focused less on type specialization and more on other optimizations.\nDynamo (7) by Bala et al, introduced native code tracing as a replacement for profile-guided optimization (PGO). A major goal was to perform PGO online so that the profile was specific to the current execution. Dynamo used loop headers as candidate hot traces, but did not try to create loop traces specifically.\nOur performance results confirm that type specialization using trace trees substantially improves performance. We are able to outperform the fastest available JavaScript compiler (V8) and the\nTrace trees were originally proposed by Gal et al. (11) in the context of Java, a statically typed language. Their trace trees ac tually inlined parts of outer loops within the inner loops (because\n=<6>?J+B:F>*:</+>?7-<#0(1923#\n=<6>?J+-?7+:,A+,5*/#0(1$23#\n=<6>?=J<6<>?:JJ+,@F:5=-<*:##00((11C(2233##\n=<6>?J+.:=+/&%#0$1C23# 4:<8+=7/6,/<J6/:2F+7?5*6?4:##00%D11$(2233##\n4:<8+7:6I:+F+=-4=#0C1923#\n*:</+@5464:<:8<+,2576:*6>.,##00%(119!2233##\n*:</+@,566;47<:5<++<=58H:/(##00C(119(2233##\n,6;7<5+4*C#0$1)23# ,5?<65FG5E,+66;/7,<-56+=:>B//=##00((11!&2233##\n.><57=+?=>/B/+.><=#0$1D23#\n.>.<5><75=7+=.+>.<E><=>=+>/?++:.?;*<#/0#$0'C11D$2233##\nerate native code with nearly the same structure but better perfor mance.\nCall threading, also known as context threading (8), compiles methods by generating a native call instruction to an interpreter method for each interpreter bytecode. A call-return pair has been shown to be a potentially much more efficient dispatch mechanism than the indirectj umps used in standard bytecode interpreters.\nInline threading (15) copies chunks of interpreter native code which implement the required bytecodes into a native code cache, thus acting as a simple per-method JIT compiler that eliminates the dispatch overhead.\nNeither call threading nor inline threading perform type special ization. ’\n.><57=+).><+.><=+>?++.;</#0$C1C23#\n::,,,,//====+??=.>5/*B/;##00%)11!$2233## :,,:/,=,=/+.==>?+@::6?;?+<A6-/,/8=##00!$119$2233##\nApples SquirrelFish Extreme (5) is a JavaScript implementa tion based on call threading with selective inline threading. Com bined with efficient interpreter engineering, these threading tech niques have given SFX excellent performance on the standard Sun Spider benc’hmarks.\n)*)+*6+:4;<56:67,8/##00$(119$2233##\n)*+,-./#0$1$23#\nGoogles V8 is a JavaScript implementation primarily based\non inline threading, with call threading only for very complex operations.\nK?</676/<# L5?><56# M/,56*# N547>F/# N:FF#O6:,/# M-?#O6:,/#\n9. Conclusions\nFigure 12. Fraction of time spent on major VM activities. The speedup vs. interpreter is shown in parentheses next to each test. Most programs where the VM spends the majority of its time run ning native code have a good speedup. Recording and compilation costs can be substantial; speeding up those parts of the implemen tation would improve SunSpider performance.\nThis paper described how to run dynamic languages efficiently by recording hot traces and generating type-specialized native code. Our technique focuses on aggressively inlined loops, and for each loop, it generates a tree of native code traces representing the paths and value types through the loop observed at run time. We explained how to identify loop nesting relationships and generate nested traces in order to avoid excessive code duplication due to the many paths through a loop nest. We described our type specialization algorithm. We also described our trace compiler, which translates a trace from an intermediate representation to optimized native code in two linear passes.\ninner loops become hot first), leading to much greater tail duplica tion.\nOur experimental results show that in practice loops typically are entered with only a few different combinations of value types of variables. Thus, a small number of traces per loop is sufficient to run a program efficiently. Our experiments also show that on programs amenable to tracing, we achieve speedups of 2x to 20x.\nYETI, from Zaleski et al. (19) applied Dynamo-style tracing to Java in order to achieve inlining, indirectj ump elimination, and other optimizations. Their primary focus was on designing an interpreter that could easily be gradually re-engineered as a tracing VM.\nSuganuma et al. (18) described region-based compilation (RBC), a relative of tracing. A region is an subprogram worth optimizing that can include subsets of any number of methods. Thus, the com piler has more flexibility and can potentially generate better code, but the profiling and compilation systems are correspondingly more complex.\n10. Future Work\nWork is underway in a number of areas to further improve the performance of our trace-based JavaScript compiler. We currently do not trace across recursive function calls, but plan to add the support for this capability in the near term. We are also exploring adoption of the existing work on tree recompilation in the context of the presented dynamic compiler in order to minimize JIT pause times and obtain the best of both worlds, fast tree stitching as well as the improved code quality due to tree recompilation.\nType specialization for dynamic languages. Dynamic lan guage implementors have long recognized the importance of type specialization for performance. Most previous work has focused on methods instead of traces.\nChambers et. al (9) pioneered the idea of compiling multiple versions of a procedure specialized for the input types in the lan guage Self. In one implementation, they generated a specialized method online each time a method was called with new input types. In another, they used an offline whole-program static analysis to infer input types and constant receiver types at call sites. Interest ingly, the two techniques produced nearly the same performance.\nWe also plan on adding support for tracing across regular ex pression substitutions using lambda functions, function applica tions and expression evaluation using eval. All these language constructs are currently executed via interpretation, which limits our performance for applications that use those features.\nAcknowledgments\nSalib (17) designed a type inference algorithm for Python based on the Cartesian Product Algorithm and used the results to special ize on types and translate the program to C++.\nParts of this effort have been sponsored by the National Science Foundation under grants CNS-0615443 and CNS-0627747, as well as by the California MICRO Program and industrial sponsor Sun Microsystems under Project No. 07-127.\nMcCloskey (14) has work in progress based on a language independent type inference that is used to generate efficient C implementations of JavaScript and Python programs.\nThe U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. Any opinions, findings, and conclusions or rec ommendations expressed here are those of the author and should\nNative code generation by interpreters. The traditional inter preter design is a virtual machine that directly executes ASTs or machine-code-like bytecodes. Researchers have shown how to gen\nnot be interpreted as necessarily representing the official views, policies or endorsements, either expressed or implied, of the Na tional Science foundation (NSF), any other agency of the U.S. Gov ernment, or any of the companies mentioned above.\n[10] A. Gal. EfficientB ytecode Verification and Compilation in a Virtual\nMachineD issertation. PhD thesis, University Of California, Irvine, 2006.\nReferences\n[11] A. Gal, C. W. Probst, and M. Franz. HotpathVM: An effective JIT\ncompiler for resource-constrained devices. In Proceedings of the International Conference on Virtual Execution Environments, pages 144–153. ACM Press, 2006.\n[1] LuaJIT roadmap 2008 - http://lua-users.org/lists/lua-l/2008-02/msg00051.html.\n[12] C. Garrett, J. Dean, D. Grove, and C. Chambers. Measurement and\nApplication of Dynamic Receiver Class Distributions. 1994.\nent -\nerbird email cli\n[2] Mozilla — Firefox web browser and Thund\n[13] J. Ha, M. R. Haghighat, S. Cong, and K. S. McKinley. A concurrent http://www.mozilla.com.\ntrace-basedj ust-in-time compiler forj avascript. Dept.of Computer Sciences, The University of Texas at Austin, TR-09-06, 2009. -\n[3] SPECJVM98 - http://www.spec.org/jvm98/.\n Engine\n[4] SpiderMonkey (JavaScript-C)\n[14] B. McCloskey. Personal communication.\nhttp://www.mozilla.org/js/spidermonkey/.\n[15] I. Piumarta and F. Riccardi. Optimizing direct threaded code by selec me - [5] Surfin’ Safari - Blog Archive - Announcing SquirrelFish Extre\ntive inlining. In Proceedings of theA CM SIGPLAN 1998 conference on Programming language design and implementation, pages 291–\nhttp://webkit.org/blog/214/introducing-squirrelfish-extreme/.\n[6] A. Aho, R. Sethi, J. Ullman, and M. Lam. Compilers: Principles,\n300. ACM New York, NY, USA, 1998.\ntechniques, and tools, 2006.\n[16] A. Rigo. Representation-Based Just-In-time Specialization and the [7] V. Bala, E. Duesterwald, and S. Banerjia. Dynamo: A transparent\nPsyco Prototype for Python. In PEPM, 2004.\ndynamic optimization system. In Proceedings of theA CM SIGPLAN Conference on ProgrammingL anguageD esign andI mplementation, pages 1–12. ACM Press, 2000.\n[17] M. Salib. Starkil’ler: A Static Type Inferencer and Compiler for\nPython. In Masters Thesis, 2004.\n[8] M. Berndl, B. Vitale, M. Zaleski, and A. Brown. Context Threading:\na Flexible and Efficient Dispatch Technique for Virtual Machine In terpreters. In Code Generation and Optimization, 2005. CGO 2005. International Symposium on, pages 15–26, 2005.\n[18] T. Suganuma, T. Yasue, and T. Nakatani. A Region-Based Compila\ntion Technique for Dynamic Compilers. ACM Transactions on Pro grammingL anguages and Systems (TOPLAS), 28(1):134–174, 2006. [19] M. Zaleski, A. D. Brown, and K. Stoodley. YETI: A graduallY\n[9] C. Chambers and D. Ungar. Customization: Optimizing Compiler\nTechnology for SELF, a Dynamically-Typed O bject-Oriented Pro gramming Language. In Proceedings of theA CM SIGPLAN 1989 Conference on ProgrammingL anguageD esign andI mplementation, pages 146–160. ACM New York, NY, USA, 1989.\nExtensible Trace Interpreter. In Proceedings of theI nternational Conference on Virtual Execution Environments, pages 83–93. ACM Press, 2007.\n\nLoops\nTrees\nTraces\nAborts\nFlushes\nTrees/Loop\nTraces/Tree\nTraces/Loop\nSpeedup\n3d-cube\n25\n27\n29\n3\n0\n1.1\n1.1\n1.2\n2.20x\n3d-morph\n5\n8\n8\n2\n0\n1.6\n1.0\n1.6\n2.86x\n3d-raytrace\n10\n25\n100\n10\n1\n2.5\n4.0\n10.0\n1.18x\naccess-binary-trees\n0\n0\n0\n5\n0\n-\n-\n-\n0.93x\naccess-fannkuch\n10\n34\n57\n24\n0\n3.4\n1.7\n5.7\n2.20x\naccess-nbody\n8\n16\n18\n5\n0\n2.0\n1.1\n2.3\n4.19x\naccess-nsieve\n3\n6\n8\n3\n0\n2.0\n1.3\n2.7\n3.05x\nbitops-3bit-bits-in-byte\n2\n2\n2\n0\n0\n1.0\n1.0\n1.0\n25.47x\nbitops-bits-in-byte\n3\n3\n4\n1\n0\n1.0\n1.3\n1.3\n8.67x\nbitops-bitwise-and\n1\n1\n1\n0\n0\n1.0\n1.0\n1.0\n25.20x\nbitops-nsieve-bits\n3\n3\n5\n0\n0\n1.0\n1.7\n1.7\n2.75x\ncontrolflow-recursive\n0\n0\n0\n1\n0\n-\n-\n-\n0.98x\ncrypto-aes\n50\n72\n78\n19\n0\n1.4\n1.1\n1.6\n1.64x\ncrypto-md5\n4\n4\n5\n0\n0\n1.0\n1.3\n1.3\n2.30x\ncrypto-sha1\n5\n5\n10\n0\n0\n1.0\n2.0\n2.0\n5.95x\ndate-format-tofte\n3\n3\n4\n7\n0\n1.0\n1.3\n1.3\n1.07x\ndate-format-xparb\n3\n3\n11\n3\n0\n1.0\n3.7\n3.7\n0.98x\nmath-cordic\n2\n4\n5\n1\n0\n2.0\n1.3\n2.5\n4.92x\nmath-partial-sums\n2\n4\n4\n1\n0\n2.0\n1.0\n2.0\n5.90x\nmath-spectral-norm\n15\n20\n20\n0\n0\n1.3\n1.0\n1.3\n7.12x\nregexp-dna\n2\n2\n2\n0\n0\n1.0\n1.0\n1.0\n4.21x\nstring-base64\n3\n5\n7\n0\n0\n1.7\n1.4\n2.3\n2.53x\nstring-fasta\n5\n11\n15\n6\n0\n2.2\n1.4\n3.0\n1.49x\nstring-tagcloud\n3\n6\n6\n5\n0\n2.0\n1.0\n2.0\n1.09x\nstring-unpack-code\n4\n4\n37\n0\n0\n1.0\n9.3\n9.3\n1.20x\nstring-validate-input\n6\n10\n13\n1\n0\n1.7\n1.3\n2.2\n1.86x",
      "paragraphs": 673,
      "mean_paragraph_words": 19.499,
      "tables": [
        [
          [
            "",
            "Loops",
            "Trees",
            "Traces",
            "Aborts",
            "Flushes",
            "Trees/Loop",
            "Traces/Tree",
            "Traces/Loop",
            "Speedup"
          ],
          [
            "3d-cube",
            "25",
            "27",
            "29",
            "3",
            "0",
            "1.1",
            "1.1",
            "1.2",
            "2.20x"
          ],
          [
            "3d-morph",
            "5",
            "8",
            "8",
            "2",
            "0",
            "1.6",
            "1.0",
            "1.6",
            "2.86x"
          ],
          [
            "3d-raytrace",
            "10",
            "25",
            "100",
            "10",
            "1",
            "2.5",
            "4.0",
            "10.0",
            "1.18x"
          ],
          [
            "access-binary-trees",
            "0",
            "0",
            "0",
            "5",
            "0",
            "-",
            "-",
            "-",
            "0.93x"
          ],
          [
            "access-fannkuch",
            "10",
            "34",
            "57",
            "24",
            "0",
            "3.4",
            "1.7",
            "5.7",
            "2.20x"
          ],
          [
            "access-nbody",
            "8",
            "16",
            "18",
            "5",
            "0",
            "2.0",
            "1.1",
            "2.3",
            "4.19x"
          ],
          [
            "access-nsieve",
            "3",
            "6",
            "8",
            "3",
            "0",
            "2.0",
            "1.3",
            "2.7",
            "3.05x"
          ],
          [
            "bitops-3bit-bits-in-byte",
            "2",
            "2",
            "2",
            "0",
            "0",
            "1.0",
            "1.0",
            "1.0",
            "25.47x"
          ],
          [
            "bitops-bits-in-byte",
            "3",
            "3",
            "4",
            "1",
            "0",
            "1.0",
            "1.3",
            "1.3",
            "8.67x"
          ],
          [
            "bitops-bitwise-and",
            "1",
            "1",
            "1",
            "0",
            "0",
            "1.0",
            "1.0",
            "1.0",
            "25.20x"
          ],
          [
            "bitops-nsieve-bits",
            "3",
            "3",
            "5",
            "0",
            "0",
            "1.0",
            "1.7",
            "1.7",
            "2.75x"
          ],
          [
            "controlflow-recursive",
            "0",
            "0",
            "0",
            "1",
            "0",
            "-",
            "-",
            "-",
            "0.98x"
          ],
          [
            "crypto-aes",
            "50",
            "72",
            "78",
            "19",
            "0",
            "1.4",
            "1.1",
            "1.6",
            "1.64x"
          ],
          [
            "crypto-md5",
            "4",
            "4",
            "5",
            "0",
            "0",
            "1.0",
            "1.3",
            "1.3",
            "2.30x"
          ],
          [
            "crypto-sha1",
            "5",
            "5",
            "10",
            "0",
            "0",
            "1.0",
            "2.0",
            "2.0",
            "5.95x"
          ],
          [
            "date-format-tofte",
            "3",
            "3",
            "4",
            "7",
            "0",
            "1.0",
            "1.3",
            "1.3",
            "1.07x"
          ],
          [
            "date-format-xparb",
            "3",
            "3",
            "11",
            "3",
            "0",
            "1.0",
            "3.7",
            "3.7",
            "0.98x"
          ],
          [
            "math-cordic",
            "2",
            "4",
            "5",
            "1",
            "0",
            "2.0",
            "1.3",
            "2.5",
            "4.92x"
          ],
          [
            "math-partial-sums",
            "2",
            "4",
            "4",
            "1",
            "0",
            "2.0",
            "1.0",
            "2.0",
            "5.90x"
          ],
          [
            "math-spectral-norm",
            "15",
            "20",
            "20",
            "0",
            "0",
            "1.3",
            "1.0",
            "1.3",
            "7.12x"
          ],
          [
            "regexp-dna",
            "2",
            "2",
            "2",
            "0",
            "0",
            "1.0",
            "1.0",
            "1.0",
            "4.21x"
          ],
          [
            "string-base64",
            "3",
            "5",
            "7",
            "0",
            "0",
            "1.7",
            "1.4",
            "2.3",
            "2.53x"
          ],
          [
            "string-fasta",
            "5",
            "11",
            "15",
            "6",
            "0",
            "2.2",
            "1.4",
            "3.0",
            "1.49x"
          ],
          [
            "string-tagcloud",
            "3",
            "6",
            "6",
            "5",
            "0",
            "2.0",
            "1.0",
            "2.0",
            "1.09x"
          ],
          [
            "string-unpack-code",
            "4",
            "4",
            "37",
            "0",
            "0",
            "1.0",
            "9.3",
            "9.3",
            "1.20x"
          ],
          [
            "string-validate-input",
            "6",
            "10",
            "13",
            "1",
            "0",
            "1.7",
            "1.3",
            "2.2",
            "1.86x"
          ]
        ]
      ],
      "table_shapes": [
        [
          27,
          10
        ]
      ],
      "images": 0,
      "page_breaks": 13
    },
    "content": {
      "expected_tokens": 14700,
      "actual_tokens": 14747,
      "matched_tokens": 14035,
      "recall": 0.954762,
      "precision": 0.951719,
      "order": 0.1835,
      "f1": 0.953238,
      "duplicate_tokens": 138,
      "missing_count": 665,
      "extra_count": 712,
      "missing": [
        "-",
        "-",
        "-",
        "-",
        "based",
        "just",
        "just",
        "just",
        "just",
        "just",
        "just",
        "just",
        "just",
        "just",
        "just",
        "just",
        "time",
        "type",
        "specialization",
        "languages",
        "languages",
        "r",
        "$",
        "$",
        "$",
        "$",
        "$",
        "$",
        "$",
        "$",
        "$",
        "$"
      ],
      "extra": [
        "trace",
        "in",
        "in",
        "in",
        "in",
        "in",
        "in",
        "com",
        "com",
        "com",
        "com",
        "com",
        "com",
        "pile",
        "code",
        "an",
        "al",
        "al",
        "ternative",
        "run",
        "on",
        "ac",
        "ac",
        "tual",
        "inter",
        "inter",
        "inter",
        "inter",
        "inter",
        "inter",
        "inter",
        "inter"
      ]
    },
    "source_derived_table_match": null,
    "visual": {
      "output_pages": 32,
      "compared_pages": 8,
      "pixel_similarity": 0.9007
    },
    "warnings": [
      "Layout fidelity is lossy; headers/footers/fonts are not fully preserved.",
      "auto: text_based; ocr=never",
      "layout_ml=off",
      "Page 1: table_low_confidence=0.40; emitted paragraphs.",
      "Page 3: table_low_confidence=0.37; emitted paragraphs.",
      "Page 4: table_low_confidence=0.38; emitted paragraphs.",
      "Page 5: table_low_confidence=0.39; emitted paragraphs.",
      "Page 7: table_low_confidence=0.57; emitted paragraphs.",
      "Page 8: table_low_confidence=0.39; emitted paragraphs.",
      "Page 9: table_low_confidence=0.39; emitted paragraphs.",
      "Page 10: table_low_confidence=0.48; emitted paragraphs.",
      "Pages 2/3: continued multi-page table.",
      "ocr_policy=never — skipped OCR (digital text layer kept).",
      "No source-derived grid to compare against; table score is structural (cells checked against the source text)."
    ],
    "warning_count": 14,
    "error": null,
    "models": {
      "onnx_providers": [
        "AzureExecutionProvider",
        "CPUExecutionProvider"
      ],
      "ocr_available": true,
      "ocr_ar_status": "on",
      "layout_model": {
        "path": "gateway\\models\\layout\\v1\\layout.onnx",
        "bytes": 130502330,
        "sha256": "250dbad1dfb9e4983fab75e1bf5085cd56ec3f41d5c7d0f8623ec74856e7aa67"
      }
    }
  }
]