Matching the Speed of Hand-Written Assembly with an Interpreter Written in Java

Act IV: Everything Everywhere All at Once

 and the team started the Maxine VM project, the idea was Java on Java.

 to Java, so I need an excuse to convince myself.

 object, but the handlers still access the backing arrays directly.

Let us encapsulate those accesses in the stack abstraction. Instead of:

class Stack {
    long top;
}

static long iconst0Handler(long bci, Frame frame, Stack stack) {
    long slot = stack.top;
    UNSAFE.putLong(frame.prims, slot * 8L + OFFSET_arrayStart, 0);
    stack.top = slot + 1;
    return bci + 1;
}

class Stack {
    long top;

    void pushInt(Frame frame, int value) {
        UNSAFE.putLong(frame.prims, top * 8L + OFFSET_arrayStart, value);
        top++;
    }

    int popInt(Frame frame) {
        top--;
        return (int) UNSAFE.getLong(frame.prims, top * 8L + OFFSET_arrayStart);
    }
}

static long iconst0Handler(long bci, Frame frame, Stack stack) {
    stack.pushInt(frame, 0);
    return bci + 1;
}

 now provides an overlay on the frame, while virtual argument expansion keeps that overlay in the source without carrying an object into the emitted assembly.

The handler no longer needs to know where zero goes.

, caching values in instance fields instead of writing them immediately to the backing array.

Again, argument expansion carries these fields between handlers in registers.

Let us return to the simplest cache from the previous article, with two primitive fields, 

 counting the cached JVM slots. We can view these fields as an extension of 

: they hold stack slots just like the array elements, except they live in registers. 

 marks the end of the materialized stack; the cached slots extend it. Object references remain in the backing frame, so before pushing one, we write back the cached values and reset 

The following table shows a selection of handlers, how they interpret the cached slots, and what they leave for the next handler.

Cache-field roles across selected handlers. Uncached operands come from the frame; a long occupies a payload slot and a padding slot, while instanceof requires an empty primitive cache.

 change across handlers, but we need no runtime type tests. The bytecode verifier establishes the operand types, and the cache protocol preserves their representation. The remaining question is how to use those bits without paying for a conversion.

, wide enough to hold the bits of any primitive value. Reading an 

 through an ordinary Java cast introduces a narrowing operation into the compiler graph. Depending on how it is lowered and allocated, that operation can introduce a register move and a separate register for the narrowed value.

For an integer operand, however, our protocol already places the value in the register’s lower 32 bits. We use a compiler directive to read those bits as an 

, without introducing a separate narrowing operation. The upper bits need not be zero; a 32-bit use simply does not read them. Writing an integer back into a cache field needs the matching treatment: preserve its low 32 bits without requesting Java’s signed widening to 

 leaves them unspecified. On x64, the 32-bit arithmetic already supplies the bits we need.

That leaves two more mysterious entries in the table: 

 value must reach the next handler unchanged, but passing it through requires no knowledge of its type.

, both fields have Schrödinger’s type: their bits have a meaning, but we have no reason to look. Matching incoming and outgoing ABI registers lets the allocator carry these values through without moves, provided it keeps those registers intact.

Passing the bits through is enough until we need to write them back to the frame. Then we must know how much stack space they occupy. Our current state machine avoids that ambiguity by counting slots rather than values: a fully cached 

 uses both cache fields, including its padding.

Caching two longs would require a state that tells us more. We will come back to this later.

 field need not preserve even its bits. For 

, we can pass whatever bits happen to occupy the register for 

. We express this freedom through compiler directives, allowing the allocator to reuse that register inside the handler without preserving or restoring its contents. We need not even put zero there before the next handler.

We have already complained about directives cluttering the source.

Fortunately, we can sweep this machinery under the 

 overlay’s carpet. The handler compilation knows the concrete stack implementation, so it can inline the overridden operations.

// Directive names in this illustrative example describe their intended contracts.
class CachedStack extends Stack {
    static final int NO_SLOTS_CACHED = 0;
    static final int ONE_SLOT_CACHED = 1;
    static final int TWO_SLOTS_CACHED = 2;

    long tos1;
    long tos2;
    int tosDepth = NO_SLOTS_CACHED;

    @Override
    void pushInt(Frame frame, int value) {
        switch (tosDepth) {
            case ONE_SLOT_CACHED -> {
                tos2 = GraalDirectives.packInt(value);
                tosDepth = TWO_SLOTS_CACHED;
            }
            case TWO_SLOTS_CACHED -> {
                UNSAFE.putLong(frame.prims, top * 8L + OFFSET_arrayStart, tos1);
                top++;
                tos1 = tos2;
                tos2 = GraalDirectives.packInt(value);
                // The cache stays full: 2 → 2.
            }
            default -> {
                assert tosDepth == NO_SLOTS_CACHED;
                tos1 = GraalDirectives.packInt(value);
                tosDepth = ONE_SLOT_CACHED;
            }
        }
    }

    @Override
    int popInt(Frame frame) {
        return switch (tosDepth) {
            case ONE_SLOT_CACHED -> {
                tosDepth = NO_SLOTS_CACHED;
                long result = tos1;
                tos1 = GraalDirectives.unconstrainedValue();
                yield GraalDirectives.unpackInt(result);
            }
            case TWO_SLOTS_CACHED -> {
                tosDepth = ONE_SLOT_CACHED;
                long result = tos2;
                tos2 = GraalDirectives.unconstrainedValue();
                yield GraalDirectives.unpackInt(result);
            }
            default -> {
                assert tosDepth == NO_SLOTS_CACHED;
                top--;
                yield (int) UNSAFE.getLong(frame.prims, top * 8L + OFFSET_arrayStart);
            }
        };
    }
}

static long iadd(long bci, Frame frame, Stack stack) {
    int right = stack.popInt(frame);
    int left = stack.popInt(frame);
    stack.pushInt(frame, left + right);
    return bci + 1;
}

Let us manually inline the two pops and the push in 

, keeping only the operations that affect 

static long iadd(long bci, Frame frame, CachedStack stack) {
    // First popInt.
    switch (stack.tosDepth) {
        case ONE_SLOT_CACHED -> stack.tosDepth = NO_SLOTS_CACHED;
        case TWO_SLOTS_CACHED -> stack.tosDepth = ONE_SLOT_CACHED;
        default -> { }
    }

    // Second popInt.
    switch (stack.tosDepth) {
        case ONE_SLOT_CACHED -> stack.tosDepth = NO_SLOTS_CACHED;
        case TWO_SLOTS_CACHED -> stack.tosDepth = ONE_SLOT_CACHED;
        default -> { }
    }

    // pushInt.
    switch (stack.tosDepth) {
        case ONE_SLOT_CACHED -> stack.tosDepth = TWO_SLOTS_CACHED;
        case TWO_SLOTS_CACHED -> { }
        default -> stack.tosDepth = ONE_SLOT_CACHED;
    }
    return bci + 1;
}

The previous article introduced handler specialization: give the incoming template variable a constant value and let constant folding do its job. For 

, each state update determines which arm of the next switch survives. The markers below trace that sequence:

static long iadd_2(long bci, Frame frame, CachedStack stack) {
    // Represents the incoming template constant, not a runtime assignment.
    stack.tosDepth = TWO_SLOTS_CACHED;

    // First popInt.
    switch (stack.tosDepth) {
        case ONE_SLOT_CACHED -> stack.tosDepth = NO_SLOTS_CACHED;
▶       case TWO_SLOTS_CACHED -> stack.tosDepth = ONE_SLOT_CACHED;
        default -> { }
    }

    // Second popInt.
    switch (stack.tosDepth) {
▶       case ONE_SLOT_CACHED -> stack.tosDepth = NO_SLOTS_CACHED;
        case TWO_SLOTS_CACHED -> stack.tosDepth = ONE_SLOT_CACHED;
        default -> { }
    }

    // pushInt.
    switch (stack.tosDepth) {
        case ONE_SLOT_CACHED -> stack.tosDepth = TWO_SLOTS_CACHED;
        case TWO_SLOTS_CACHED -> { }
▶       default -> stack.tosDepth = ONE_SLOT_CACHED;
    }
    return bci + 1;
}

Restoring the operand operations, constant folding and dead-code elimination leave:

static long iadd_2(long bci, Frame frame, CachedStack stack) {
    int right = GraalDirectives.unpackInt(stack.tos2);
    int left = GraalDirectives.unpackInt(stack.tos1);
    stack.tos1 = GraalDirectives.packInt(left + right);
    stack.tos2 = GraalDirectives.unconstrainedValue();
    stack.tosDepth = ONE_SLOT_CACHED;
    return bci + 1;
}

    ; regBCI:   incoming bci         (long)
    ; regFrame: incoming frame       (non-null)
    ; regPrims: incoming frame.prims (non-null)
    ; regRefs:  incoming frame.refs  (non-null)
    ; regCode:  incoming frame.code  (non-null)
    ; regTop:   incoming stack.top   (long)
    ; regTos1:  incoming stack.tos1  (long)
    ; regTos2:  incoming stack.tos2  (long)
    add     regTos1_32, regTos2_32              ; Java int addition
    inc     regBCI
    jmp     tail_call_target(regCode, regBCI, ONE_SLOT_CACHED)

, which fetch two operands or one operand from the backing frame, respectively.

The handlers describe bytecode behavior; the stack implementation supplies caching behavior wherever those operations occur.

Inlining brings them together, and constant folding derives each specialized variant. This is an aspect-oriented way to implement the optimization.

We still design the transitions in the stack abstraction, but need not calculate the outgoing 

 separately for every handler. The compiler fills in that column for us. That state selects the next already-specialized handler at the threaded tail call.

That only works while the compiler can determine the outgoing state on each path. Different branches may lead to different constant states, but invocation bytecodes are less cooperative: they may pop arguments in a loop whose iteration count depends on the target method. Our implementations flush the cache before that loop and finish with a known empty-cache state. Code that operates directly on the backing frame likewise needs the cached values written back first.

Flushing is less alarming than it sounds. These are the stack stores the uncached interpreter would have performed earlier; caching merely postpones them, and may eliminate some before a flush is needed.

Writing cached values back does not require allocating the overlay object. It can remain virtual, provided we do not let it escape through a call.

One Interpreter, Many Stack Implementations

Now we can change the stack implementation without rewriting the handlers. A different subclass can encode a different state machine.

Our first model spends two cache fields on one 

, including its padding, because the state counts slots rather than values.

, occupying two. We then have six populated states: 

, with pairs ordered from bottom to top. Seven, including the empty cache.

Each cache field now holds a complete payload. The state tells materialization how much space to reserve for it in the backing frame.

, we write two integers or floats and advance 

, we write two longs or doubles and advance 

 by four. The fields still extend the backing array, but the number of slots they represent depends on the state.

We can go further and distinguish the actual primitive types: 

. That lets the stack implementation choose a register representation as well as a slot width. We keep integer payloads in 

 fields for floating-point payloads, which can travel in XMM registers on x64. A 

 retains its raw bits rather than being numerically widened to a 

. Consecutive floating-point handlers can then operate on those payloads without shuttling them between general-purpose and floating-point registers.

The state determines which fields contain valid payloads and must be written back during materialization; unused representations must not be treated as stack values. The expansion configuration exposes that validity to the compiler, so those unused fields need not occupy registers either.

There is a bill for all this information. Four single-value states and sixteen ordered pairs, plus the empty state, give us twenty-one states. Each incoming state creates another handler variant to compile, and each viable variant contributes generated code.

Some handler/state combinations are impossible, and we can say so. The typed 

 rejects a cached operand whose kind is not 

 using an unreachable-path directive. For 

, the first pop succeeds, but the second encounters the cached double. Once specialization exposes that contradiction, the compiler propagates the unreachable marker back to the entry, discarding the preceding work and reducing the variant to a tiny trap. This reduces both the remaining compilation effort and the emitted code size.

To reduce the number of variants further, we can also choose not to cache some perfectly legal combinations.

For example, we could keep only the empty state, the four single-type states, and the four same-type pairs: nine states. A push that would create a mixed-type pair would instead write the existing cache and the incoming value to the frame, then return to the empty-cache state. 

, but we can still omit it and flush instead.

Whether we add states or drop them, the changes stay in the stack subclass and its expansion configuration.

The handlers still push, pop, and compute.

With those pieces in place, let us step back and see how far we have come.

 to remove checks and widen their indices. We 

 to make different tradeoffs for these tiny handlers. Finally, we 

, keeping more interpreter state in registers.

To see how these changes translate into performance, we selected Sieve and Mandelbrot from the 

suite. Both are arithmetic-heavy workloads that exercise the handler and operand-stack operations discussed above. All measurements are interpreter-only, with JIT compilation disabled.

Our starting point is Crema before threaded tail calls. For TOS, we compare the basic slot cache, the category cache, and the typed cache.

Threading gives both workloads a substantial boost, but neither catches HotSpot until we teach the compiler the handler-specific tradeoffs.

Two differences likely help here. We quicken these bytecodes for the array type; in particular, 

 no longer needs HotSpot’s byte-versus-boolean check. Our two-entry primitive cache can also keep both the index and value in registers, whereas HotSpot caches only the top operand.

 faster. Mandelbrot is heavy on double arithmetic: the basic slot cache holds only one double, while the category cache can keep both operands in registers.

The typed cache is the interesting twist. Although llvm-mca estimates 

 than the category-cache version for the interpreter instruction sequence corresponding to Mandelbrot’s hot loop, the typed cache runs slower in the benchmark. One possible culprit is code size: more handler variants mean a larger instruction footprint, potentially increasing pressure on the instruction cache.

Two arithmetic-heavy workloads do not settle the performance of an entire interpreter. They do show that readable Java and competitive interpreter code can coexist — even when getting there takes quite a bit of teaching.

We started by asking how close ordinary Java could get to the assembly we wanted. The answer depended on how we divided the work between the interpreter’s source code and the compiler. The source describes the execution model and supplies knowledge the compiler cannot discover on its own. The compiler turns that description into machine code, sometimes with policies tailored to the interpreter.

The harder question is where the design should live. When the design lives only in comments or in the programmer’s head, the implementation can drift away from it. Expressed through the right abstraction, it becomes something we can maintain and extend — and something the compiler can turn into efficient code.

Escape analysis deserves a credit of its own. Together with scalar replacement, it lets an object remain a useful source abstraction without requiring an object in the generated code. Its fields become independent compiler values that can live in registers. Argument expansion carries that idea across handler boundaries. The design stays in the source. Its objects need not survive into the assembly.

We still care about every load, move, and spill. We would just rather not write each one ourselves.

Chasing the Perfect Assembly (Part 2)

人工智能

Java解释器性能优化：通过MaxineVM项目展示如何利用栈抽象、模板变量和编译指令实现手写汇编级速度。文章深入探讨了类型化缓存、逃逸分析、参数扩展等技术，实现Sieve提速48%、Mandelbrot提速41%，证明可读Java代码与高性能解释器可共存。

文章

问答

视频

教程

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云智能顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

EdgeOne AI 安全实战专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

聚焦“写作效率、视觉美观与运行性能”三方面进行全面升级，为您提供更高效、稳定的创作环境

社区富文本&Markdown编辑器全新改版上线，欢迎大家体验!

诚挚邀请您参与本次调研，分享您的真实使用感受与建议。您的反馈至关重要，感谢您的支持与参与！

Chasing the Perfect Assembly (Part 2)

Chasing the Perfect Assembly (Part 2)

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐