Kotoba native performance: Embench progress, updated 13 October 2026

Current closeout: paused, objective unmet

Optimization stopped on 13 October under the approved budget condition. This publication and handoff do not resume it. No new candidate was adopted after main PR #1295. Latest complete main Round85 remains 2.954× Kotoba time / C; C is faster overall.

Adopted: the complete previously qualified compiler closure bounded byte lookup (PR #1293) and descriptor reuse across two certified scalar calls (PR #1295) are integrated in main. Earlier adopted passes include masked XOR private literal-width specialization call-result comparisons stack operand reuse signed division by two and cached bounds checks. Source and artifact hashes rather than this abbreviated list identify the product. There is no minimum5% improvement cutoff.

Unadopted: all seven two-argument fuel-cache variants are preserved as reproducible source patches with hashes and verification status. Older broad descriptor-witness and conditional-fallthrough candidates retain their historical archives. Their results are excluded from the adopted-main full19 table.

Unadopted candidate measurements — ON/OFF, not Kotoba/C

Each row below is a separate program-image probe on Joseph: fixed OFF/ON/OFF/ON one warmup and one accepted sample per slot raw pre/post and background CPU idle each at least70% first CPU-qualified sample and at most3 CPU-only attempts. Ratios are candidate time / adopted-baseline time.1.00 is equal; larger is slower. They are not C comparisons full19 scores or statistical effects.

V3: SG was faster in both pairs (median about3.96% shorter); Pico pairs were mixed (median about1.68% longer). V4 did not establish an SG gain; V6 SG was slower in both pairs (median about3.86% longer). V5 generated all39 baseline-identical images and was not timed. No gain is obtained by adding percentages across these different probes.

Latest preserved candidate: v7, still unadopted

V7 restores exact returned-value comparison copy elimination across a fuel reload. A new positive fixture removed one instruction;80 pairs across10 fuel levels preserved full results fuel and all17 arena counters. Three native self-build generations converged. All39 stable program images and containers are identical to V3 and the initial V7 producer so the original95+76+312=483 observations are inherited by complete image/source/export identity not relabelled as fresh V7 executions.

Self-build instruction/reference/pool/export mapping fuel-cache and scratch-register CFG checks,121 removed context reload sites and context-read checks for74 cached compiler roots passed. These are source/physical/correctness evidence. Formal policy/public/source checks further generic boundary/refusal controls and a fresh complete original19 C comparison with adoption review remain pending. V7 has no new full19 performance score release or adoption claim.

The archive manifest six independent source patches against the pinned V3 base preserved verifier source and handoff document identify the next steps. Raw native artifacts audit traces and failed attempts remain in the original local workspaces; a source archive does not claim remote availability of all raw evidence.

Latest main integration and Round 85 — 13 October 2026

PR #1295 merged as 74b082f534d6326a5d5df9a0f415be1425b85aa6. All 19 CI checks passed; remote main ancestry, all 21 changed files, the 16-module source and native compiler artifact were read back and verified.

The compiler reuses a checked vector descriptor after two fully certified scalar calls. Every read resets the count and tracked-slot writes invalidate it. Original index checks and traps remain; admission does not use workload names. The two picojpeg images retain the previously observed 1.8930% shorter fixed-slot timing, while the other 37 images are byte-identical to previous main. This individual exploratory observation is not a total-suite gain.

Qualification passed three native self-build generations, 39 fresh images, the original 483 result/fuel/state observations, 54 generic pairs, 52 lookup controls and 34 policy refusals. There is no minimum 5% improvement cutoff.

Fresh Round 85 accepted all 19 Kotoba and all 19 C measurements across 40 retained attempts. Normalized Kotoba execution time / C is 2.9542453261184902 (2.954×). Driver geometric scores are 12729.292501173408 for translated Kotoba and 37605.452876386684 for original C.

The previous Round 84 ratio was 2.876720502080424. The larger ratio in this single cycle does not establish an overall improvement or a stable regression. C remains faster overall. Optimization is paused; the C-or-faster objective remains unmet. Raw CPU idle before/after and estimated background idle must each be at least 70%; only CPU-condition retries are allowed, at most three, selecting the first qualified sample rather than the fastest. The historical Round 84 and Round 83 records follow.

Round 85: all 19 workload ratios

History — resumed work and verified Round 84

Work resumed after the paused snapshot below. Its Round 83 table and dated pause history are retained; they do not describe the resumed compiler or Round 84. The C-or-faster goal remains unmet.

A generic bounded byte lookup replaces eligible pure scalar threshold trees while preserving full 64-bit inputs, original fuel, traps and workload. Original qualification covered 39 native images and 483 comparisons; the Round 84 main composition reproduced all 39 qualified images byte for byte.

Four fixed OFF/ON samples per workload observed 2.0851% shorter PicoJPEG time and 0.7699% shorter QRduino time. These exploratory two-pair observations have no uncertainty estimate and are not a combined or total-suite gain. There is no minimum 5% improvement cutoff; correctness and regression checks still apply.

Main PR #1293 merged as 9bdff08f95effda253a08b2eed190efe98af55fc; remote main ancestry, source and native-image bindings were verified. Independently verified Round 84 accepted all 19 Kotoba and all 19 fresh C measurements across 41 retained attempts. Normalized Kotoba time / C was 2.876720502080424 (2.877×); C remained faster overall. Driver geometric scores were 12916.907866571351 for translated Kotoba and 37158.333683249715 for original C. This single exploratory cycle does not establish a statistical improvement over Round 83. Translated Kotoba measurements remain exploratory and are not certified official Embench scores.

Round 84 workload comparison

How to read the numbers

1.000× means equal execution time. Above 1.000× means Kotoba takes longer and C is faster; below 1.000× means Kotoba is faster. The total is the geometric mean across all 19 workloads, not the arithmetic mean. CRC32 is the one workload with a ratio below one in this cycle.

What was verified

At the earlier Round 83 pause, the adopted native compiler was local source commit fc49cb2ee6fea1a41a5abbc8ed5b52aaa9f19471. Main integration and release publication had not occurred at that snapshot. This result does not describe the released CLI.

The original 19 workloads and 95 profile cases were preserved. Qualification covered 39 native images and 483 paired comparisons, including fuel boundaries and interaction cases. Original results, state and resource accounting were checked. Compiler self-hosting and additional generic controls were also verified. Correctness qualification alone does not establish performance.

Measurement method and limits

Both arms were timed on the same macOS arm64 host; the C arm uses Apple clang 17.0.0. Upstream Embench revision 09c2ed8c3b7008c95d08b038de4a3f6dc103ed70 supplies the untouched speed driver and score formula. The C executables were previously frozen and freshly timed in this cycle.

Kotoba uses GSF 8192 and C uses GSF 32768. The driver normalizes each arm by its declared global scale factor. The ratio is C relative-speed score divided by Kotoba relative-speed score: 37345.764989 / 12718.199531 = 2.936403. These raw scores are host-specific and are not scores per MHz.

CLOCK_MONOTONIC_RAW brackets the workload body. One warmup body, resets and verification occur outside that interval; representation allocation inside the body remains timed. Each language has one accepted sample per workload. CPU admission requires at least 70% raw idle before and during timing, plus 70% background idle during timing. CPU-only retries retain all attempts and select the first admitted sample, not the fastest.

This is one exploratory measurement cycle without a statistical confidence claim. Translated-port representation and execution paths differ from original C; using the official driver and formula does not turn the Kotoba port into a certified, unmodified official submission. Historical partial measurements are excluded from this total.

Changes adopted locally

The current compiler combines optimizations for masked XOR, private literal-width calls, forwarding call results into comparisons, repeated stack loads after disjoint scalar stores, signed division by two, and reuse of successful cached vector bounds checks. Each adopted change retained the required correctness and ABI checks.

There is no minimum improvement percentage cutoff. Even small validated improvements can accumulate. For example, a prior picojpeg change with an observed 0.085% reduction was adopted locally; that small-sample observation is not a statistically established effect. Per-change observations come from different runs and must not be added into a total speedup.

Work preserved at the pause

Descriptor-bounds witness reuse and conditional-fallthrough vector-view reuse each passed the full 39-image and 483-comparison correctness checks against the adopted baseline. They remain unadopted: additional generic or semantic controls, performance measurements and adoption review are pending. Their candidate results are excluded from the measured table.

At that pause, the statemate timing files had been transferred and custody checked, but the timing campaign had not started. The bounded byte-lookup candidate was then a source prototype, with compiler execution and runtime qualification pending.

At the earlier snapshot, optimization agents and the C-or-faster goal were paused. Evidence, worktrees and unfinished candidates are retained for a deliberate future restart. No performance goal completion is claimed.