Project Details
tape — Order Book Replay & Market Surveillance 2026

Project Overview
Market abuse is not visible in a trade print. Layering, spoofing and quote stuffing are patterns in the order book over time — orders placed to be seen and cancelled before they ever trade. By the time an event reaches a trade feed the evidence has already been deleted, so a surveillance system has to reconstruct the book from the raw message stream and put a human back at any microsecond of the day to look at it.
tape replays a full trading day of NASDAQ order flow message by message — 400,391 AAPL messages — reconstructs the limit order book, runs five market-abuse detectors over it, and streams the result to a browser viewer with a price ladder, a trade tape, and an alert queue. Clicking an alert seeks to 500 ms before the pattern and pauses, because at any speed above 1x it would be over before it could be watched.
The whole repository is built around one rule: same input bytes, same output book, on any machine, at any time. That is why the book core is pure — no clock, no randomness, no I/O — and why a finding is still defensible months after it fired.
The Numbers
- Binary ITCH 5.0 parsing: 100,280,207 msg/sec at 0 bytes allocated per message — measured with HotSpot's thread allocation counter and JMH -prof gc over the real 821 MB file, not asserted in a comment.
- Book reconstruction: 4,221,119 msg/sec — a full trading day in ~95 ms. Ingest to TimescaleDB runs 358,908 rows/sec.
- Seek to any timestamp: p50 9.78 ms, p99 17.29 ms — snapshot-anchored, so it never replays the day to get somewhere.
- Viewer at 1000x: 119.1 / 119.9 / 120.2 fps min/p50/max against 12,800–17,600 msg/sec, zero frames dropped.
- Accuracy: 400,391 comparisons against the vendor's own reconstruction, 0 divergences.
- Alerts, full day: 80 — layering 9, spoofing 26, quote-stuffing 13, momentum-ignition 32, marking-the-close 0.
Every figure is reproducible from the repository, on your own hardware, with a documented command — and each one names its method and machine.
What the Tests Actually Prove
The claim is reproduction, so the tests are built to be able to fail. A fixture generated from this engine's own output would pass however wrong the engine was — so none of them are.
- Checked against the vendor, not itself: the golden test replays all 400,391 messages and compares the top 9 levels against LOBSTER's own orderbook file after every single one. The browser suite asserts against a fixture derived from that same vendor file by script.
- It proves exactly nine levels, and says so: a level-10 extract is a windowed view, so the 10th level is not reconstructible by anyone from this feed. Measured: 189,448 transitions fully explained by their message, 210,942 changing only the deepest visible level, and 0 unexplained changes inside the window.
- The ITCH parser is checked against an independent decoder: every field of the first 100 messages of all nine handled types asserted against a Python decoder written from the same specification and sharing no code — then a whole symbol's day replayed into both books, all 203 price levels matching on both sides.
The AI Layer Explains; It Never Decides
Detection is arithmetic over the message stream and the reconstructed book — no model, no clock, no randomness. An alert is an accusation, and the thing that decides whose order flow gets looked at should be a rule a person wrote, can read, and can be argued out of. "The model flagged it" is not a reason; a threshold with a documented false-positive section is.
So the model gets the job rules are bad at: turning a jsonb payload of order ids and half-ticks into a sentence a reviewer can read at 4pm on a Friday. Narratives are cached, cost-tracked, and never gate an alert — the evidence is complete whether or not a narrative was ever generated.
A complexity router sends routine alerts to Claude Haiku 4.5 and the hard ones to Claude Opus 5: 65 simple and 15 complex narratives came to $0.126 for the day against $0.363 all- frontier — 65.2% saved, at $0.00158 per alert. The router emits a complexity class rather than a model name, so the routing decision stays auditable independently of which model is wired to it, and cache savings are reported separately so routing is never credited with money it did not save.
What It Does Not Claim
- The detectors do not identify manipulation: they identify shapes manipulation tends to leave, and every one has a legitimate twin. A market maker repricing a ladder is the shape of layering; cancel-and-replace is the shape of spoofing — a replacement test cut that from 138 alerts to 26 without eliminating it.
- The thresholds are not calibrated to any regulatory standard: they were fitted to one criterion — a reviewable queue on one day of one symbol, about twelve alerts an hour. Every sweep and what it cost is documented.
- There is no participant identifier: LOBSTER carries none, so a "stack" of four orders may be four unrelated traders. Real surveillance runs on account-attributed data; this runs on anonymous flow, which caps what any alert can claim.
- Single symbol, single day, single venue: AAPL, 2012-06-21, NASDAQ. No cross-venue view and no NBBO context — an order that looks like a spoof here may be one leg of a legitimate strategy somewhere this system cannot see.
Architecture & Engineering
- Dependencies point inward only: api → adapter → port → model → book. The pure core compiles if Spring is deleted from the build; the clock is a port, so replay speed is a parameter rather than a wall-clock dependency.
- Zero-copy ingest: the ITCH reader is a memory-mapped cursor rather than an iterator, which is what makes the per-message allocation count zero — and it refuses a file over the 2 GB FileChannel.map cap rather than silently reading a prefix.
- Streaming path: TimescaleDB holds messages, snapshots and alerts; a replay engine serves SNAPSHOT / DELTA / TRADES over a WebSocket with a REST control plane for play, pause, seek and speed.
- One command to a running stack: ./scripts/stack.sh up builds the jar and bundle, starts four containers, ingests 400,391 messages, runs the detector pass and waits until the session is queryable. Market data is never redistributed — the fetch script pulls from the vendor with SHA-256 verification before anything is written.
- CI in under ten minutes: six independent jobs in parallel, including the full compose stack with the browser and load suites.
Project Information
Solo Developer
Market Data Infrastructure / Surveillance
Java 21, Spring Boot, JMH, Testcontainers
React, TypeScript, Vite, Zustand, AG Grid
TimescaleDB, Docker Compose, LOBSTER CSV and NASDAQ TotalView-ITCH 5.0
Layering, spoofing, quote-stuffing, momentum-ignition, marking-the-close
Claude Haiku 4.5 / Opus 5 narrative router, cached and cost-tracked
400,391 messages replayed, 0 book divergences, 80 alerts