Projects

Project Details

dutchbook — Prediction Market Coherence Monitor 2026

dutchbook dashboard: live status showing 14,951 markets tracked, 67,617 relation edges and a 17s median violation lifetime, above a breakdown of relation edges by source

Project Overview

A prediction market price is a probability, and probabilities have to obey arithmetic. "Bitcoin above $150k" cannot happen without "Bitcoin above $120k" also happening, so the first can never be priced higher than the second. Eight candidates of whom exactly one must win have to sum to a dollar. When prices break rules like these, the contradiction is a fact about the venue that needs no opinion about the world — and a set of bets exists that wins money whatever happens. That set is called a dutch book.

dutchbook extracts 67,617 such relations across 264,128 Polymarket markets — 85.4% of the active catalog — checks them continuously against a live order-book feed, and records how long each violation survives. The headline finding is a median lifetime of 15 seconds across 2,248 closed episodes: half of every logical inconsistency found was gone before you could finish reading about it, which is far too fast to be a person and says the correction is either automated or was never a correction at all.

The more useful half of the project is what it refuses to claim. Only 3.9% of episodes survived being priced against real depth and fees, and the 90 that did trace back to just two relations — both of which turned out to be bugs in this system's own extractor. The executable-arbitrage count in the report is therefore published as zero.


The Numbers

  • Catalog: 864,077 markets ingested, 14,951 active — 85.4% of the active catalog appears in at least one constraint.
  • Constraint graph: 67,617 relation edges and 24,201 partition groups — threshold ladders 65,249, nested deadlines 2,238, human-reviewed LLM proposals 128, explicit complements 2.
  • Lifetime: median 15s, p75 27s, p95 5.5m, max 3.6h over 2,248 closed episodes.
  • Apparent vs confirmed: 2,308 episodes, 90 survived live depth and fees — 3.9%. No partition violation was ever executable, because an n-member basket needs n simultaneous fills and pays the spread n times.
  • Detection latency: median 0.06s — the screen is event-driven off the CLOB WebSocket, not polled; a book update marks only the constraints that market participates in.
  • Dashboard: 6.7s cold, 90 ms warm after replacing a TTL cache that was pure overhead with stale-while-revalidate.

Every figure in the report is computed from the database at run time by pnpm report; none is written by hand, so a stale number cannot survive a regeneration.


The Analysis That Disproved Its Own Result

Median net edge on the confirmed violations was 30.66¢ per share, maximum 59¢. A risk-free thirty cents on a dollar-denominated contract, sitting untouched for sixteen seconds, is not something a venue with real participants leaves lying around — so the report said so before checking, and then checked.

The 90 confirmed episodes were 2 distinct constraints, each re-detected 45 times as it flickered across the threshold. Both were wrong. "OpenAI does not IPO by 2026" was recorded as entailing "does not IPO by 2027" — true for a positive event, backwards for a negated one. The second didn't even need the subtlety: a deadline of Dec 31 recorded as entailing one of Sept 30 is wrong on its face.

This was the second appearance of that class of bug. The first put an inverted direction on 888 markets — the ladder extractor read "hit 35%" as an upward threshold when Polymarket's own criteria resolve Yes at or below the value — and produced a $435 "risk-free arbitrage" that paid zero in one of three states. Nothing alerted either time, because nothing was broken: every component downstream did its job flawlessly on a false premise. A reversed entailment is the worst failure mode this system has precisely because it does not fail loudly — it produces a confident, well-formed, fully-priced answer.

That is also why LLM-proposed edges require a recorded human verdict before entering the graph. A missing edge costs an opportunity nobody notices; a reversed one costs money with total confidence.


What It Does Not Claim

  • The window is hours, not a month: 4.7 hours across two calendar days. Distributions have n=2,248 behind them and are worth reading; every rate, weekday effect and intraday pattern is unavailable and is reported as unavailable rather than estimated.
  • The size/lifetime hypothesis is unanswerable here: Spearman rho is +0.157 — larger contradictions lasted longer, and the sign holds inside each constraint type, so it is not Simpson's paradox. But 90.3% of episodes lasted under two 60-second intervals, so for nine in ten the recorded lifetime is the sampling rate rather than the market.
  • 15 seconds cannot distinguish arbitrage from jitter: either automated traders close these gaps faster than a human can act, or independently drifting quotes crossed a threshold and crossed back. The measurement cannot tell those apart, and the report picks neither.
  • The fee model has three named weaknesses: the taker rate varies 0.04–0.07 by a category no endpoint publishes, so the highest rate is applied to everything — deliberately under-confirming. The cost register is printed from source rather than restated, so an assumption cannot change without the table changing with it.
  • Coverage is of markets, not of logic: cross-event implications are almost entirely absent, and conditional or compound structure has no representation in the constraint language at all. Every "no violation" means "none among the constraints we know about".

Architecture & Engineering

  • Ingest is a reconciliation, not an append: each crawl hashes every market and writes one row per changed field to market_revisions, with old and new values. Markets that stop appearing are marked missing_since rather than deleted. The payoff is that the change log is evidence — if a resolution rule is silently edited after a market opens, the diff is in the database.
  • Idempotency is a tested property: running the same crawl twice must produce no new rows, no revisions and no moved hashes. The test runs against real Postgres because the properties under test are database properties, and the two runs use different clocks so the one column permitted to differ has genuinely moved. It is the load-bearing test in the repo.
  • Two stages, because a midpoint is not a price: stage 1 screens the whole graph on cached midpoints for nearly nothing; stage 2 fetches live books for the survivors, constructs the correcting basket, walks it level by level and charges fees and slippage. The two disagree 96% of the time and that disagreement is the product — the losers are stored as apparent with the reason.
  • Trades are priced at the profit-maximising size: found by ternary search, since profit is concave in size. Pricing at max executable size was a shipped bug that reported $0.00 on every confirmed trade — break-even by definition — and hid itself by falling below the alert threshold.
  • One rate limiter, because the budget is global: Polymarket limits per IP, so the bucket, Retry-After parsing and full-jitter backoff live in one module both clients import — guaranteed to agree by being the same code rather than by two copies that agree today.
  • It documents where it breaks: a load test at 10× the catalog (150,000 markets, 918,000 constraints) peaks at 1,124 MB against a 512 MB machine. It does not get slow, it gets OOM-killed — stage 1 materialises the whole graph per run, and the fix is to stream it in batches.

Project Information

Role

Solo Developer

Type

Market Data Infrastructure / Quantitative Analysis

Stack

TypeScript, Fastify, Postgres via Drizzle, BullMQ on Redis

Infra

Fly.io, Docker Compose, pgvector, testcontainers

Data Sources

Polymarket Gamma REST catalog, CLOB REST order books and WebSocket feed

Constraints

Implication, complement and partition — from threshold ladders, nested deadlines, neg-risk groups and reviewed LLM proposals

Tests

769, with database-backed ones on real Postgres via testcontainers

Scale

864,077 markets, 67,617 relations, 2,308 violation episodes

Links
Live DashboardGitHub RepositoryWrite-upFull Report