Project Details
LeaseCheck — Lease Compliance Auditor 2025
Project Overview
LeaseCheck is a compliance auditor that analyzes residential lease PDFs to identify clauses that potentially violate landlord-tenant law. Users upload a lease and receive flagged violations, each with a specific legal rule citation and a plain-language explanation of why the clause is problematic.
It addresses a real gap: standard-form leases often contain unenforceable or illegal provisions — non-refundable deposits, blanket pet or guest bans, illegal entry terms — that tenants and housing workers cannot reasonably identify without legal expertise. The current scope is Ontario, Canada, evaluating clauses against provisions of the Residential Tenancies Act.
Processing Pipeline
Four-Stage Architecture
Ingestion:
Extracts text from PDFs with pypdf and segments it into clauses using numbered markers or paragraph splitting, preserving character offsets and ordinals.Retrieval:
Embeds each clause and performs vector similarity search against a rules corpus using pgvector, retrieving the k nearest legal rules by cosine distance.LLM Review:
Evaluates each clause against its candidate rules with Pydantic-constrained outputs, returning a structured verdict — ok, violation, or unclear — with a rule citation and rationale.Reviewer UI:
A Next.js app that renders the marked-up lease with margin annotations, colored verdict rails, and a triage workflow for accepting or dismissing findings.
Key Features
Analysis
- Jurisdiction-specific rules (Ontario RTA)
- Semantic clause-to-rule matching
- Structured, constrained LLM verdicts
- Hallucination guards on rule citations
Reviewer Experience
- Annotated lease with margin notes
- Colored verdict rails
- Interactive accept / dismiss triage
- Live violation counters
Technical Details
Rules Corpus
12 Ontario RTA provisions covering violations such as non-refundable deposits, blanket pet or guest bans, illegal entry terms, and NSF fees exceeding statutory caps. Each rule stores a code, title, plain-language description, and a 1536-dimensional embedding from text-embedding-3-small.
Retrieval & Review
Clause embeddings are matched to candidate rules via pgvector cosine search (default k = 4). The LLM may only cite rule codes that appear in the retrieved candidate set, preventing fabricated citations.
Evaluation Harness
A synthetic, hand-labeled Ontario lease fixture (24 clauses — 12 violations, 12 compliant) is run through the real ingestion and review pipelines. Metrics include precision, recall, F1, and attribution accuracy, with a k-value ablation over retrieval depths of 2, 4, and 8.
Project Information
AI / RAG Web Application
Ontario Residential Tenancies Act
Python, FastAPI, SQLAlchemy, Alembic, PostgreSQL, pgvector, OpenAI API, Next.js, TypeScript, Docker Compose
Retrieval-Augmented Generation, Vector Search, Structured LLM Outputs, Document Parsing
Results & Limitations
Fixture Performance
On the labeled fixture, the pipeline reached precision, recall, F1, and attribution accuracy of 1.000 ± 0.000 across all three retrieval depths, with zero unstable clauses.
Honest Framing
The perfect scores reflect the in-distribution nature of a single synthetic fixture and a small retrieval corpus, not validated real-world performance. Meaningful evaluation would require expanded rules, multiple authentic leases, and genuinely ambiguous cases.
A Triage Tool, Not Legal Advice
Output is meant to help a reviewer surface likely issues quickly; findings still require verification against primary legal sources.