Enterprise document verification for UK compliance: extract structured data from PDFs, verify against official sources (Companies House, EPC, Land Registry), and keep a full audit trail with human-in-the-loop resolution.
Get oriented
Problem → product flow → outcome. Short walkthrough—not a code review.
Teams in regulated workflows cannot safely treat “the model read the PDF” as evidence. Generic RAG or summarisation gives fluent answers without deterministic checks against authority sources, traceable decisions, or a clear path when data disagrees. For UK-facing operations, proving consistency with Companies House, EPC, or Land Registry-style references—and showing your working to auditors—is as important as speed. Silent failures and missing audit trails create legal and operational risk.
Regulators and auditors ask for proof: what the document claimed, what you checked it against, and who signed off when something didn’t match.
RAG can sound confident while ungrounded; the system needs deterministic checks where the schema allows, traceable pipeline stages, and idempotent async work—not a single opaque completion.
The browser and API stay responsive: files go to object storage, heavy work runs in background workers, and every meaningful step can be traced in the database—including when a human must decide. Under the hood that maps to concrete services and queues, not a single monolithic script.
AVAE is a verification-first pipeline: ingest documents, run structured extraction and orchestrated reasoning, compare claims to trusted external checks where integrated, persist results and decisions in a durable audit model, and route mismatches to human review instead of hiding them. The product is built as an MVP with a clear separation between a fast API surface, object storage for files, asynchronous workers for heavy processing, and a Next.js dashboard for outcomes and review.
In plain terms: the browser talks to an API, files land in object storage, and heavy processing runs in background workers so uploads stay fast. Verified outcomes, audit metadata, and graph checkpoints live in PostgreSQL; Redis backs operational state like progress and rate limits.
The system uses a Next.js frontend (with Clerk authentication) talking to a FastAPI backend. Documents are stored in Amazon S3; background processing is driven by AWS SQS and workers (with Redis used for caching, rate limiting, and task state). The processing pipeline uses LangGraph for multi-step orchestration with Postgres-backed checkpoints (including human-in-the-loop flows), PostgreSQL with pgvector for retrieval where needed, and LlamaParse / LlamaIndex-family tooling plus traditional PDF libraries for parsing. External verification hooks include UK-oriented integrations such as Companies House, EPC Open Data, and Land Registry–style checks as configured. Optional Streamlit UI exists alongside the main Next.js app for experimentation.
Short email works—mention regulated PDFs, verification, or LangGraph pipelines. Or watch the overview first, then reach out.
Building an auditable path from raw PDF to verified fields—without relying on a single black-box summary—while keeping latency and cost manageable for an MVP.
I combined structured parsing (LlamaParse / LlamaIndex and fallbacks like PyMuPDF, pdfplumber) with explicit pipeline stages in LangGraph so each step can be logged, retried, and inspected. Checkpoints in PostgreSQL support resumable runs and human-in-the-loop handoffs when the graph detects uncertainty or mismatch.
Takeaway: Every step is inspectable; the graph—not one black-box completion—owns resumability and HITL.
Connecting document claims to external ground truth (UK registries and datasets) in a way that is configurable and traceable in the audit log.
I integrated verification modules against official or open APIs (e.g. Companies House, EPC Open Data, Land Registry–oriented flows per environment keys) so results record what was queried, what matched, and what failed—feeding both automated decisions and review queues.
Takeaway: Ground-truth checks are logged as first-class outcomes, not side comments in a chat transcript.
Running CPU- and IO-heavy extraction and LLM steps without blocking uploads or the interactive UI.
The FastAPI surface accepts work quickly while SQS-backed workers perform the heavy lifting; Redis supports progress and operational concerns like rate limits. This pattern keeps the web tier responsive and allows scaling workers independently.
Takeaway: Heavy work stays off the request path; the API stays fast while workers scale.
Storing embeddings and structured outcomes in one place while keeping migrations and vector use maintainable.
I used PostgreSQL with SQLAlchemy and Alembic migrations, pgvector where vector search is required, and clear separation between application tables, audit-oriented records, and LangGraph checkpoint storage (including langgraph-checkpoint-postgres).
Takeaway: One Postgres estate for app data, vectors, and graph state—migrations stay coherent.
Layer → choice → why—faster to scan than a flat tag list.
| Layer | Technology | Why |
|---|---|---|
| Frontend | Next.js, React, TypeScript, Clerk, TanStack Query | Dashboard, PDF viewing, authenticated sessions, and API-driven UI state. |
| API | FastAPI, Python, Pydantic | Fast request handling, uploads, job lifecycle, and integration boundary for the worker tier. |
| Async & workers | Amazon SQS, Celery, Redis | Heavy extraction and LLM work off the request path; progress, limits, and task state. |
| Orchestration & AI | LangGraph, LangChain, OpenAI, LlamaParse, LlamaIndex | Multi-step pipelines, checkpoints/HITL, parsing, and embeddings-backed retrieval where needed. |
| Data | PostgreSQL, pgvector, SQLAlchemy, Alembic, LangGraph checkpoint store | Structured outcomes, vectors, migrations, and resumable graph state. |
| Storage & cloud | Amazon S3, AWS SDK, Docker | Document storage, deployment, and environment parity for local vs cloud. |
Honest boundaries: what exists today vs what comes next—no fake ship dates.
Insurance, legal, property, or similar high-stakes evidence—email, use the form below, or replay the overview in the hero. A short call is enough to see whether this architecture fits.