Personal project · proof of concept

flowpilot

A proof of concept for AI-assisted customer support where a person approves every outcome.

FastAPI · Pydantic · SQLAlchemy · OpenAI API · React · TypeScript

view on github back to projects
FlowPilot request inbox with a list of customer requests and an analysis panel

overview

Support teams want AI to draft replies, but a wrong answer sent to a customer is expensive. FlowPilot explores a middle path: the model turns a customer message into a structured, validated analysis and an editable draft, grounded in company documents, and nothing happens until a person approves it.

It's a local proof of concept, not a support platform. The interesting part is the engineering around the model: strict output schemas, validation, retrieval with checked citations, safe failure states and an audit trail.

screens

Structured analysis of an invoice discrepancy: classification, priority and extracted amounts
Structured analysis: classification, priority, entities and intent, validated before saving.
Draft reply with a cited Refund Policy source and an approve button
Grounded draft: the cited policy excerpt sits next to the editable reply.
Request marked completed with a local ticket created after approval
Approval: only an explicit approval creates the ticket.
Knowledge base listing three policy documents and their index status
Knowledge base: documents are chunked, embedded and re-indexed on edit.
Activity log listing received, analysis and approval events
Activity log: every step is recorded, without storing message bodies.

Screenshots were taken in FlowPilot's built-in mock mode with its fictional seed data. Mock analysis uses keyword rules, not a language model.

How a request flows

  1. 01Customer messageValidated by FastAPI
  2. 02Retrieve contextTop 3 policy excerpts by embedding similarity
  3. 03AnalyseOpenAI Structured Outputs into a Pydantic schema
  4. 04ValidateSchema, citations and workflow state checked again
  5. 05Human reviewEdit the draft, see the sources
  6. 06ApproveTicket and audit event in one transaction

features

Structured analysis

Classification, priority, entities and intent come back as a strict schema. Missing facts are null, and no model self-rating is shown as a confidence score.

Grounded drafts

Retrieval finds up to three relevant excerpts from company documents. Drafts cite them inline as [S1], and the saved source snapshots survive later document edits.

Human approval gate

Analysis never approves itself. A person edits the draft and explicitly approves before a local ticket is created.

Audit trail

Received, analysed, edited and approved events are recorded with metadata only, not message bodies or credentials.

Works offline

A deterministic mock mode runs the whole workflow without an API key, and is clearly labelled as keyword rules, not AI.

Synthetic requests

A local generator creates realistic test requests, including questions the seeded policies can answer.

how it's built

Frontend

  • React, TypeScript and Vite
  • TanStack Query for server state; React Hook Form and Zod for forms
  • Inbox, review, knowledge base, activity log and configuration views

Backend

  • FastAPI with Pydantic v2 validation and consistent error responses
  • SQLAlchemy 2 and Alembic migrations on SQLite (PostgreSQL configurable)
  • OpenAI SDK: Structured Outputs for analysis, embeddings for retrieval
  • Pytest suite that never spends API credits

engineering

Validate twice, fail visibly

The SDK parses model output into Pydantic, and the workflow validates it again before saving. Refusals, malformed output, timeouts and rate limits leave a retryable failed state; they are never silently replaced with a fake success.

The database owns workflow state

Analysis claims a request with an atomic conditional update, so concurrent attempts get HTTP 409. A five-minute processing lease recovers from crashes, and a unique constraint guarantees one ticket per request.

Citations are checked, not trusted

The backend rejects citations to sources that were not retrieved and flags drafts that cite nothing. That proves a passage was retrieved, not that every claim follows from it, which is why the human review stays mandatory.

Untrusted input stays untrusted

Customer text and retrieved excerpts are labelled as untrusted data in the prompt, and the model has no tools or external actions available.

status

Works today

  • Request inbox, structured analysis, draft editing and approval
  • Knowledge base with retrieval and citation checks
  • Activity log, mock mode and synthetic request generator
  • Backend and frontend test suites

Not yet, or out of scope

  • Not production-ready: no authentication, tenancy, real inboxes, outgoing email or CRM integration.
  • Analysis runs synchronously in the request; there are no background workers.
  • Retrieval is in-process cosine similarity over SQLite, sized for a demo; thresholds need evaluation on real questions.
  • The approval flag is a workflow gate, not an authenticated reviewer identity.