Style Match
Your closet, digitized — with real AI infrastructure behind it.
Overview
Style Match is an AI wardrobe app: photograph your closet and a multi-stage vision pipeline classifies, segments, and re-renders every garment as a clean studio flat-lay; a compatibility engine decides whether a new piece is worth buying against what you already own and suggests outfit combinations from your existing closet; and a virtual try-on pipeline puts any garment on a photo of you. It's currently in active internal TestFlight testing — I'm withholding the product name and screenshots until it's ready to launch, but the system behind it is fully built and running.
I own every layer: an async FastAPI backend (SQLAlchemy 2.0 + asyncpg on Postgres/pgvector) that reconciles five external AI providers into one job pipeline, an SQS worker fleet that keeps every expensive step off the request path, and the Flutter client with its native iOS share and action extensions. A single clothing photo can touch a cloud vision model, a segmentation model, a pixel-editing model, a locally-loaded embedding model, and on-device Apple Vision output before it ever lands in the closet.
Full stack
- Flutter · Dart 3 · Riverpod 3 (codegen) + Freezed
- FastAPI · Python 3.13 · Pydantic v2
- SQLAlchemy 2.0 async + asyncpg · PostgreSQL + pgvector · Alembic
- AWS SQS worker fleet · S3 · CloudFront · EC2
- Gemini · OpenAI · Replicate (segmentation, background removal, pixel-preserving edit)
- FashionCLIP + SigLIP embeddings · Apple Vision (on-device)
- Firebase Auth + FCM · Sentry · structlog + CloudWatch
- AWS CodePipeline/CodeBuild/CodeDeploy · Docker (split API/worker images)
Engineering highlights
The parts a code review would find interesting.
01A hallucinated jacket snap, and what it took to fix it
Asked to redraw a person-worn photo as a flat-lay product shot, the generation model painted snap buttons onto a plain welt pocket — because 'jackets like this usually have snaps' overrode what the reference photo actually showed. Root cause: it's a generative model, not an editor, and redraws from priors even with a reference image. A verify-and-regenerate loop helped, but the structural fix was routing fidelity-critical garments through a true pixel-level in-context edit model instead — which eliminated the hallucination in every test case, while a competing edit model made it worse. It shipped fully built but flag-off, gated on human review, because the automated verifier meant to judge the fix was itself caught hallucinating the same missing pocket.
02A six-round AI-vs-AI debate that killed three latency theories before a line of code changed
When the capture-to-polished pipeline stalled around a minute, instead of guessing at a fix, two AI models — Claude proposing changes, Codex attacking them — debated it out loud against real file-and-line citations. Over six rounds, each caught real bugs in the other's proposed diff: a token-limit cut that would silently drop an entire photo's detected garments, a background-job deferral that raced the client's actual save gate, a parsing fallback that would drop valid-but-unrecognized items. It ended in honest non-convergence: both sides agreed the fix got detection under ten seconds, but refused to claim a stretch target neither could actually hit — and said so in writing.
03An async worker fleet that heals itself
Every expensive step — embedding, cutout generation, matching, virtual try-on — runs off the request path through an SQS queue with a hard per-job timeout, so one hung API call can never freeze a worker. A second, independent loop periodically sweeps the database for anything stuck mid-job past a staleness window and re-drives it directly, catching every case where the enqueue itself silently failed — and every row that triggers a paid AI generation is atomically claimed first, so a queue redelivery racing that recovery sweep can never pay for the same result twice.
04Five AI providers, one closet item
A single clothing photo can touch a cloud vision model for tagging, a second model as a low-confidence escalation path, specialized segmentation and pixel-editing models for the cutout and flat-lay, locally-loaded embedding models for matching, and on-device vision output uploaded from the phone itself — all reconciled into one JSONB-tracked pipeline state machine. The request-serving API and the GPU-touching worker are built from separate Docker images specifically so none of the embedding models' dependencies ever ship to the process handling live traffic.
05On-device person segmentation, for free
Multi-person outfit photos were bleeding garment crops across people. Rather than add another paid cloud call, the fix runs entirely on-device: Apple's Vision framework segments each person into a silhouette mask locally on the phone, which the backend intersects with its own detection mask. Zero marginal cost, zero extra API calls, and it degrades invisibly to the prior behavior on devices where the on-device model isn't available.
06The vision model was already computing the answer, and it was being thrown away
Weather-appropriate outfit ranking ran on a coarse lookup table mapping garment type to a warmth score, blind to sleeve length or fabric weight — so a linen dress and a wool dress scored identically. The fix wasn't a new model call: the tagging pass was already reasoning about sleeve length and material to pick a season label, that reasoning just wasn't being saved anywhere. Persisting it upgraded ranking accuracy with no new cost and no behavior change for existing items.