Talha Malik
0
CV
Case Study · FalconXoft

Hawk-Eye

A computer vision platform that tells the difference between a customer and an intruder — from a 14px face on a low-bandwidth camera feed.

About

Built for a dealership and facility-security client. Two products share one backend: Hawk-Eye for daytime dealership traffic and customer interaction tracking, and Night-Hawk for after-hours intrusion detection.

FACE RECOGNITION ✦ EDGE + CLOUD ✦ ARCFACE EMBEDDINGS ✦ REAL-TIME CV ✦ FLEET HEALTH ✦ FAILURE ANALYSIS ✦ FACE RECOGNITION ✦ EDGE + CLOUD ✦ ARCFACE EMBEDDINGS ✦ REAL-TIME CV ✦ FLEET HEALTH ✦ FAILURE ANALYSIS ✦
What it does

Real time, on hardware you can't trust the internet for.

Hawk-Eye watches live camera feeds across dealership locations, detects people, and recognises whether they're enrolled staff or an unknown visitor — in real time, on hardware that sometimes has to run over an unreliable internet connection. Same recognition core, two operating modes, one backend.

Architecture

Hybrid edge and cloud.

Camera hardware sits on-site; the recognition model runs on GPU in the cloud. Between them is the actual engineering problem: a dealership's internet is not a data centre's internet.

The edge agent

Runs as a supervised, self-updating service on-site. A camera worker detects people locally and buffers frames rather than streaming everything upstream. A clip extractor compresses buffered footage into short .mp4 files. A cloud uploader — backed by a local SQLite queue — pushes clips to storage with exponential backoff, so a dropped connection queues rather than loses data. A heartbeat thread reports agent health every 60 seconds. An updater applies new versions with a stop-swap-assert-rollback sequence, so a bad update doesn't strand a camera nobody is visiting that week.

The cloud backend

The control plane: a Django service issuing signed upload URLs, tracking fleet health, running the GPU-backed recognition pipeline, and serving the API both products are built on. Subscriptions, billing, and access scopes are shared across Hawk-Eye and Night-Hawk through one data model, so a dealership can hold either product, or both, without duplicated infrastructure.

The recognition core

Enrollment is where accuracy is won or lost.

Face recognition runs on InsightFace's buffalo_l (ArcFace), producing 512-dimensional embeddings compared by cosine similarity against enrolled staff.

Early on, every valid staff photo was averaged into a single embedding — collapsing useful variation in angle and lighting into one blurred reference point. The enrollment pipeline now stores one embedding per accepted image plus a centroid, with each incoming photo passed through a quality gate: exactly one face, confidence above threshold, size, blur, brightness, and pose angle all checked, with a human-readable reason logged for every rejection rather than a silent drop.

A system that rejects bad enrollment photos without saying why looks, from the outside, exactly like a system that's silently broken. Both failure modes appeared at different points in this project — and the second is the harder one to catch.

The incident

Two months of silence. No alarm.

A dealership stopped generating detection events. No errors. No alerts. Just quiet — for two months, until someone noticed. The investigation was a full bisection: GPU pod, embeddings API, service tokens, clip storage, presigned URLs, camera records, DNS, model versions, task queues. Every layer came back healthy. That's what made it take a full day — the fault wasn't in any single component.

The actual cause

The client's camera captured faces at roughly 14 pixels. The recognition pipeline's quality gate required 40. Every face for that dealership had been silently discarded at the door for two months. A comparison against a working dealership on identical hardware and code made the gap undeniable.

14px
Faulty dealership
vs.
154px
Working dealership

Two separate defects made it invisible. The size gate discarded frames with no counter, no warning log, and no signal anywhere in the pipeline's output. There was no way to distinguish "no one visited" from "everyone was rejected." A single funnel metric — frames in, faces found, rejected by size, published — would have surfaced this within hours instead of months.

Deeper still: the endpoint that queues a clip for processing returned success before the clip was actually processed. Twenty-two thousand clips were marked complete, and that status proved only that a request had been received — nothing about whether recognition had actually run.

A related defect: an edge agent's startup sequence had one unguarded initialisation step. If it threw, the camera-processing thread died — while a completely separate heartbeat thread kept reporting the agent as healthy. The agent could be dead and green at the same time, with no server-side signal able to tell the difference.

That's a harder class of bug than an outage: it's an outage that actively insists nothing is wrong.

The question behind the incident

Can upscaling fix a 14px face?

It's a reasonable idea and it's wrong — and proving that rigorously mattered more than the answer itself. The 40px gate isn't the actual requirement; it's a proxy for "enough real detail to distinguish one person from another." A model that hallucinates plausible-looking pixels can satisfy the proxy without satisfying the requirement. Worse, it does so with confidence — a wrong match that looks sharp is more dangerous than an honest rejection, because nobody would think to question it.

Every result was scored by cosine similarity against a real enrolled embedding — never by whether an image merely looked sharper. A known-identity face was shrunk to 14px, then upscaled with plain bicubic interpolation as a baseline, and separately with Real-ESRGAN at multiple settings, each paired against a bicubic control at the identical output resolution — the only way to separate "the image is bigger" from "the AI added real signal."

Bicubic (baseline)
0.496
cosine similarity
Real-ESRGAN
0.228
less than half the baseline
Real detail needed
~60px
to cross the recognition threshold

Super-resolution didn't add identity signal; it destroyed more than half of what little was there, replacing real (if blurry) detail with plausible-looking noise. A face-specialised enhancement model, given a fair test after an earlier run silently failed to even detect a face in the 25×32px input, performed slightly worse still. No synthetic upscaling method got close to real optical detail. The fix was never going to be a smarter model downstream — it was telling the camera to get closer, or accepting that some footage simply isn't usable for identification.

When a metric can be gamed and a model can game it, testing against the real downstream signal — not the proxy, not "does it look better" — is the only test that means anything.

Stack
Django · PostgreSQL · Celery · Redis · InsightFace (ArcFace, buffalo_l) · ONNX Runtime · FAISS · PyTorch · FFmpeg · Docker · Kubernetes · AWS S3 / Supabase Storage · Real-ESRGAN, GFPGAN (evaluated) · Django · PostgreSQL · Celery · Redis · InsightFace (ArcFace, buffalo_l) · ONNX Runtime · FAISS · PyTorch · FFmpeg · Docker · Kubernetes · AWS S3 / Supabase Storage · Real-ESRGAN, GFPGAN (evaluated) ·
What this project is really about

A recognition system is only as trustworthy as its failure mode. Every fix in this project, from per-clip funnel metrics to rejecting a plausible-looking but wrong fix for low-resolution faces, served the same principle: measure the real thing, not a proxy for it — and never let "no errors" stand in for "working."