Vision Lock runs YOLO-World, BoT-SORT, SigLIP 2, and MediaPipe through a FastAPI WebSocket service and a typed React scanner. It tracks several objects at once, keeps labels attached during motion, supports click and gesture acquisition, and reports real latency, throughput, dropped frames, CPU, memory, and GPU telemetry without persisting camera data.
Stack, status, evidence, and public actions are rendered from the typed project record.
Category
AI Systems
Type
System
Priority
Strong
Overview
What this project is
Vision Lock runs YOLO-World, BoT-SORT, SigLIP 2, and MediaPipe through a FastAPI WebSocket service and a typed React scanner. It tracks several objects at once, keeps labels attached during motion, supports click and gesture acquisition, and reports real latency, throughput, dropped frames, CPU, memory, and GPU telemetry without persisting camera data.
Problem
Why it matters
Most object-detection demos stop at one model call. They do not let an operator change the vocabulary, preserve labels through motion, acquire a target, inspect a second ranking model, or understand where latency is spent.
Solution
Approach
A local-first vision pipeline that combines runtime-defined detection, temporary multi-object track IDs, independent locked-crop ranking, gesture state, bounded WebSocket streaming, and live resource telemetry behind one polished browser interface.
Architecture
System shape and stack
Architecture
One bounded real-time path
The browser permits one frame in flight. FastAPI routes it through separate detector, tracker, ranker, gesture, and resource-monitoring responsibilities, then returns one typed result for the overlay.
Interaction
Labels that move with the scene
YOLO-World accepts a text-defined object vocabulary, BoT-SORT assigns temporary track IDs, and click or gesture commands lock a target whose crop is independently ranked by SigLIP 2.
Evidence
Measured performance without invented accuracy
A reproducible CC0 fixture and benchmark CLI record warm latency distributions, throughput, resource use, and aggregate tracking counts. The project explicitly makes no accuracy claim without labeled ground truth.
- Python
- PyTorch
- OpenCV
- FastAPI
- WebSocket
- YOLO-World
- BoT-SORT
- SigLIP 2
- MediaPipe
- React
- TypeScript
- Docker
Technical Highlights
Visible technical signal
- Four simultaneous detections shown in the canonical real-model run
- One-frame-in-flight protocol that bounds backlog and counts dropped capture intervals
- 11.239 FPS measured throughput with 64.294 ms p50 end-to-end latency on the recorded RTX 4060 run
What It Proves
Builder signal
End-to-end computer vision engineering across model integration, real-time protocols, tracking state, frontend overlays, privacy defaults, reproducible performance measurement, and honest research reporting.
Boundaries
Context that should stay visible
The recorded workspace fixture is derived from a Wikimedia Commons photograph reviewed under CC0. Full provenance is documented in the repository.
The public domain hosts a recorded local run. Live camera inference stays local because the model stack is not deployed as a public GPU service.