An AI-powered personal memory engine.
- Project Loci
- Year 2025
- Role Lead Developer
- Stack React, FastAPI, YOLOv8, CLIP
Background
We all misplace things. The mental overhead of remembering where everything is grows with every drawer, shelf, and bag. Existing solutions are just note-taking apps — they require manual effort and discipline that nobody sticks with.
Loci changes that. Photograph items, and the app automatically tags, indexes, and makes them searchable using natural language. No manual tagging, no folders, no effort.
Challenge
Build a mobile app that can detect objects in photos using AI, generate semantic embeddings for natural language search, work entirely offline, and sync seamlessly when reconnected — all without external cloud services or recurring costs.
The app needed to feel native on Android while being built with web technologies. Two ML models had to run on the backend without expensive GPU infrastructure.
How it works.
Photograph anything
Take a photo of your keys on the kitchen counter. Loci automatically detects the objects using YOLOv8 and generates semantic embeddings with CLIP, storing everything in a searchable index.
Search naturally
Type "keys" or even "metal things in the kitchen" — semantic search finds it. The query text is encoded with CLIP, then cosine similarity is computed against stored image embeddings. No exact keywords needed.
Works offline
Full IndexedDB queue with background sync. Mutations are queued while offline and replayed on reconnect with careful ordering and conflict resolution. The app never feels broken without internet.
Share and collaborate
Organize memories into collections with multi-user sharing — read, write, and admin permissions. Generate QR codes or time-limited share links for individual memories.
Under the hood.
| Mobile | React + Capacitor (Android) |
| Backend | FastAPI + SQLite |
| Object Detection | YOLOv8 (nano) |
| Semantic Search | CLIP (ViT-B/32) via sentence-transformers |
| Auth | Custom JWT + bcrypt |
| Offline | IndexedDB + Service Worker |
Architecture
Features.
What I built.
47-endpoint REST API
Full API with JWT authentication, batch operations, analytics, and a sharing system — all built from scratch with FastAPI and SQLAlchemy.
Offline sync engine
IndexedDB-based mutation queue with conflict resolution. Handles create, update, and delete operations while offline, replaying them in order on reconnect.
Semantic search pipeline
Embedding generation, vector storage, and cosine similarity matching. Searching "red bag" finds photos of red bags even if they were never tagged with those words.
Android widget
Native home screen widget with SharedPreferences bridge, providing quick access to memory count and instant search from the Android launcher.
Challenges.
Removing third-party auth
Initially used Supabase for authentication. Replaced it entirely with a custom JWT + bcrypt implementation to eliminate external dependencies, reduce costs, and own the full auth flow.
Offline-first on mobile
Queuing mutations in IndexedDB while offline and replaying them on reconnect required careful ordering and conflict handling. Edge cases around delete-then-recreate and concurrent edits took the most time.
Running ML on a budget
Both YOLO and CLIP run on the backend server, not on-device. Kept it lightweight by using YOLOv8n (nano) and CLIP ViT-B/32 — small enough to run on a single CPU instance.
What's next
Search by photo using reverse image lookup with existing CLIP embeddings. OCR for documents and receipts. GPS-based location auto-detection. Multi-image per memory. Play Store release.