Data infrastructure
Autonomous acquisition
A self-healing adapter fleet reading 800+ independent sources nightly.
The system
The system ingests structured schedules from hundreds of sources that were never designed to be read by machines. Thirty-five adapter families cover the major platforms; a browser-driving fallback handles the long tail. The whole sweep runs unattended, every night.
When a source changes its markup, the adapter notices, repairs itself, and files the diff — maintenance is the product, so maintenance is automated.
The hard parts
No APIs exist
Of the long-tail sources sampled, zero exposed structured data. The fleet drives real browsers where it must and raw HTTP where it can.
Self-healing
Adapters detect schema drift and regenerate their own selectors; a model assist is reserved for the stubborn few percent.
Unattended by design
The full sweep runs nightly on a scheduler at 6× concurrency and completes in fourteen minutes.
In numbers
TypeScript · headless browsers · Firebase · scheduled functions
Happy to walk through this one properly — what it does, how it's put together, and what it would take to build something like it for you.