Consumer Marketplace ·Confidential, US consumer platform
Adding a city stops meaning months of parsers, because the pipeline reads a page it has never seen before.
A local culture discovery platform whose completeness is the product. Venues have no API, independent promoters have a social page, and every listing uses its own markup.
Where this fits
Best for marketplaces whose catalog lives on other people's websites, where one parser per source is a ceiling on how fast you can grow.
14k+ events ingested, none through a site-specific parser. Around 95% extraction accuracy.
- 6+ months in production
- 2 US cities live
- Mismatches go to human review
What we did
- Built an ingestion pipeline that reads any events page without a parser
- Extracted title, venue, date, price and links into one structured record
- Validated and normalised every record before it reaches the feed
- Routed mismatches to human review instead of into the database
Result
A new source is added by pointing the pipeline at it, not by writing a parser and then maintaining it through every redesign. Over 14,000 events have gone through it across two US cities in more than six months of production.
The hard part
95% accuracy sounds fine until you count it: five percent of 14,000 is several hundred wrong events, and a wrong date or venue on a discovery product is worse than a missing one, because the user turns up at the wrong place. So the pipeline's real job is not extraction, it is knowing when it is unsure. That confidence has to come from schema validation, cross-field consistency and normalisation against known venues, not from the model's own account of itself.
- Stack
- Next.jsReactNestJSCapacitorPostgreSQLTypeORMOpenAIRedisRenderVercel
- Focus
- AI Data ExtractionWeb ScrapingStructured OutputConsumer Marketplace