Startup Community Poznań · Poznań, Poland
Three Gates Before You Spend the Budget
30-minute talk opening the Startup Community Poznań meetup on fast AI validation. Three gates, two days each, that kill or pass an AI idea before the budget goes out. Gate 1 — is AI even the right mechanism: deterministic rules mean a form, no data means collect data first, full-text search means full-text search; then the ladder prompt → prompt with context → RAG → workflow → agent, with 150,000 policy PDFs a month at Insly as the workflow case (Claude Haiku 4.5 on Bedrock EU, 98.5% extraction accuracy, $5,500 a month). Gate 2 — does it close on paper: users × queries × cost per query, so 4,000 queries a month runs ~$0.60 on an open model hosted in the EU, ~$70 on a frontier model and $942 on a 24/7 cloud GPU, a 250× spread for the same feature, plus the hidden costs nobody counts (two-page system prompts, retries, growing chat history, embeddings in a loop, 30–60 s cold starts). Gate 3 — does it work on MY data: general models answer confidently and wrongly on a closed domain, so the only credible test is your own — 30 questions, expected answers, scoring, a hallucination counter, pass threshold set before the test. Our result: a 7B model beat an 11B one (0.949 vs 0.915, ~1.5× faster end to end) and the harness caught a build injecting junk tokens that manual clicking missed. Punchline: three RAG prototypes, two weeks each, two kill decisions, production on the third — six weeks instead of a 6+ month R&D cycle.
