Jacob Cascone

AI lesson-planning platform for K-12 teachers · lessondraft.com · Solo builder & operator · Live, real paying subscribers

• ›Full production SaaS built solo: Next.js frontend, Stripe subscription billing across 3 tiers, Supabase-backed accounts, and 40+ AI-generated tools (lesson plans, worksheets, rubrics, IEP goals, report-card comments) — no signup required to try the core generator.
• ›Designed a multi-provider AI generation pipeline with automatic failover across free-tier, self-hosted, and premium models, with self-healing health checks that route around a dead provider automatically — keeps marginal AI cost near zero without sacrificing uptime.
• ›Audited the full Stripe billing surface end-to-end against live checkout data and found four independent revenue-blocking bugs in one pass — including a mispriced plan that had silently overcharged for five months and a webhook gap that would have let failed-payment customers keep paid access indefinitely. All four fixed and re-verified live.
• ›Built custom production monitoring (synthetic customer-journey canaries, health checks) and a cross-machine backup path for the AI generation layer.

Published on npm · npmjs.com/package/llm-ladder · Zero runtime dependencies

• ›Extracted and generalized LessonDraft's production provider-failover system into a standalone, documented library: multi-provider LLM failover with circuit breaking and proactive budget-aware standdown.
• ›Built a chaos-test suite that injects real failure modes (timeouts, 429 rate limits, malformed responses) and asserts on actual failover behavior — not just happy-path success. Verifies a dead provider gets skipped after its circuit opens, a circuit re-closes after its window, a canary-confirmed failure opens a circuit on the first strike instead of waiting for three, and a budget guard stands a provider down using its real check-then-record ordering.
• ›10/10 tests passing, deterministic (injectable clock, no real sleeps, full suite runs in under 100ms). The budget-guard test initially had wrong math and failed on its first run — left that in the README as the honest story, not smoothed over.

Open-source RAG tool · github.com/cascone26/standards-rag · Real document, real eval

• ›A retrieval-grounded Q&A tool over the real, public Common Core Math Standards PDF (93 pages) — a from-scratch BM25 retrieval implementation (zero cost, no embedding API) feeding Claude for citation-grounded synthesis.
• ›Built a real retrieval-precision eval against 10 questions with verified ground truth: 70% precision@5, 30% precision@1 on the first run — reported as-is with a root-cause explanation for each miss (PDF-extraction artifacts on standard codes, vocabulary mismatch between natural questions and document phrasing) rather than tuned to look better.
• ›Chose lexical retrieval over embeddings deliberately for a single domain-specific document with precise vocabulary — a documented architecture decision, not a shortcut.

Open-source agent · github.com/cascone26/agentic-grader · Human-approval gate, 8/8 tests

• ›An agentic batch-grading pipeline with a real human-approval gate: low-confidence grades are routed to a teacher for review instead of being silently auto-committed — a working prototype of the same product category as Flint/Frizzle, not a toy demo.
• ›Designed the grading function as an injectable interface, the same pattern llm-ladder's chaos tests use for provider calls — this makes the actual hard problem (deciding when NOT to trust the AI) fully unit-testable with deterministic fake graders, independent of any live LLM call: 8/8 tests passing.
• ›Two failure principles enforced and tested: a grading error is never treated as a silent zero (it's flagged for review with the error attached), and an invalid confidence value throws loudly instead of silently misrouting a submission.

Multi-subject AI learning platform · Solo builder & operator · Live-app screenshots + case study (it's login-gated to 2 accounts, so this is the proof)

• ›Refactored a single-subject study tool into a subject-agnostic architecture — one shared AI-chat layer, progress-tracking interface, and content-grounding registry, with each new subject (8 real classes, chess, trivia) plugging in as an adapter instead of duplicated code.
• ›Built a chess module integrating a real chess engine client-side with zero server cost, with move-quality analysis wired into the same cross-subject progress system.
• ›Found and fixed two silent production incidents via direct live investigation — a corrupted environment variable that silently broke every database-backed feature app-wide, and a stale auth-token sync gap breaking every AI route.
• ›Root-caused a broken deploy pipeline blocking a second engineer's first PR — the repo was split across 3 stale Vercel projects from a prior account migration — and fixed it so pushes auto-deploy with working auth, backed by a GitHub branch-protection ruleset and a CI typecheck/build gate on every PR.

Personal game-asset project

• ›Designed a multi-round verification pipeline — a 31-agent contact-sheet review, then an adversarial "skeptic" AI pass, then personal human spot-check at full resolution — to QA a 36,000+ sprite catalog against intended name/appearance. Found that even the adversarial AI layer had real error rates in both directions, closed only by human-in-the-loop review; the final verified set is hard-blocked in the generation pipeline.