Case study 01 · KaDeep Technologies

KaDeep Studios — Low-code automation that survives the real web

A dual-engine QA IDE where AI authors, skills play, and every run leaves evidence. BrowserStack-class authoring without making the model the test.

Role: AI Product Manager — I owned problem framing through production on named surfaces: the create / “what do you want to test today?” frame, AI generate with human-in-the-loop preview, skills-first playback, two-stage heal with an audit trail, the OTP inbox, build evidence, and v0.5 access / RBAC.
Live: studios.kadeep.ai (opens in a new tab)
Desktop app (macOS + Windows)
Stack: React 19ViteTailwind v4ZustandTanStack QueryNestJS

Create surface — “what do you want to test today?” Record a URL or generate with KaDeep AI.

90-second brief

How to read this

Enterprise QA needed authoring that survives UI churn without turning every flake into an LLM invoice. v0.3 almost shipped the other way: hostname special-cases and model-first heal.

Hostname special-case
Died
Fixes today’s login drawer. The next site still breaks. I would have been selling a demo, not a product.
LLM on every flake
Died
Spend tracks UI volatility — the worst unit economics. Recovery becomes a black box no QA lead can explain to a release manager.
Skills-first, LLM-second
Shipped
Playbooks take the common path. The model is invoked only on a true miss, and the heal leaves evidence. This is what shipped.

Decision: Deterministic playbooks attempt recovery first. Claude is the backup, not the first move. A heal the report cannot narrate is a product failure.
Result I will defend: Live at studios.kadeep.ai. Analytics adopted by 100+ internal QA and product users. Release turnaround dropped 25% across three core modules after we redesigned QA — I do not have the baseline week in writing. Booking.com 14-step replay ~66s. OTP is a product surface, not a ticket.
What I got wrong: I treated OTP as an edge case until login demos kept dying. I also left the heal path under-explained — QA will not adopt a recovery they cannot narrate.
Who loses if this is wrong: The release manager, if a silent heal papers a defect. The SDET, if PMs flood the suite with NL tests nobody can maintain.

Metric spine

North star: Trustworthy release signal; A lead can act on a scheduled suite without asking an SDET to watch the video. The buyer is a decision at the gate, not “more tests.”
Guardrail: Wrong-heal / false-green; A decoy click or silent rewrite that papers a real defect. Reporting heal-rate alone would be a fail. Wrong heal is worse than a red build.
Not instrumented: Playbook hit rate; Stage-1 skill or fuzzy match ÷ locator failures. Would prove the LLM is not doing everyday work. Not on a dashboard yet.
Not instrumented: Cost per useful heal; Token $ on stage-2 ÷ heals that passed post-verify and were kept. Cost per call is the wrong unit. Not instrumented.
Not instrumented: Weekly active authors; 100+ is seats and adopters, not weekly authors split by QA / PM / SDET. I will not dress the denominator.

Skills-first, LLM-second. Deterministic playbooks attempt recovery first. The model is invoked only on a true miss.

v0.3 almost shipped the other way — hostname special-cases and an LLM on every flake. That makes spend track UI volatility, and the recovery path becomes a black box no QA lead can explain to a release manager. Named skills make the common path cheap, inspectable, and repeatable. The LLM stays attached to genuine novelty. Same idea as a senior SDET: cheap playbooks before an expensive model.

Two-stage self-healing flow

UI change detected
A locator or assertion fails under churn.
Skills / fuzzy match
Named playbooks and semantic match run first.
LLM fallback
Claude only if stage 1 is below threshold — then verify and audit.