Trial Conversion Engine

The problem this solved

A mid-market SaaS company was drowning in trial signups that nobody knew what to do with. The product was good enough to attract volume, but the sales team couldn’t tell which trials were worth chasing, and by the time they got around to the ones that were, the moment was gone. Product analytics showed engagement. The CRM showed contact info. Neither answered the actual question: which of these trials is my next customer, and how fast do I need to move?

The Trial Conversion Engine takes a raw trial signup and turns it into a scored, routed, sales-ready opportunity. It enriches the record with firmographic data, extracts behavioral signals from product usage, scores against ICP cohorts, routes to the right sales motion based on confidence tier, generates an intelligence dossier for the AE, and drafts a personalized first-touch email — all before the trial is 90 seconds old. An independent quality gate reviews every output before sales sees it.

The result: the sales team stopped guessing which trials to prioritize, and the trials that routed to an AE arrived with a full context brief already written.

Architecture

The system is an 8-stage pipeline. Each stage has a defined input, output, and failure mode. Five stages are deterministic — pure business logic, auditable, reproducible. Three are AI-powered — they handle tasks that require judgment or synthesis. One is a hybrid.

The design principle: deterministic where the decision needs to be auditable, AI where the task requires judgment. Routing is never left to the LLM.

The stages

Design principles

Deterministic routing over probabilistic

When a lead scores Tier 1 with active engagement, it routes to an AE. There’s no “maybe” layer, no probabilistic scoring that might send it somewhere else on a different day. AI adds judgment in enrichment and dossier generation. Routing is rules-based and auditable.

This is a deliberate architectural choice. Sales teams don’t trust black boxes. When an AE asks “why did this lead get fast-tracked?” the system shows the score breakdown, the cohort match, and the behavioral signals. Every routing decision is reproducible — run the same lead through twice, get the same route.

Pre-mortem methodology

Every failure mode is identified before implementation, not discovered after. Five specific failure modes are documented and mitigated by design:

• API rate limits during bulk processing — mitigated by pre-computed results with file-based persistence. The pipeline can process ahead of time; results survive server restarts.
• Garbage-in-garbage-out cascade — if enrichment hallucinates a wrong industry, every downstream stage produces incorrect output. Mitigated by confidence scoring at every AI stage. When confidence drops below threshold, the system flags the lead rather than silently passing bad data downstream.
• LLM output schema violations — the LLM returns malformed JSON, wrong field names, or conversational preamble instead of structured data. Mitigated by a robust extraction layer with typed defaults. The pipeline degrades gracefully — it never crashes on a schema violation.
• Quality gate theater — the gate rubber-stamps everything, undermining the review layer. Mitigated by 6 specific grading criteria with numeric thresholds, and architectural separation — the gate is a different LLM instance with a different prompt.
• State desynchronization — the dashboard shows stale data or crashes on missing fields. Mitigated by file-based persistence and defensive rendering.

Confidence scoring

Every AI stage reports its confidence as a numeric value. Low-confidence enrichment gets flagged with a visual indicator, not passed through silently. The dashboard shows warnings on any stage where confidence dropped below threshold.

This matters most in enrichment, where the LLM is synthesizing company information that may be incomplete or ambiguous. A confidence score of 0.4 on industry classification means “I’m guessing” — and the pipeline treats it accordingly.

Independent quality gate

The agent that checks the output is architecturally separate from the agents that produce it. Different LLM instance. Different system prompt. Different evaluation criteria.

The gate scores against accuracy, personalization, tone, actionability, brevity, and completeness. Each criterion has a numeric weight. The aggregate score determines pass, regenerate, or flag-for-human.

Calibration as ongoing work

The first pipeline run processed 12 sample leads. 8 routed correctly. 4 were misrouted. That’s not a failure — that’s calibration data.

Root cause analysis on the 4 misroutes revealed three independent issues: a hospitality industry gap in the ICP cohort data, a churned-behavior threshold that was too restrictive, and missing engagement sub-rules for Tier 2 leads. Each issue had a documented fix scoped to a specific file and line number. The architecture supports this — scoring logic is isolated, routing rules are explicit, and every decision is traceable. Real systems need calibration. The architecture is designed for it.

Tech approach

Key implementation choices for this build:

• Claude and OpenRouter for AI stages — enrichment, dossier generation, email drafting, and quality gate. Claude Sonnet for stages that require synthesis and nuance. Cost-optimized models for classification and grading. Full pipeline cost: under $0.04 per lead.
• Direct CRM API for intake and routing — the system reads from and writes to the CRM through its API, which is what made sub-minute processing possible.
• File-based persistence for auditability and debugging — every stage’s output is stored as a JSON artifact. Results survive server restarts. Atomic writes prevent corruption. During calibration, you can inspect exactly what each stage produced and why.
• Dashboard with three views: Pipeline (all leads with stage-by-stage status and routing), Dossier (deep dive into a single lead’s intelligence brief with interactive score breakdown), and Observability (aggregate metrics, cost tracking, confidence distributions).

This is one approach

A standalone scoring-and-routing system was the right answer for this client because their trial volume was high enough to matter, their existing product analytics didn’t connect to the CRM in any useful way, and the sales team needed more than notifications — they needed context. For a smaller trial volume, a lighter touch might be enough: better HubSpot scoring properties, a smarter Slack alert, a routing rule on top of an enrichment tool the team already pays for. The diagnosis decides how much system the situation actually justifies.

Where an engagement starts

Not every trial conversion problem needs a standalone pipeline. Most engagements start by figuring out whether it does.

Start with an audit. Look at the trial flow end-to-end: what data is captured, where it lives, which signals the sales team actually uses, and where leads are falling through. Sometimes the gap is instrumentation — events that should be firing but aren’t. Sometimes it’s routing logic that exists but isn’t trusted. Sometimes it’s an ICP model that was never written down. The audit tells you which of those is the real bottleneck.

When the audit points at a scoring-and-routing build, the engagement looks like this:

1. ICP and data source design — define scoring cohorts from the existing customer base, map the product analytics events worth using, identify enrichment signals that actually differentiate.
2. Architecture scoped to your stack — pipeline stages tailored to your CRM and data sources, routing rules mapped to your actual sales motions.
3. Staged build with checkpoints — each pipeline stage delivered and reviewed independently. You see working output at every checkpoint, not just at the end.
4. Calibration against real data — run the pipeline against actual trial signups, diagnose misroutes, tune scoring thresholds.
5. Handoff with documentation — the system is yours. Full code, architecture docs, calibration playbook.

Ongoing calibration is available as needed — new cohorts as the ICP evolves, scoring adjustments as the product does.