Future ideas

Direct JSON editing — agent stays in the loop

Let power users open and edit the generated scenario JSON in the UI (with validation). When they save, the system should notify the conversational agent so it understands what changed and can continue naturally if the user keeps chatting.

• After an edit, show a clear summary of consequences—e.g. which objectives, personas, or test data were affected and what the agent will assume going forward.
• Keep chat history and structured state consistent so “continue the conversation” feels coherent, not blind to manual edits.

Ground the sample chat with real bot transcripts

Today, “Sample chat” is illustrative: the model invents plausible turns from the scenario we’ve built, but it does not know your real bot—so it cannot know what production would actually say.

Let teams attach or paste real transcripts from the actual chatbot (redacted as needed). Those snippets become optional context for the LLM when generating or refreshing the sample dialogue—style, length, tone, and common phrasing should line up better, while still being clearly labeled as a simulation, not a guarantee.

• Support multiple short excerpts (per channel, locale, or intent) with sensible token limits.
• Keep separation between grounding material and the formal scenario JSON so tests stay deterministic where they need to.

Stored instructions & context

A place for custom instructions and domain-specific notes (policies, product lines, compliance hints) that are injected into the agent’s context so every session starts with the right organizational knowledge.

Choose the LLM provider in the UI

The server already supports multiple providers (OpenAI, Bedrock, Anthropic, Gemini, Grok, etc.). Expose a safe, controlled selector in the interface—subject to how you manage credentials and tenancy—so testers can match the builder to the model they use elsewhere without redeploying.

Versioning for scenarios

Treat each scenario as something that evolves over time: named or numbered versions, timestamps, short change notes, and the ability to compare two versions (diff-style) or roll back to a known-good snapshot.

• Versions could be created automatically on each generate / save, or only when the user explicitly checkpoints.
• Links runs and reports to the version that was executed so results stay interpretable months later.

Shareable read-only links

Generate a time-limited or revocable URL that opens the scenario (summary + JSON) in view-only mode—ideal for reviewers, compliance, or partners who should not edit or chat with the builder agent.

• A stripped view-only UI: no new session, no LLM spend, optional watermark so exports read clearly as “preview / not for execution.”
• Optional expiry + revoke so old links don’t leak long-lived access.

“Here’s a real chat — turn it into objectives”

Many teams already have a conversation they’re happy with: a golden path, a support resolution, or a demo that worked end-to-end. Today that lives in logs or copy-paste; the leap to a repeatable test scenario is manual and error-prone.

Add a flow where users paste or upload a real transcript (user + bot turns, channel metadata optional). The agent infers objectives, persona, test data, and success criteria from that dialogue—then opens the normal builder so they can refine, add edge cases, or tighten assertions before generating JSON.

• Complements transcript grounding for sample chat: here the transcript is the source of the scenario, not just stylistic context.
• Clear UX for PII redaction, turn boundaries, and “this turn is the bot” labeling when format is ambiguous.
• Optional: mark intents like “treat as happy path” vs “extract failure conditions” to steer extraction.

Live logs → agentic scenario — then scale to the whole bot (IVR golden paths)

Picture someone watching production or staging logs and spotting a conversation that went perfectly—placing an order, completing a transfer, finishing a booking. They copy the full transcript, paste it into this agent, and ask it to design an agentic test scenario from that exchange. The agent produces a structured draft; they keep talking in plain language to tighten objectives, data, and edge cases until they’re happy.

Now imagine pairing that with flow intelligence: e.g. feed an IVR / call-flow (or equivalent), automatically enumerate happy paths (“golden paths”), then generate agentic scenarios for each path, run them, and hand results back—so in one pass the team gets a baseline suite they can open, review, and fine-tune in the builder instead of authoring everything from scratch.

• Composable pipeline: log snippet → single scenario (above) is the atomic step; bulk generation + execution is the multiplier for IVR and other structured bots.
• Human in the loop: automation proposes coverage; humans approve, edit, and prioritize—especially for compliance and brand risk.
• Differentiation: directionally, closing flow → golden paths → agentic tests → run → report in one product story is uncommon; exact competitive claims still need validation against the market you sell into.

Voice in, voice out — natural conversation with the builder

Typing long requirements works, but speaking can feel closer to how people describe tests in meetings. Add voice input (browser capture or push-to-talk) so utterances are transcribed into the same chat pipeline as typed messages.

Pair that with high-quality text-to-speech—for example ElevenLabs—so assistant replies are read aloud with a consistent voice. The goal is a more human, “talking to the bot” rhythm while you’re still designing scenarios, not executing the real production bot.

• Opt-in & mute: TTS off by default or per-session; respect prefers-reduced-motion / accessibility preferences.
• API keys & cost: ElevenLabs (or similar) via server-side proxy so keys stay off the client; clear indication when audio is billable.
• Streaming: align TTS with SSE chunks where possible so responses don’t wait for the full paragraph before playback starts.

“Run this test now” (teaser control)

Once a scenario feels finalized in the builder, users naturally want the next step: execute it against the real bot (e.g. Route 13). Full wiring needs orchestration, credentials, and target environments—but we can still teach the workflow with a prominent control that shows what “done” unlocks.

Idea: add a non-functional “Run this test now” button (disabled or preview-only) near generate / save. Clicking it could open a short explainer: what would happen if integrated—queue a run, pick environment, open the console, attach results—so teams align on product direction without pretending the action works today.