The LLM + MCP Tools design handles single-turn Q&A well. But real customer service requires multi-turn dialog management, slot filling, sentiment-aware escalation, and structured conversation flows. Here's how Conversational AI fills the gap.
The current LLM-only approach handles simple lookups. But telecom customer service has structured, multi-step workflows that need deterministic flow control.
Consider: "I want to upgrade my plan from 499 to 999"
The Conversational AI layer sits between the API controllers and the LLM, providing structured dialog management.
The Conversation Router decides per-turn whether to use a deterministic dialog flow (for state-changing operations like plan upgrades, payments, profile updates) or free-form LLM (for informational queries like "what's my bill?" or "explain data rollover"). This gives you the reliability of state machines where it matters and the flexibility of LLM where it shines.
Six new Java services that add structured dialog management to the existing prototype.
Classifies user intent before the LLM sees the message. Extracts structured entities (amounts, dates, plan names). Uses a lightweight classifier (logistic regression or small BERT) for speed, falling back to LLM for ambiguous cases.
Manages conversation state for multi-turn flows. Each flow (billing inquiry, plan change, complaint) is a finite state machine with defined transitions, required slots, and confirmation gates. Persisted in Redis per session.
Tracks which information has been collected and what's still needed. Generates targeted prompts for missing slots. Validates slot values against business rules (e.g., plan code must exist, amount must be positive).
Detects customer frustration, anger, or confusion in real-time. Triggers tone adjustments ("I understand this is frustrating...") or escalation to human agents when sentiment drops below threshold.
The brain that decides per-turn: use a structured dialog flow (deterministic, safe for mutations) or free-form LLM (flexible, great for queries). Routes based on detected intent + current dialog state.
When the bot can't resolve (3+ failed attempts, negative sentiment, explicit request), it prepares a structured context summary and transfers to a human agent via queue with full conversation history.
| User Says | NLU Output | Router Decision | Handler |
|---|---|---|---|
| "What's my bill?" | Intent: BILL_INQUIRY Entities: none | Free-form LLM | LLM → get_current_bill tool → response |
| "Change my plan to 999" | Intent: PLAN_CHANGE Entity: target_plan=POST999 | Structured Flow | Dialog Manager → PLAN_CHANGE flow → slot filling → confirmation → tool |
| "This is the third time I'm calling!" | Intent: COMPLAINT Sentiment: FRUSTRATED | Escalation Path | Sentiment triggers empathy prompt → Agent Handoff if persistent |
| "Update my email to new@mail.com" | Intent: PROFILE_UPDATE Entity: field=email, value=new@mail.com | Structured Flow | Dialog Manager → PROFILE_UPDATE → OTP re-verify → confirmation → tool |
| "How does data rollover work?" | Intent: FAQ Entities: topic=data_rollover | RAG + LLM | Vector search FAQ → LLM formats answer |
| "Talk to a human" | Intent: AGENT_REQUEST | Handoff | Agent Handoff Manager → context summary → queue |
Example of a structured multi-turn conversation managed by the Dialog Manager.
Customer: I want to upgrade my plan
→ NLU: intent=PLAN_CHANGE | Router: STRUCTURED_FLOW | State: INIT → COLLECT
Bot: I'd be happy to help you upgrade! You're currently on the Postpaid ₹499 plan. Which plan would you like to switch to? We have Postpaid ₹999 (150GB data, unlimited calls) and Fiber ₹799 (200GB).
Customer: The 999 one
→ NLU: entity=POST999 | Slot filled: target_plan=POST999 | Missing: timing | State: COLLECT
Bot: Great choice! The Postpaid ₹999 gives you 150GB data and unlimited calls. Should the change take effect immediately or from your next billing cycle (April 1)?
Customer: Next cycle
→ Entity: timing=NEXT_CYCLE | All slots filled | State: COLLECT → VALIDATE → CONFIRM
Bot: Here's a summary of your plan change:
• Current: Postpaid ₹499/month
• New: Postpaid ₹999/month (150GB, unlimited calls)
• Effective: 01 Apr 2026
• Monthly increase: ₹500
Shall I proceed with this upgrade? (Yes/No)
Customer: Yes, go ahead
→ Confirmation: YES | State: CONFIRM → EXECUTE → COMPLETE | Tool: upgrade_plan(POST999, NEXT_CYCLE)
Bot: Done! Your plan has been upgraded to Postpaid ₹999. It will take effect on 01 Apr 2026. You'll receive a confirmation SMS shortly. Is there anything else I can help with?
| Flow ID | Trigger Intent | Required Slots | Confirmation? | Mutates Data? |
|---|---|---|---|---|
| BILL_PAYMENT | PAY_BILL | amount, payment_method | Yes | Yes |
| PLAN_UPGRADE | PLAN_CHANGE | target_plan, timing | Yes | Yes |
| PROFILE_UPDATE | UPDATE_PROFILE | field, new_value, otp | Yes | Yes |
| COMPLAINT_FILE | FILE_COMPLAINT | category, description | Yes | Yes |
| PLAN_INQUIRY | PLAN_INFO | none (optional: plan_code) | No | No |
| BILL_INQUIRY | BILL_INFO | none (optional: month) | No | No |
| USAGE_CHECK | DATA_USAGE | none (optional: period) | No | No |
| AGENT_TRANSFER | HUMAN_REQUEST | none | No | No |
Key Java classes and how they integrate with the existing Spring Boot prototype.
The Conversational AI layer is inserted between the controllers and the AI orchestrator. The ChatController now calls ConversationRouter instead of calling AiOrchestratorService directly. The router decides whether to use a structured dialog flow or free-form LLM per turn.
| New File | Purpose | Lines (est.) |
|---|---|---|
NluEngine.java | Intent classification + entity extraction via lightweight model | ~180 |
DialogManager.java | State machine for structured conversation flows | ~250 |
SlotFillingEngine.java | Tracks required slots, generates prompts for missing fields | ~200 |
SentimentAnalyzer.java | Real-time sentiment scoring with escalation triggers | ~120 |
ConversationRouter.java | Hybrid routing: structured flows vs free-form LLM | ~220 |
AgentHandoffManager.java | Context summarization and queue-based agent transfer | ~150 |
DialogFlowDefinitions.java | YAML-driven flow definitions (states, slots, transitions) | ~180 |
ConversationState.java | Redis-persisted session state (current flow, slots, history) | ~100 |