Module 11 — AI Support Systems & Governance

Learning Objectives

Design AI-powered support and self-service systems with appropriate guardrails
Identify the key risks in AI customer-facing features: hallucination, PII exposure, bias
Apply a governance framework for AI product decisions
Write an AI feature policy for a product team
Define when to escalate AI decisions to human review

Lesson 11.1 — AI in Customer-Facing Support

AI support systems — chatbots, virtual assistants, automated resolution engines — are among the most common and highest-risk AI features PMs ship. They sit at the intersection of user trust, operational efficiency, and compliance exposure.

The support AI paradox: The same features that make AI support fast (autonomous responses, no wait time, always available) also make it risky (hallucination, inappropriate escalation, PII in chat logs). Good PMs design both sides simultaneously.

AI Support Architecture Patterns

Pattern	How it works	Best for	Risk level
FAQ deflection	Matches user query to known answers from a knowledge base	High-volume, low-variance queries	Low
Intent classification + routing	Classifies intent and routes to correct team or flow	Complex support with multiple departments	Low–Medium
Generative response	LLM generates a custom response using RAG-retrieved context	Complex, nuanced queries with varying context	Medium–High
Autonomous resolution	AI takes action (refund, account change) without human review	Simple, reversible, low-stakes transactions	High

Escalation Design

Every AI support system needs a clear escalation path. Design escalation triggers before launch:

Confidence threshold: If AI confidence is below X%, route to human agent
Sentiment detection: Angry or distressed users get human handling
Topic blocklist: Legal, medical, regulatory, or sensitive topics always escalate
Repeat contact: If the same user contacted support N times in Y days, escalate
Explicit user request: "Talk to a human" must always be honoured immediately

Lesson 11.2 — AI Risk: PII, Hallucination, and Bias

PII in AI Systems

Personal Identifiable Information (PII) in AI pipelines creates serious compliance risk. As a PM, you are responsible for ensuring your AI features don't inadvertently collect, log, or expose PII.

Risk	Example	Mitigation
PII in prompts	User pastes account number into chat	PII detection middleware strips or masks before sending to LLM
PII in logs	Chat transcripts logged with names and emails	Log at intent level, not message level; anonymise before storage
PII in RAG chunks	Support docs contain example customer names	Audit knowledge base for PII before ingestion
Model memory	Conversational AI "remembers" PII across sessions	Stateless sessions by default; explicit opt-in for memory features

Hallucination Management

Hallucinations — confident but false AI responses — are not fully preventable. PMs design systems that minimise harm when they occur:

Scope containment: Limit the AI to topics where you have high-quality RAG documents
Citation requirements: Prompt the AI to cite the source chunk for every claim
Uncertainty surfacing: Prompt the AI to say "I'm not certain" rather than guess
Human review for high-stakes answers: Refunds, legal info, billing disputes always go to human
User reporting: Easy thumbs-down + reason helps you identify hallucination patterns quickly

Bias in AI Features

AI systems can perpetuate or amplify bias in ways that create real harm and regulatory exposure. Key bias risk areas for PMs:

Training data bias: If the model was trained on non-representative data, outputs may systematically disadvantage certain groups
Prioritisation bias: Recommendations or routing that consistently deprioritises certain demographics
Language bias: Quality degradation for non-English or non-standard English users
Feedback loop bias: When AI recommendations influence user behaviour, and that behaviour re-enters training data, bias compounds

Lesson 11.3 — AI Governance Framework for PMs

Governance is not a legal formality — it is the operational structure that makes AI features safe to ship and maintain.

The PM AI Governance Checklist

Category	Question	Required before launch
Data	What data does this feature use? Is any of it PII?	Data flow diagram + PII classification
Consent	Do users know they are interacting with AI?	AI disclosure in UI
Accuracy	How do we measure and monitor output quality?	Accuracy baseline + alerting threshold
Escalation	What happens when AI is wrong or uncertain?	Escalation path documented and tested
Bias	Have we tested for differential performance across user groups?	Bias evaluation report
Audit trail	Can we reconstruct what the AI said and why?	Logging spec + retention policy
Override	Can humans override AI decisions?	Human-in-the-loop mechanism defined
Sunset	How do we retire this feature safely?	Deprecation plan

Regulatory landscape: EU AI Act, GDPR, CCPA, and sector-specific rules (e.g., PSD3 for payments) are increasingly constraining AI feature design. Build governance habits now — retrofitting is costly.

Lab 11 — AI Support Feature Policy & Risk Audit

You will design the governance layer for a new FlowScale AI support feature: an AI chat assistant for billing queries.

Define escalation triggers

FlowScale's AI billing assistant can handle: payment status queries, failed payment explanations, dunning schedule info, and invoice downloads. Use this prompt:

Prompt: "I'm designing an AI billing assistant for FlowScale, a B2B SaaS subscription platform. The assistant handles: payment status, failed payment explanations, dunning schedules, and invoice downloads. Create a comprehensive escalation trigger list covering: confidence thresholds, sentiment signals, topic blocklist (what should ALWAYS go to a human), repeat contact rules, and explicit user request handling. Format as a table with trigger, condition, and action."

Write a PII data flow

Map the data flow for the FlowScale AI billing assistant. Identify every point where PII could enter the system and what mitigation applies. Use the prompt:

Prompt: "Map the PII data flow for an AI billing assistant. The flow includes: user inputs query in chat → query sent to LLM API with RAG context → response generated → chat transcript logged → user feedback collected. For each step, identify: what PII could be present, what the risk is, and what mitigation should be in place. Output as a flow diagram description and a risk table."

Complete the governance checklist

Using the 8-category governance checklist from Lesson 11.3, complete each row for the FlowScale AI billing assistant. For each category, write a one-sentence answer and specify the deliverable that must exist before launch.

Deliverables

Escalation trigger table: minimum 8 triggers covering all 5 categories
PII data flow: step-by-step flow with risk and mitigation for each step
Governance checklist: all 8 categories completed with pre-launch deliverable specified

How to Verify Completion

Escalation table includes an explicit trigger for "user asks to speak to a human" with action "immediate human handoff, no AI retry"
PII data flow identifies the LLM API call as a PII risk point and specifies masking or stripping as mitigation
Governance checklist includes an AI disclosure requirement (users must know they're talking to AI)
At least one topic is on the blocklist (e.g., legal disputes, account termination) that always routes to human
Checklist includes an audit trail entry specifying what is logged and for how long

Done when: A compliance officer reviewing your governance checklist can confirm: (1) PII risks are identified and mitigated, (2) users are informed they are talking to AI, (3) a human escalation path exists and is always accessible, and (4) audit logs are defined.

Module 11 Quiz

7 questions · click an option to answer · review all before checking your score

1. Which AI support architecture pattern carries the highest risk level?

Autonomous resolution — where the AI takes real-world action (refunds, account changes) without human review — carries the highest risk. Errors are acted upon immediately and may be hard to reverse. This pattern requires the most rigorous guardrails.

2. A user types their full credit card number into your AI support chat. What mitigation should be in place?

Prevention is better than detection. PII detection middleware that strips or masks sensitive patterns before sending to the LLM API prevents the data from ever reaching an external system or being stored in LLM logs. User warnings alone are insufficient.

3. What is the correct approach to AI hallucination as a PM?

Hallucinations cannot be fully prevented. The PM's job is harm minimisation: scope the AI to topics with good RAG coverage, require citation, prompt uncertainty surfacing, and route high-stakes queries to humans. Disclaimers alone are legally and ethically insufficient.

4. Which escalation trigger must ALWAYS be honoured immediately?

User autonomy is non-negotiable. When a user explicitly asks to speak to a human, that request must be honoured immediately — no retry, no "let me try to help you first". Failing to honour this destroys trust and may violate consumer protection regulations.

5. What is a feedback loop bias in AI systems?

Feedback loop bias is particularly insidious because it compounds over time. An AI that slightly over-recommends certain content influences what users click, those clicks become training data, and the next model version amplifies the original skew further.

6. What must a PM ensure before launching any AI-facing customer feature?

AI disclosure — users knowing they are interacting with an AI system — is a fundamental requirement and increasingly a legal obligation (EU AI Act, consumer protection laws). 100% accuracy is impossible; third-party audits are best practice but not universally required for every feature.

7. When designing AI logging for a support system, what is the recommended approach?

Logging full messages creates PII exposure risk. Logging nothing creates an accountability gap. The balanced approach is to log at the intent/outcome level (what problem was being solved, was it resolved) and anonymise any message-level data before storage.

Score: 0 / 7