Session Mastery

Reading AI Summaries Critically

Module 4, Lesson 4: Reading AI Summaries

Learning Outcomes

By the end of this lesson, you should be able to:

  • Use AI-generated summaries as a preparation tool without becoming dependent on them
  • Identify four key signal types in AI summaries: themes, shifts, red flags, and contradictions
  • Verify AI-generated observations against your own reading of client submissions
  • Recognize when to trust your professional judgment over the AI's interpretation

AI as a Tool, Not a Crutch

Artificial intelligence is transforming coaching. Tools like CoachSheets' AI summary feature can process a client's worksheet submission in seconds and generate a structured recap highlighting key themes, emotional tone, and potential areas for exploration. This is a genuine advance—it saves coaches time, catches patterns they might miss, and provides an objective second pass on client data.

But here is the danger: AI summaries are seductive. They arrive formatted, confident, and complete. It is very easy to read the summary, nod along, and walk into the session feeling prepared. This is a mistake.

AI summaries are a first pass—a useful signal amplifier, not a replacement for your own engagement with the client's work. The most effective coaches use AI to augment their attention, not outsource it. You should always read the original submission alongside (or before) the summary. The AI sees patterns; you see a person.

"AI will not replace coaches. But coaches who use AI will replace coaches who do not." — Adapted from a common observation in AI-augmented professions

What to Look For

When reading an AI summary, train yourself to look for four specific signal types.

1. Themes (Recurring Topics)

The AI will identify themes that appear across multiple submissions or within a single one. Ask yourself:

  • Does this theme resonate with what I have observed? If the AI flags a theme you had not noticed, go back to the original submission and look for it. The AI may have spotted something you missed.
  • Is this a new theme or a recurring one? New themes are worth exploring. Recurring themes that have not shifted suggest the client is circling rather than progressing—which is itself useful information.
  • Is the theme framed helpfully? AI can mislabel themes. A submission about work stress might be summarized as "career concerns" when the client's actual focus was "identity and self-worth tied to professional performance." The AI gets the topic right but the depth wrong.

2. Shifts (Changes in Perspective or Energy)

Shifts are the most valuable signal in an AI summary. They tell you something has changed since the client's last submission.

  • Positive shifts: "Client's language around conflict is less defensive than previous submissions." This is progress worth naming and reinforcing.
  • Negative shifts: "Client's energy level appears significantly lower than in the past three submissions." This is worth checking in on—it could indicate burnout, a setback, or a temporary low.
  • Subtle shifts: "Client used 'I want' three times in this submission; previously used almost exclusively 'I should'." Language shifts often precede behavioral shifts. Flag them.

3. Red Flags (Warning Signs)

AI summaries can flag language that suggests distress, avoidance, or concern. CoachSheets' AI is trained to identify:

  • Language indicating overwhelm or burnout ("I can't keep up," "I'm exhausted all the time")
  • All-or-nothing thinking ("I always fail," "Nothing I do matters")
  • Isolation language ("No one understands," "I have to handle this alone")
  • References to substance use, self-harm, or suicidal ideation

Critical: If the AI flags a potential red flag, do not ignore it. But do not assume the AI is correct either. Read the original context. The AI may exaggerate a passing comment or miss cultural or linguistic nuance. Verify before acting.

If the red flag is credible and serious (e.g., references to self-harm or suicidal ideation), follow your safeguarding protocol. This is a coaching skill, not an AI function. The AI is an early warning system; you are the responsible professional.

4. Contradictions (Saying One Thing, Doing Another)

Contradictions are gold for coaching. When the AI flags a contradiction, it is pointing to exactly the kind of tension that coaching can help resolve.

  • Values vs. behavior: "Client states work-life balance is a top priority but describes working 60-hour weeks."
  • Words vs. actions: "Client committed last session to delegating more but reports having delegated nothing."
  • What they say vs. how they say it: "Client describes the situation as 'fine' but uses language like 'drowning' and 'surviving.'"

When you see a contradiction flagged, resist the urge to confront the client with it directly. Instead, stay curious: "I noticed something interesting in your submission. You wrote that you feel things are 'fine' at work, but you also mentioned 'drowning in deadlines.' Help me understand both of those."

How to Verify AI Summaries

Verification is a three-step process:

  1. Read the original submission first. Form your own impressions before looking at the AI summary. This prevents the AI from anchoring your interpretation.
  2. Compare your impressions to the AI's. Where do they align? Where do they diverge? Divergence is not a problem—it is an invitation to re-read more carefully.
  3. Hold the discrepancy. If you disagree with the AI, sit with the disagreement for a moment. Is the AI picking up on something you missed? Or is the AI over-reading? Your professional coaching judgment and intuition matter.

Over time, you will develop a sense of when the AI is reliable and when it tends to miss the mark. Different clients and different worksheet types will produce different AI summary quality. Learn the system's patterns so you can use it wisely.

When to Trust Your Judgment Over the AI

AI summaries are probabilistic, not definitive. They are generated by language models that predict the most likely next patterns based on training data. They do not know your client. They do not know the context of a six-month coaching relationship. They do not know that this client always sounds more negative in worksheets than they are in person.

Trust your judgment over the AI when:

  • You have relationship context the AI lacks. If you know the client tends toward dramatic language in worksheets, the AI's "red flag" may be less concerning than it appears.
  • The AI misses cultural or linguistic nuance. A client who says "I feel terrible" may be using common cultural idioms that the AI over-interprets.
  • The AI contradicts your direct observation. If you just spent a session with the client and they seemed energized and clear, but the AI says their worksheet shows low energy, your observation is real and valid. Both can coexist—explore the gap.
  • The client has explicitly asked you not to over-interpret certain patterns. Some clients are aware of their patterns and do not want them spotlighted every session. Respect that.

Exercise: Reading AI Summaries

Below are three AI-generated summaries of client worksheet submissions. For each summary, identify:

  1. One theme the AI has surfaced
  2. One potential shift (positive or negative)
  3. One potential red flag (if any)
  4. One question you would want to verify by reading the original submission

Summary A: "Client describes ongoing frustration with a team member who misses deadlines. Theme of control vs. delegation recurs from last session. Client's language is more solution-focused than previous submission ('I could try,' 'maybe I need to'). No red flags detected. Energy noted as 'determined.'"

Summary B: "Client reports 'feeling fine' but also uses the word 'exhausted' four times and 'can't' seven times. Theme of overwhelm present across last three submissions with no observable shift. Potential red flag: language suggesting social withdrawal ('I don't want to talk to anyone'). Client's goals section is blank."

Summary C: "Client describes a career milestone achieved this week (promotion). Language is celebratory but contains multiple references to imposter syndrome ('I don't deserve this,' 'they'll figure out I'm faking'). Shift from previous submission: last submission focused on self-doubt; this submission includes both self-doubt AND accomplishment. Red flags: none. Contradiction noted between stated worthiness and achievement."

Summary

AI summaries are a powerful coaching tool when used correctly. Read them as a first pass, not a final analysis. Look for themes, shifts, red flags, and contradictions. Always verify AI observations against your own reading. Trust your professional judgment when it conflicts with the AI. The goal is not to replace your coaching intuition but to amplify it.

Assignment

  1. Complete the exercise above: write a brief analysis (200-300 words total) of the three AI summaries, identifying themes, shifts, red flags, and verification questions.
  2. If you use CoachSheets, spend one week reading every AI summary after you read the original submission (not before). Note three instances where the AI added value and one where your judgment differed from the AI's. Write a 150-word reflection on the experience.

Knowledge Check

  • What are the four signal types to look for in an AI summary?
  • Why should you read the original client submission before reading the AI summary?
  • What is the proper response if an AI summary flags a potential red flag about self-harm or suicidal ideation?
  • Give an example of an AI-flagged contradiction that would be productive to explore in a coaching session.
  • Under what circumstances should you trust your own judgment over the AI's?

External resources:

Have an insight, real-world case study, or improvement for this lesson? Contribute your expertise. Propose an edit directly or explore the community curriculum discussions.
All Proposals
Finished reading this lesson?Mark your progress to advance toward your official certification.

Peer Discussions & Insights

Join the conversation with other practitioners.

No discussion notes yet. Be the first to share your takeaways!