Skip to content

UX Audit — Chat Transcript UI

Date: 2026-04-07
Scope: Plan steps, transcript, file/command chips, diff viewer, search/filter
Skills Applied: ux-audit (dogfooding walkthrough), ui-design-review (heuristic + WCAG), ux-researcher (personas + journey mapping)

Executive Summary

The plan-step UI is functionally rich — file chips with inline diffs, terminal command chips, step-scoped search, live streaming indicators, and step-level diff navigation are all strong features. The information architecture is sound: steps as the primary organizing unit with tool calls nested underneath works well conceptually.

However, three systemic problems undermine the experience:

  1. Accessibility is severely lacking — clickable divs without keyboard support, no ARIA expanded/pressed states, no screen reader announcements for dynamic content, color-only status indicators. This is a WCAG Level A failure across the board.

  2. Information density vs. scannability tradeoff is wrong — completed steps show the same visual weight as active steps, chips and badges compete for attention, and there's no visual hierarchy between "what the agent is doing now" and "what it already did."

  3. The chip system creates cognitive load — users must learn 4 color-coded chip types (blue create, green edit, violet read, amber terminal) and understand that clicking a chip reveals nested content below the chip row, which is unexpected in a tag/badge pattern.

Design maturity score: 6/10 — strong engineering, weak polish on interaction patterns and accessibility.

Skill 1: UX Walkthrough (Dogfooding)

Persona

Solo developer — kicked off a Copilot agent to "add authentication to the Express app." It's been running for 8 minutes. They want to know: Is it stuck? What files did it change? Should I approve the next action?

Flow: "What is the agent doing right now?"

  1. Land on job detail page → Tab: "Live" → See step list
  2. Scan step list → Active step at bottom has blue left border + spinner
  3. Read current tool → Blue pulsing dot + tool name (e.g., "Editing auth.ts")
  4. Expand step → Click step header → See tool calls, agent message, file chips

Findings:

# Severity Finding
W1 HIGH Active step doesn't auto-scroll into view. If there are 10+ completed steps, the active step is below the fold. The "Jump to current step" button exists but it's a workaround, not a default. First-time users won't know to look for it.
W2 HIGH Completed steps have too much visual weight. Every completed step shows file chips, command chips, tool counts, duration, and relative time. When scanning for "what's happening NOW," the eye has to scan past all this noise. Completed steps should collapse their chip rows by default.
W3 MEDIUM "View changes in this step" button is easy to miss. It's a bare text link with a tiny git icon at the very bottom of an expanded step. It should be a more prominent, styled button — this is a key action.
W4 MEDIUM Step expansion is a binary toggle with no preview. There's no hover preview or partial expansion. Clicking expands everything (tool calls, agent message, chips) at once. Users who just want to see "what files changed?" have to see tool call details too.
W5 LOW Relative time badges are redundant. Each step shows both duration ("2.5s") and relative time ("3m ago"). For completed steps, duration is more useful; relative time adds noise.
W6 LOW Streaming text is line-clamped to 2 lines. Long agent reasoning gets truncated with no way to see more without expanding.

Flow: "What files did it change?"

  1. Scan completed steps → See file chips on each step (blue/green/violet)
  2. Click a green chip → Inline diff expands below the chip row (unexpected position)
  3. Read the diff → LCS-based, line numbers, green/red coloring
  4. Want full diff view → Must find "View changes in this step" text link, or switch to Changes tab

Findings:

# Severity Finding
W7 HIGH Clicking a chip opens content below the entire chip row, not inline. Mental model: clicking a chip should reveal details about THAT chip. Instead, ALL chip content goes into a single expanded panel below all chips. If I click "auth.ts" (edit), I see diffs for auth.ts but the panel position is disconnected from the chip I clicked.
W8 MEDIUM Read chips (violet) are confusing. Files the agent read aren't "changes" — showing them alongside created/edited files conflates observation with action. A user asking "what did it change?" has to filter out violet chips mentally.
W9 MEDIUM Only one file can be expanded at a time. Clicking a second chip collapses the first. Users comparing changes across files must toggle back and forth.
W10 LOW Chip parent directory shown as suffix. auth.ts components/ reads backwards — should be components/auth.ts or just show the chip tooltip on hover.

Flow: "Should I approve this?"

  1. See approval banner → Inline in step list, color-coded
  2. Read description → Approval shows what action is proposed
  3. Decide → Approve or reject button

Findings:

# Severity Finding
W11 MEDIUM Approval context is disconnected from the work. The approval banner appears inline but doesn't show which files are affected or what the agent plans to do next. Users must expand surrounding steps to understand context.

Resilience Testing

# Severity Finding
W12 HIGH Post-run expansion was completely broken. (Fixed in bdc110fe — step_id was missing from snapshot transcript. But this shows the SSE-vs-REST parity is fragile.)
W13 MEDIUM SSE reconnection doesn't restore expanded state. If the connection drops and reconnects, all step expansions collapse. The snapshot reloads fresh state with no expansion.
W14 LOW Filter chips don't survive page refresh. Active filter is local state, lost on reload.

Big Questions

  • Would I come back? Yes, but I'd want a "summary view" that just shows me: what changed, is it stuck, does it need me. The current UI makes me dig.
  • Could I teach a colleague in 2 minutes? Maybe — the chip system and expand/collapse would need explaining.
  • One thing to make it twice as easy? Auto-expand the active step and auto-collapse completed steps, so the default view answers "what's happening now?" without clicking.

Skill 2: UI Design Review (Heuristic + WCAG)

A. Accessibility (WCAG 2.1/2.2) — CRITICAL FAILURES

Level A Violations

ID WCAG Component Issue Impact
A1 2.1.1 Keyboard StepHeader <div onClick> with no role="button", no tabIndex, no onKeyDown. Keyboard users cannot expand/collapse steps. Blocks all keyboard users
A2 2.1.1 Keyboard FilesTouchedChips Chip <button> is correct but touch targets are ~20×16px (px-1.5 py-0.5 text-xs). Below WCAG minimum. Difficult on mobile/touch
A3 2.1.1 Keyboard CommandChips Same touch target issue as A2. Difficult on mobile/touch
A4 1.3.1 Info & Relationships StepListView Step list uses <div> not <ul> or role="list". Steps lack role="listitem". Screen readers can't announce "list of N items." Structural confusion
A5 1.3.1 Info & Relationships StepSearchBar Search results in plain divs, not role="listbox" with role="option". No aria-selected on active result. Screen reader users can't navigate results
A6 4.1.2 Name/Role/Value StepHeader Status icons (spinner, circle, check) have no aria-label. Color is the only indicator. Screen readers silent on step status
A7 4.1.2 Name/Role/Value FilesTouchedChips Icons <FilePlus>, <Pencil>, <Eye> have no aria-hidden and no text alternative on the button. Cluttered screen reader output
A8 4.1.2 Name/Role/Value CommandChips <Terminal> icon same issue. Markdown toggle <Code> icon uses title not aria-label. Inaccessible toggle
A9 1.4.1 Use of Color All chips Create/edit/read/terminal distinguished only by color (blue/green/violet/amber). No text label for the operation type. Color-blind users can't differentiate

Level AA Violations

ID WCAG Component Issue
AA1 4.1.2 All expandables No aria-expanded on any toggle button/div. Screen readers can't determine if a section is open or closed.
AA2 1.4.11 Non-text Contrast Chip icons 10px icons at color/10 opacity against card background may not meet 3:1 contrast ratio for UI components.
AA3 2.4.7 Focus Visible StepHeader div Not focusable (no tabIndex), so no focus indicator appears.
AA4 1.4.13 Content on Hover Chip title tooltips Full path shown via title attribute only — not persistent, not keyboard-accessible, not hoverable.
AA5 — StepSearchBar Filter chip buttons have no aria-pressed to indicate toggle state.

B. Visual Design Assessment

Strengths: - Clean Tailwind-based design system with consistent color palette - Good use of border-left treatment for step status (blue active, green done, gray pending) - Monospace for code/paths, sans-serif for labels — clear type separation - Dark mode support via Tailwind dark: variants

Issues: - HIGH: Visual hierarchy is flat — completed steps, active steps, and pending steps all have similar visual weight except for the thin border-left color - HIGH: 4 chip color categories (blue/green/violet/amber) plus 3 step status colors (blue/green/gray) = 7 semantic colors competing in one view - MEDIUM: Text density is high — tool count, duration, relative time, summary, label, and chips all on one step. No information grouping or progressive disclosure - MEDIUM: Expanded content (tool args, results, diffs) appear in a max-h-64 scrollable box. Nested scrollable regions inside a scrollable list is disorienting

C. Usability Heuristics (Nielsen)

Heuristic Score Notes
1. Visibility of system status 7/10 Active step shown with spinner + current tool. But no progress overall (e.g., "step 3 of 7").
2. Match between system & real world 6/10 "Steps" is clear. But "collapsed" vs "visible" tool visibility is internal jargon. Chip colors lack real-world mapping.
3. User control and freedom 5/10 No way to collapse all / expand all. No way to expand only chips without tool details. Filter chips are toggle-only, no combine.
4. Consistency and standards 7/10 Chip pattern is consistent. But StepHeader is a div-with-onClick while chips are buttons — inconsistent interactive patterns.
5. Error prevention 6/10 No issues with destructive actions (approve/reject has confirmation). But filters silently dim steps with no clear "clear filter" action.
6. Recognition vs recall 5/10 Chip colors require learning (what does violet mean again?). No legend or tooltip explains the chip type system.
7. Flexibility and efficiency 6/10 No keyboard shortcuts for step navigation. No "expand next" or "collapse all" shortcuts.
8. Aesthetic and minimalist design 5/10 Too much shown by default on completed steps. Chips, badges, times — all visible without asking.
9. Error recovery 7/10 Search results are navigable. Filter can be toggled off.
10. Help and documentation 3/10 No tooltips explaining chips, no onboarding for the step system, no help text.

D. Responsive Design

  • Desktop (1280px+): Good. Sidebar + step list + diff viewer.
  • Mobile (375px): Functional but crowded. Chips wrap to multiple lines. Expanded diffs in max-h-64 scrollable boxes within the main scroll = double-scroll problem.
  • Tablet (768px): Untested — likely OK since it's mostly vertical.
  • Touch targets: Chips are px-1.5 py-0.5 ≈ 24×16px. Well below 44×44px minimum. ToolCallRow has min-h-[44px] on mobile but chips don't.

Skill 3: UX Research (Personas + Journey Mapping)

Personas

Persona 1: "Alex the Reviewer" (Tech Lead)

  • Context: Supervises 3 agents running in parallel. Checks in every 30 min.
  • Goal: Quickly assess which jobs need attention, which are fine.
  • Pain: Must click into each job and mentally scan steps to determine status. Wants a "needs attention" summary per job.
  • Design implication: Need a job-level summary that says "3/7 steps done, editing 4 files, no errors" without opening the detail view.

Persona 2: "Jamie the Solo Dev" (Primary persona)

  • Context: Running one agent, watching it work. First week using CodePlane.
  • Goal: Understand what the agent is doing, feel confident it's on track.
  • Pain: Information overload — chips, badges, tool counts, durations all visible at once. Doesn't know what violet chips mean. Unsure whether to expand steps.
  • Design implication: Default to a scan-friendly view. Show less on completed steps. Explain chip types on hover or with a legend.

Persona 3: "Sam the Power User" (Daily user)

  • Context: Uses CodePlane 8 hours/day. Runs 10+ jobs.
  • Goal: Efficiency. Jump to what matters, skip the rest.
  • Pain: No keyboard navigation through steps. Can't expand/collapse all. Can't filter "show me only steps that changed files."
  • Design implication: Keyboard shortcuts, batch expand/collapse, persistent filters.

Journey Map: "Review Agent Output"

Start → Scan Steps → Identify Key Changes → Review Diffs → Decide → Approve/Reject

Stage           | Emotion | Pain Points
----------------|---------|--------------------------------------------
Scan Steps      | 😐 Neutral | All steps same visual weight. Hard to find the important ones.
Identify Changes| 😟 Confused | Chip colors not self-explanatory. Must click each to see.
Review Diffs    | 😊 Good | Inline diffs are nice. Step-scoped diff button = great feature.
Decide          | 😐 Neutral | Approval context disconnected from changes.
Approve/Reject  | 😊 Good | Clear action buttons.

Key Opportunity: The "Scan Steps" and "Identify Changes" stages are where users lose the most time. Reducing the information shown for completed steps and making the step-level summary ("edited 3 files, added auth middleware") more prominent would save users 5-10 seconds per step.

Prioritized Recommendations

P0 — Fix Immediately (Accessibility Blockers)

  1. StepHeader: Add role="button", tabIndex={0}, onKeyDown, aria-expanded
  2. Files: StepHeader.tsx

  3. WCAG: 2.1.1, 4.1.2

  4. All chips: Add aria-expanded, aria-hidden on icons, aria-label with operation type + full path

  5. Files: FilesTouchedChips.tsx, CommandChips.tsx

  6. WCAG: 4.1.2, 1.4.1

  7. Step list: Add role="list" on container, role="listitem" on step divs

  8. Files: StepListView.tsx

  9. WCAG: 1.3.1

  10. Filter chips: Add aria-pressed

  11. Files: StepSearchBar.tsx

  12. WCAG: 4.1.2

  13. Status icons: Add aria-label on Loader2/Circle/CheckCircle

  14. Files: StepHeader.tsx
  15. WCAG: 4.1.2, 1.4.1

P1 — High Priority (UX)

  1. Auto-collapse completed step chips. Only show chips on the active step and the most recently completed step. Others show a compact summary ("edited 3 files").

  2. Files: StepContainer.tsx, FilesTouchedChips.tsx, CommandChips.tsx

  3. Increase chip touch targets. Add min-h-[32px] px-2 py-1 (at minimum) on mobile, ideally min-h-[44px].

  4. Files: FilesTouchedChips.tsx, CommandChips.tsx

  5. Add chip type labels. Show "Created", "Edited", "Read", "Terminal" text (or sr-only text) alongside icons to reduce color-only reliance.

  6. Files: FilesTouchedChips.tsx, CommandChips.tsx

  7. Promote "View changes in this step" button. Make it a styled button with icon, not a bare text link.

  8. Files: StepContainer.tsx

P2 — Medium Priority (Polish)

  1. Search input: Add aria-label, role="combobox" pattern, role="listbox" on results.

    • Files: StepSearchBar.tsx

  2. Add aria-live="polite" to streaming text and current tool indicator.

    • Files: StepContainer.tsx

  3. Remove relative time badge from completed steps (keep duration only). Reduce badge count.

    • Files: StepHeader.tsx

  4. Add expand-all / collapse-all control to step list header.

    • Files: StepListView.tsx

  5. Add step progress indicator ("Step 3 of 7") to header when steps exist.

    • Files: StepListView.tsx header

P3 — Low Priority (Nice to Have)

  1. Add keyboard shortcuts for step navigation (j/k to move, Enter to expand).
  2. Persist filter state across page refreshes (URL params or session storage).
  3. Show a chip legend/help tooltip on first visit.
  4. Consider a "summary mode" for completed jobs that hides all tool details and just shows step labels + file lists.