# Evaluation plan (proposed) — Turbine 7

> Kit 9 · Turbine 7 · written before any pilot · nothing below has been measured · fictional case

## Status

The build exists; no learners have used it. Every number on this page is a planned threshold or decision rule, not a result.

## What the build already records

Per shift, inside the browser only (nothing is sent anywhere): the call and its score for each case, the named pattern, the production-desk reply, critical errors, the pass, which parts were opened before the call (3D), and the handover-note rubric scores. A pilot would export these by hand or through an LMS wrapper; the build does not collect data on its own.

## The questions

| # | Question | Measure | When | Decision rule |
|---|---|---|---|---|
| 1 | Can technicians make the call? | Score and critical errors per case | Pilot, 8–10 technicians, two shifts each | Any case with more than 30% wrong calls gets its evidence or feedback rewritten before rollout |
| 2 | Do they read the evidence or guess? | Pattern accuracy against call accuracy; in 3D, whether the deciding part was opened before the call | Same pilot | Right calls with wrong patterns means guessing: add a worked example for that pattern |
| 3 | Can they defend the call in writing? | Handover-note scores; a shift lead re-marks a sample of 20 notes with the same rubric | Same pilot | If the auto-check and the shift lead disagree on more than a quarter of notes, change the check or drop the auto-score |
| 4 | Is the 3D worth its cost? | Half the pilot uses 2D, half 3D; compare question 2 and time per shift | Same pilot | If 3D adds no difference in evidence-reading, ship 2D as the default and keep 3D for the huddle |
| 5 | Does it change restarts on shift? | Sample of real restart log entries: share whose handover note names a pattern | 3 months after rollout | Agree the sampling with shift leads before launch so it is not seen as policing |
| 6 | Is production pressure still winning? | Restarts into rising trends, from the incident log | 12 months | Too rare to credit to training alone; review with the operations manager as an operations measure |

## Pilot outline

- **Who:** 8–10 control-room technicians across at least two shifts, plus two shift leads.
- **What:** two simulator shifts each (one per week), one huddle, the alarm-pattern card on the wall.
- **Collect:** the in-build results, a 5-minute survey ("Which case was hardest and why?", "Would you use the card?"), shift-lead notes from the huddle.
- **Before the pilot:** the expert-review checklist completed by a wind O&M SME, and the escalation boundary agreed with the shift leads.

## What this plan cannot show

A pilot this small cannot show an effect on incidents. It can show whether the cases are fair, whether people read the evidence, and whether the patterns become shared language on the desk. Those are the claims the case study will make, if the pilot happens.