Code what people wrote, such as what patrons ask a library's chat page. Research → Coding runs in four steps: codebook, sample, code, compare.
Codebooks
Pick a codebook:
- Help-seeking: executive or instrumental requests (Nelson-Le Gall).
- ICAP engagement: passive, active, constructive, interactive (Chi and Wylie); applying it to chat is an interpretation to state.
- Question depth: shallow, intermediate, deep (Graesser and Person).
- Accuracy of replies: accurate, partly accurate, inaccurate, didn't answer.
- What patrons ask: a library's request categories.
- Your own codebook: 2 to 20 codes.
Each code has a label, a definition, examples and near misses. Changing codes after coding makes a new version, with a note on why; each sample keeps its version.
| Unit | Coders read |
|---|---|
| Messages | The learner's message, with the AI's answer before it |
| Turns | The message, the answer before it, and the reply to it |
| AI replies | The question, the reply, its sources and the learner's rating |
| Spoken answers | The question, what counts as a good answer, the recording and transcript |
Draw a sample
- Choose the codebook. Select New sample and pick it.
- Set where it is drawn from. Under Drawn from, choose the period, study, chat pages or courses, and agents. Only people included in the study is on by default.
- Draw it. Enter Items to sample (up to 2,000) and, optionally, a Seed, then select Draw the sample.
The same seed and choices over a finished period draw the same items. People who leave the study leave its samples.
Code, alone and with model coders
On Code, code one item at a time with keys 1 to 9. Coding is blind: you see only your own codes until you open Review & decide. Any member, viewers included, can code.
Model coders → Add a model coder adds a model as one more coder, billed to a key you choose. Under How strictly, Only these codes gives no code when nothing fits, never the closest. These codes, and suggest new ones may also propose codes. About your study adds context to its prompt.
For each item it returns codes, a confidence, a quote kept only if it appears word for word, and a reason. You can read its exact prompt.
Suggestions lists proposed codes: Accept as a new code, Merge into a code or Dismiss.
Agreement, then final codes
Agreement compares every two coders on items both coded: percent agreement, Cohen's κ and a confusion table. For your methods section drafts a paragraph to copy.
In Review & decide, select Show everyone's codes, which ends blind coding for you. Pick the final codes or No code fits, add a note, and Decide. Use the agreed codes for N settles items where coders agree. Owners and editors decide. Final codes feed an observational study's Summary and coding_final.csv.
- The research export: codes, codebooks and prompts in your data.
- Consent and taking part: who stays in your samples.