Before you download anything
Does this kit fit your study?
This approach works by finding patterns across a large number of individually-marked questions. Some VCE studies give you that; some don't. Open a recent exam, its examiner report and the study design, and answer these three.
Question 1 of 3
Open a recent exam. How many separately-marked questions are there in the whole paper?
Question 2 of 3
Open the matching examiner report. Does it give you a number for each question — an average mark, or a distribution of how many students scored 0, 1, 2 and so on?
Question 3 of 3
Open the study design. Could you list what's examinable as discrete content points a single question could target?
Verdict
Answer all three to see where you stand
The result isn't softened to be encouraging. A tool built on data that can't support it looks authoritative and says almost nothing, which is worse than no tool.
Facilitator
Do this live, together, before anyone opens a browser tab. It takes a minute and it's the difference between an afternoon well spent and a tool that misleads a faculty. Expect English, Literature, EAL and Languages to land on stop — say plainly that a real tool for those studies measures different things (prompt archetypes, criteria against report commentary, stimulus patterns, text list movement) and needs its own worked exemplar. Don't offer to adapt this one.
Phase 1
The project
A Claude Project holding your study's official documents. Everything downstream is read out of these files, so this is the only phase where nothing can be automated — it needs your judgement about what's current.
Step 1
Gather the source files
On the VCAA site: Study Designs → your study, and the Past examinations page.
≈30 min
| File | Why it matters |
|---|---|
| This kit, as a document↓ below | So Claude follows the method instead of improvising one when you go off-script |
| Study design (current) | Becomes the key knowledge index — the spine of the whole tool |
| Support materials / FAQs | Clarifies grey areas. Often answers exactly what students get wrong |
| Specifications, conventions, glossaries | Subject-specific rules a question writer must respect |
| Sample exam + answers | The intended shape of the paper before any live exam existed |
| Past exam papers — every year under the current study design | The question corpus. Every one gets coded |
| Insert / resource / stimulus books | Essential if your exam supplies stimulus. Without them the papers don't parse |
| Examiner reports — every matching year | Where the performance data comes from. Skip one and that paper's questions have no average |
Current study design only
Older papers were written against a syllabus that no longer applies, and mixing them in corrupts the coverage analysis — you'll get key knowledge points that look well-assessed but were assessed under different wording. One or two compliant papers is still workable: the tool leans harder on the sample exam and reports honestly that the evidence base is thin. Say so rather than padding it out with superseded papers.
The failure that hides itself
Scanned PDFs won't work, and they fail silently — Claude simply doesn't see those questions, and never says so. Open each file and try to select text with your cursor. If you can't select it, it's a scan: run it through OCR (Acrobat, or free tools like OCRmyPDF) and upload that version instead.
Download the method document
Everything on this page, written out, plus the reasoning behind each step. It goes into your project alongside the VCAA files. The prompts on this page are self-contained, but a project holding only your PDFs knows your subject and not the method — and the moment you ask Claude something the prompts don't cover, it invents a method rather than following one. This is the file that stops that.
Download the method (.md)
Upload it into your project at Step 2. It's a text file — Claude reads it directly, and so can you.
Don't take the selectable-text test on trust. Before you upload thirty files, upload one exam paper to a chat and run this:
I've attached one VCE {{STUDY}} exam paper. Don't analyse it yet.
Tell me only this:
1. Can you read the text of this PDF, or is it a scanned image?
2. Quote the exact wording of the last separately-marked question in the paper, and its mark allocation.
3. How many separately-marked questions are in the whole paper, and what do the section totals add to?
If you can't read any part of it, say which pages.
If it can't quote the last question back to you accurately, the file is unusable and so is every other file that looks like it. Fix them all before Step 2.
Name files so the year and type are obvious — 2025-Biology-exam.pdf, 2025-Biology-report.pdf. Claude cites by filename, so this is what makes its coding checkable later.
Facilitator
This is the step people do at home, and the step where the whole build quietly dies. If you have time in the session, get everyone to run the pre-flight check on one paper while you're in the room — a scan discovered now costs five minutes, a scan discovered in Phase 2 costs the afternoon.
Step 2
Create the project and upload
Your own project, for your subject only.
≈10 min
- In the Claude sidebar, open Projects, then Create project.
- Name it for your study — VCE [your study] Exam Analysis.
- Upload everything from Step 1 into the project's knowledge — including
vce-exam-analysis-method.md, the method document you downloaded.
Why the method goes in too
With it in the project, Claude has both halves: the method for building the tool, and your study's documents to build it from. Without it, every prompt on this page still works — but the first time you ask something the prompts don't cover, you get a confident answer assembled from nothing in particular. The project instructions in Step 3 point at this file by name, so upload it before you paste them in.
The reference tool comes later
You don't need it yet. It's a finished tool for a different VCE study, which Claude reads as the structural model when you build yours in Phase 3 — a working exemplar beats any amount of description in a prompt, so it's the single most reliable thing you can do. It's in the handover pack you'll download at the end of Phase 2, along with your filled-in project instructions.
One subject per project
Don't share a project with a colleague teaching a different study, and don't add a second study's papers to yours. Projects don't partition: if Biology papers and Legal Studies papers sit in the same project, Claude reads both at once with no way to tell which study it's working on — and it will happily code a question from the wrong subject. A colleague building for their own study makes their own project and downloads the method document again.
Facilitator
Screen-share this one rather than describing it. The two things people get stuck on are finding Projects in the sidebar, and the difference between attaching a file to a chat message and adding it to the project's knowledge. Show both, and say which one this is.
Step 3
Paste in the project instructions
In your new project, open its instructions and paste this in whole. It's what holds the accuracy line for every message that follows.
≈2 min
ROLE
You are an experienced VCE {{STUDY}} teacher, exam assessor and data analyst,
working with a teacher who is building an exam analysis tool for their faculty.
The method we are following is in this project as vce-exam-analysis-method.md.
Follow it. Where a request and the method disagree, say so rather than quietly
picking one.
YOUR SOURCES
Everything you produce must be grounded in the files in this project:
- The current study design is the authority on what is examinable and on
correct terminology. Never introduce content outside it.
- The examiner reports are the authority on state-wide performance and on what
loses marks. Draw on them heavily.
- Past exam papers are the corpus. Quote question stems accurately.
Cite the file and year for every specific claim. If the files don't tell you
something, write UNKNOWN rather than guessing.
ACCURACY RULES — these matter more than fluency
- Never invent a statistic, an average mark, or a question that doesn't exist.
- Never estimate a figure that could be read from a file. Read it.
- When you tag a question against key knowledge, you must be able to point to
the words in the question stem that justify the tag.
- Flag your own uncertainty inline rather than presenting a guess cleanly.
WHAT NOT TO DO
- Don't reproduce an entire past exam paper on request.
- Don't state that a future exam will contain something. Rotation patterns are
hypotheses about likelihood, never forecasts, and the whole study design
remains examinable every year.
HOLD ME TO THE VERIFICATION
Phase 2 is where this becomes analysis or decoration, and it is the part most
likely to be rushed — checking tags is slower and duller than watching a tool
appear. Every error caught there is an error that would otherwise have shipped
invisibly.
- Tell me to save the key knowledge index and each coded paper to its own .txt
file as I go, and to run the lot through the Tag Checker at
https://almanac.jotty.au/tagcheck/ before reading tags by eye. It finds
unknown codes, impossible averages, section totals that don't reconcile and
distributions that disagree with their own averages, so my attention goes to
the judgement call instead. It also marks which questions can't be judged
without the exam paper open.
- If I say the coding is done, don't congratulate me and move on. Ask which of
the four checks I ran, and ask me to name one tag I disagreed with and
changed. If I can't name one, I haven't read the justifications — say so, and
send me back.
- Don't reassure me that a thin dataset will be fine. If I have one compliant
paper, say what that limits and what the tool must state in its own overview.
RETURNED TAG CHECKS
A file headed "TAG CHECK RETURNED BY A TEACHER" is a verification pass done
outside this project. Follow the instructions at the top of it. In particular:
check each change against the source paper before applying it, and query rather
than silently apply anything that contradicts the paper. A teacher's correction
can be wrong too.
THIS IS A TEACHER'S TOOL
There is no student view of it and there won't be. Given a ranked likelihood
list, students revise the top few and drop the rest, which is the opposite of
what the analysis is for. What reaches students is my teaching and revision
advice, filtered through my own judgement. If I ask for a student-facing
version, say plainly why not.
The rule about future exams matters. Your own examiner reports warn students against relying on past papers — the tool has to hold that line, or it teaches exactly the behaviour VCAA is cautioning against.
Why the instructions push back at you
Three of these blocks aren't about accuracy, they're about pacing. Nobody is standing behind you during Phase 2, so the project has to do it: ask for the four checks by name, refuse to accept "done" without one tag you changed, and decline to talk you into a student version. Paste it in whole — the parts that sound like nagging are the parts that catch the failure.
Phase 2
The dataset
This is the real work, and it's where your subject expertise earns its keep. Three passes, each verified before the next. The dataset is the product — the interface in Phase 3 is only a window onto it.
Step 4
Build the key knowledge index
The spine of everything else. A gap here becomes an invisible blind spot.
≈15 min
Read the study design and produce a complete index of examinable key knowledge
for Units 3 and 4, as a JavaScript object.
Format each entry as:
"CODE": { o:"U3 AoS1", label:"short plain-language description" }
Use a code scheme of [Unit][Outcome]-[number], e.g. "U3O1-1". Add separate
codes for any cross-cutting terminology the study design defines separately
and that questions could target directly.
Rules:
- Cover every key knowledge point in Units 3 and 4. Completeness matters more
than brevity — this index is the spine of everything else.
- Keep labels under about 15 words, in language a student would recognise.
- Cite the study design page number for each entry.
Save this index to a file — a plain
.txt is fine. You'll load it into the Tag Checker at Step 5, which is what turns a code like U3O2-4 back into words you can actually check a question against. Save each paper's coded questions to its own file as you go, too.
Tick both checks first.
Step 5
Code every question
One paper per answer, always — several in one answer produces shallower tagging. You can still queue a run of papers and say next between them. Then check the tags yourself: this is the highest-value hour in the whole process.
≈25 min per paper
Tick every paper you're going to code
{{QHEAD}}{{QLIST}}{{QMID}}Code every question in the {{PAPER}} {{STUDY}} exam as JavaScript objects in this
shape:
{ paper:"{{CODE}}", sec:"A", q:"1", marks:4, cmd:["Identify","Discuss"],
kk:["CODE1","CODE2"], avg:2.3, dist:[19,9,17,35,20],
stem:"the question stem, quoted accurately",
why:{ "CODE1":"the words in the stem that justify this tag",
"CODE2":"the words in the stem that justify this tag" },
examiner:"what the report says about this question specifically",
choice:"what the student gets to choose, if anything" }
- cmd: every command term in the question, in the order they appear.
- kk: the key knowledge codes from the index that the question actually
assesses. Usually one to three. Only tag what the question truly requires,
not everything it touches — over-tagging flattens the coverage analysis.
- avg: the average mark from the examiner report for that question. Read it
from the mark distribution table. If the report gives a distribution rather
than an average, calculate it and show your working. Omit avg entirely if
the report doesn't cover this paper.
- dist: the mark distribution from that same table, as an array running from 0
marks up to full marks — so a 4-mark question has 5 numbers. Use the report's
own figures, usually percentages of the state cohort. Omit dist if the report gives
only an average. You are already reading this table for avg, so capturing it
costs nothing and it is what makes avg checkable later.
- why: one entry per kk code, quoting the words in the stem that justify that
tag. Quote the stem, don't paraphrase it. Every code in kk needs an entry.
- examiner: what the report says about THIS question — what students got wrong,
what a full-mark answer did. One or two sentences, close to the report's own
words. You are already on that page reading avg and dist. Omit it where the
report says nothing about the question; don't pad it out with generic advice.
Use exactly these field names. Don't add fields I haven't asked for.{{QTAIL}}
What a run does and doesn't save you
It saves the re-pasting, not the reading. Each paper still comes back in its own reply, and still needs its own file and its own pass in the Tag Checker — say next only once you've saved the one in front of you. Asking for two papers in a single answer is the one thing this prompt won't do: a reply running out of room drops questions, and it drops the
why justifications first, so what comes back looks complete and is thinner than it should be. That's why the prompt asks for a count line at the end of each paper — check it against the paper before you save.
Which model, and how to tell whether it's coping
Use the most capable model your plan offers, with extended thinking on if it's offered. This step asks for the hardest thing in the kit — reading a mark distribution table out of a report, matching it to the right question, quoting a stem exactly, then defending a tag. The model is a per-conversation choice rather than something the project remembers, so it's worth a glance each time you start a paper, not once at the beginning.
You don't have to take the setting on trust, and this is the better habit anyway. Three things in the first paper that comes back will tell you, and the Tag Checker reports all three: do the
You don't have to take the setting on trust, and this is the better habit anyway. Three things in the first paper that comes back will tell you, and the Tag Checker reports all three: do the
why entries quote words from the stem or just paraphrase it; does dist reconcile with avg; is nearly every question carrying four or more tags. If those look poor, change something — the model, a conversation that's gone long, a paper you tried to rush — rather than only fixing the file in front of you. Thin coding has causes, and it produces the same confident wrong answers however it got there.
Change the paper above each time you run it — it rewrites the sentence and the paper code in the example together. Get that code wrong and Claude copies it, and every question in the file ends up stamped with the wrong paper: nothing downstream can tell, and the coverage matrix quietly attributes a year's questions to another year. Keep track here — a paper isn't done when it's coded, it's done when it's verified.
Two terms this kit never lets drift
Two terms, always paired, never shortened: the state cohort is the VCAA examiner report and every candidate who sat the paper. The school cohort is one school's own students. The word cohort never appears on its own anywhere in this kit or in the tool you build, because on its own it could mean either.
They are never the same figure and they never share a field:
They are never the same figure and they never share a field:
avg and dist are the state cohort, always, and school cohort figures live in a separate file the tool loads at run time. A number in the wrong field is the one error nothing downstream can detect.
One failure, five disguises
Three of the five live in this step. A stale paper code stamps a file with the wrong year. Several papers in one answer thins the tagging. A weak model or a long conversation thins it the same way. The other two arrive later: a school figure landing in
They look like five problems and they're one. In every case the output is well formed — right shape, right fields, plausible numbers — so there is nothing to see, and no correct version to compare it against. A truncated file is loud. These are silent, and they stay silent for as long as the tool exists.
You can't catch this kind of error by looking at the result. Only against a second source, or by preventing it at the point of entry — which is what the paper picker, the run queue, the count line and the checks below are all for.
avg, and a digit misread off a scanned marks table.They look like five problems and they're one. In every case the output is well formed — right shape, right fields, plausible numbers — so there is nothing to see, and no correct version to compare it against. A truncated file is loud. These are silent, and they stay silent for as long as the tool exists.
You can't catch this kind of error by looking at the result. Only against a second source, or by preventing it at the point of entry — which is what the paper picker, the run queue, the count line and the checks below are all for.
Don't skip this next part
Checking tags is slower and duller than watching a tool appear, and this is where nearly everyone stalls. Every error you catch here is an error that would otherwise have shipped invisibly — a beautiful interface over a badly coded dataset produces confident, wrong answers about where the marks sit, and nobody notices, because it looks authoritative.
Do this in the Tag Checker
The Tag Checker is the other half of this kit. Drop in the index and every question file — it works out which is which — and the dataset becomes a proof sheet: codes resolved to their labels, justifications beside the stems, and every mechanical error found for you. It flags; it never fixes.
You can change the tags in place there: codes are picked from your own index, so a code that doesn't exist can't be created, and every tag change needs a reason — which then becomes the stored justification. There's a tick per question, so you can stop at 23 and come back. It also marks which questions can't be judged without the exam paper in front of you, and filters to just those. When you're done it writes a corrections file to paste back into your project.
Nobody needs a Claude account to use it. That makes this step delegable: send a colleague the dataset files, the papers and that link, and they send back the corrections file. Two people checking a paper each is closer to how you'd moderate anything else — and it beats the person who did the coding being the only one who ever reads it.
You can change the tags in place there: codes are picked from your own index, so a code that doesn't exist can't be created, and every tag change needs a reason — which then becomes the stored justification. There's a tick per question, so you can stop at 23 and come back. It also marks which questions can't be judged without the exam paper in front of you, and filters to just those. When you're done it writes a corrections file to paste back into your project.
Nobody needs a Claude account to use it. That makes this step delegable: send a colleague the dataset files, the papers and that link, and they send back the corrections file. Two people checking a paper each is closer to how you'd moderate anything else — and it beats the person who did the coding being the only one who ever reads it.
Once the flags are clear, the part no checker can do:
Facilitator
If you do one thing in the live session, do this: code one paper together on the screen, and find a wrong tag in front of the room. People need to see that the tagging is genuinely arguable before they'll believe it's worth checking. Say out loud that this hour is the difference between analysis and decoration.
A good sequence: open the Tag Checker, drop in a dataset with a planted fault, let the room watch every mechanical error appear at once — then close the laptop lid on that and find a wrong tag by eye. The contrast is the lesson. The machine half is instant; the judgement half isn't, and that's the hour you're asking them for.
A good sequence: open the Tag Checker, drop in a dataset with a planted fault, let the room watch every mechanical error appear at once — then close the laptop lid on that and find a wrong tag by eye. The contrast is the lesson. The machine half is instant; the judgement half isn't, and that's the hour you're asking them for.
Tick all five checks first. This is the gate that matters.
Step 6
Command term glossary and report findings
A generic definition of Explain is available anywhere. What a full-mark Explain looks like in your study, according to your assessors, is not.
≈10 min
1. From the coded questions, produce a command term glossary:
Term: { marks:[min,max], def:"generic definition",
note:"what the examiner reports specifically say about how this
term is handled in THIS study, with the year cited" }
2. From the examiner reports, list every weakness they name across all years,
with the year and the exact question it relates to. Mark which ones recur
in more than one report — those are the persistent ones.
3. List the terminology confusions the reports name directly. Search for
"confused", "incorrectly", "misunderstood", "uncertainty around".
Read the note field on a few terms. If it's a generic definition restated, push back and ask for the report wording and the year — that field is the reason the glossary is worth having.
Optional
Your own school's results
Only if you have your school's VASS results for a paper you've coded. Everything after this works without it.
≈15 min per paper
The examiner reports tell you how the state cohort did — every candidate who sat the paper. If your school can give you the VASS External Assessments — Questions/Criteria page for a paper in your dataset, the tool can also show how your students did on the same questions — and, more usefully, where they sit relative to their own overall standard.
Two places to type it, one file either way
In the tool you build. Step 7 includes a School figures view: pick the paper, and it lays out a row per coded question with the section, question number and marks already filled in. That is worth more than the convenience — a row for a question your tool hasn't coded can't be created, so a file for the wrong subject can't be built at all.
Or at School Results, a page of its own, which needs no Claude account and no built tool. Use it before Phase 3 exists, when a colleague is doing the typing, or to repair a file. Both write the same file and each reads the other's.
You type four numbers a row: school mean, state mean, school %, state %. Every row checks itself — the printed percentage is just the mean over Max — so a row only turns green when the two agree, and a mistyped digit can't slip through.
Do it that way rather than uploading the report or a photo of it. The page carries a privacy notice, your school's name and a teacher's name, and none of that needs to go anywhere. What comes out is a small file of aggregate figures no student appears in.
Or at School Results, a page of its own, which needs no Claude account and no built tool. Use it before Phase 3 exists, when a colleague is doing the typing, or to repair a file. Both write the same file and each reads the other's.
You type four numbers a row: school mean, state mean, school %, state %. Every row checks itself — the printed percentage is just the mean over Max — so a row only turns green when the two agree, and a mistyped digit can't slip through.
Do it that way rather than uploading the report or a photo of it. The page carries a privacy notice, your school's name and a teacher's name, and none of that needs to go anywhere. What comes out is a small file of aggregate figures no student appears in.
Why the tool won't fill the state column in for you
It could. It holds the examiner report average for every question, and the state column is that same figure on the report's scale. It stays blank on purpose.
The state column is a second transcription, from a different document — and comparing it with
It costs two more numbers a row. It is the difference between a check and a decoration.
The state column is a second transcription, from a different document — and comparing it with
avg is the only check in this whole method that reads two sources against each other. Everything else checks the dataset against itself. Pre-fill that column and the check compares a number with itself: still green, now meaningless. So you type it without the tool's figure on screen, and only then does it tell you whether the two agree.It costs two more numbers a row. It is the difference between a check and a decoration.
Know what you're looking at
The School column is every student taking the subject at your school, not your class. If your school runs several classes, this is a faculty signal and it isn't a verdict on any one teacher's work. If it runs one, school and class are the same people — worth deciding, before you circulate anything, who that comparison is for.
A school whose students are stronger sits above the state on nearly everything, so the raw gap says very little. The finding is where a question departs from your own average gap. The page states that average and names the rows furthest from it.
A school whose students are stronger sits above the state on nearly everything, so the raw gap says very little. The finding is where a question departs from your own average gap. The page states that average and names the rows furthest from it.
What comes out is one object per paper, kept separate from the coded questions so that school figures can never be mistaken for the examiner report's. It is also the save file: export part-way, load it back, carry on typing. Keep it and re-load it next year rather than typing the page again.
For reference only — the tool and the page both write this file for you
const SCHOOL_{{CODE}} = {
meta: { study:"{{STUDY}}", paper:"{{CODE}}", label:"{{PAPER}} exam",
scope:"school", classes:"several", n:22,
source:"VASS External Assessments — Questions/Criteria",
entered:"2026-02-14" },
rows: [
{ sec:"A", q:"1", marks:4, max:8,
school:5.5, schoolPct:69, state:4.5, statePct:56, diff:"Higher" }
]
};
- study and paper together say which exam these figures belong to. Every VCE
study numbers its papers the same way, so E25 on its own means the 2025 exam
of some subject — the study name is what makes it yours. The tool you build
checks both and refuses a file that isn't for it.
- max is the report's own scale, normally twice the marks, because external
assessments are marked at least twice. school and state are on that scale.
To compare with a question's marks: school / max * marks. Derive it when you
need it; never store the converted figure as though the report printed it.
- state is a second source for the avg already in your dataset. Converted, the
two should agree to about 0.1.
- diff is VCAA's own significance wording, quoted verbatim. Never recompute it.
Then hand the file to your project with this:
I'm adding my school's own results for {{PAPER}} to this project, as a separate
SCHOOL_{{CODE}} object. Don't merge it into the coded questions.
First, check it against what we already have and report before changing anything:
- Every row must match a coded question by section and question number, and every
coded question in that paper should have a row. Name both kinds of mismatch.
- The marks on the report are VCAA's published allocation. If any disagrees with
the marks in our dataset, stop and tell me — the paper was coded against a
different allocation and that matters more than anything else here.
- Convert each state figure to the marks scale (state / max * marks) and compare
it with that question's avg. They come from the same source and should agree to
about 0.1. Where they don't, show both and say which files you'd check. Don't
pick one.
Then tell me, in plain language:
- this cohort's average gap against the state, across all questions
- the questions furthest above and furthest below that average — those are the
findings, not the raw differences
- which of those VCAA itself marked as a statistical difference, quoting its
wording
Treat a cohort above the state on nearly everything as the ordinary case for a
strong group, not as a result.
Where it goes in the tool: the cohort performance view, as a second block beside the state cohort figures, and the revision priorities view. It does not feed the prediction model — what VCAA chooses to examine has nothing to do with how your students did, and mixing the two would quietly change what the rankings mean.
If you built your tool before September 2026
Early builds accepted a school file on its paper code alone. Every VCE study numbers its papers the same way —
E25 is the 2025 exam in every subject — so a file from another study loads, joins whatever rows share a section letter and a question number, and draws a cohort view out of the coincidence. Real numbers, wrong exam, nothing on the view saying so. Step 7's prompt now builds the check in; the prompt below retrofits it to a tool you already have.
The school-file loader in our tool accepts a file it shouldn't. Every VCE study
numbers its papers the same way, so a paper code identifies a year and not a
subject: a SCHOOL_ file from another study loads, joins whatever rows happen to
share a section and question number, and renders a cohort performance view built
on the collision.
Rewrite the loader so it verifies the file BEFORE accepting any of it. Run these
in order and stop at the first failure:
1. meta.paper must be a paper this tool has coded. Remove any fallback that
picks the first object in the file when nothing matches — that fallback is
the bug.
2. If meta.study is present, it must match this tool's study, compared folded
to lower case with spaces and punctuation stripped.
3. Every row must join a coded question on sec and q. Rows are a subset of the
questions, never a superset.
4. Every row's marks must equal the coded question's marks.
5. Where the coded question has avg, state / max * marks must land within 0.5
of it. If more than a third of the checkable rows are out, the file
describes a different paper.
On any failure, load NOTHING — not the rows that passed — leave the tool exactly
as it was, and say which check failed, with the first three offending rows named.
Point me at almanac.jotty.au/school/ to repair the file. Never partly load.
A file with no meta.study was written before the study stamp existed. Accept it
if it passes the structural checks, and say the study name could not be checked
rather than reporting the file as verified.
Change nothing else yet — same views, same figures, same layout. Then tell me
which of the five checks my current file passes.
Once that is in, ask me whether I want the School figures entry view as well: it
generates the rows from the coded questions so a file for the wrong subject
cannot be built in the first place, and it types the state column blind so the
check against the examiner report still means something. Don't build it until I
say.
doesn't affect the 9 steps
Phase 3
The tool
A single self-contained HTML file your faculty can open, share and print from. It's only as good as the dataset underneath — which is why it comes last.
Step 7
Build it in stages
Don't ask for the whole thing at once. Shell plus two views, check them, then add views one per message.
≈20 min
Before you run this
Upload
reference-exam-analysis-tool.html — from the handover pack you downloaded at the end of Phase 2 — into your project's knowledge. The prompt below tells Claude to read it as the structural model, so without it there this step produces something generic. Use the most capable model your plan offers here too, for the same reason as Step 5 — though this failure is at least one you can see, because a weak build produces a tool that visibly doesn't work.
Using reference-exam-analysis-tool.html as the structural model,
build the equivalent tool for VCE {{STUDY}} as a single self-contained
HTML file — no external dependencies, no browser storage.
Use the verified dataset we built: the key knowledge index, the coded
questions, the command glossary and the report findings. Do not add data
that isn't in it, and do not smooth over gaps.
Start with the shell plus two views only:
- Overview: headline findings, papers included, what's been assessed in
every live paper and what's never been assessed at all.
- Question explorer: every coded question, filterable by paper, section,
command term and key knowledge.
Match the reference tool's approach to layout and interaction, but derive the
visual treatment from {{STUDY}} rather than copying the reference's palette.
Never build school figures into the file. Add a control in the shell that reads a
SCHOOL_ file from disk when I choose one, holds it in memory for that session
only, and shows which paper is loaded and that it is school data. With nothing
loaded the tool must work exactly as it does without the feature, and every
school block must say "no school cohort figures loaded" rather than showing an
empty chart. One file then circulates with no school results in it and each
teacher points it at their own.
The loader must CHECK THE FILE BELONGS TO THIS TOOL before it accepts anything,
because every VCE study numbers its papers the same way — E25 means the 2025 exam
in every subject, so a paper code on its own proves nothing. The reference tool
implements all five of these; match it. Run them in order, stopping at the first
failure:
1. meta.paper must be a paper this tool has coded. If it isn't, refuse and
list the papers this tool does hold.
2. If meta.study is present, it must match this tool's study. Compare them
folded to lower case with punctuation and spaces stripped, so "VCE Biology
Units 3 & 4" and "vce biology units 3and4" match. If it disagrees, refuse
and print both names.
3. Every row must join a coded question on sec and q. Rows are a subset of the
questions — never the other way round. If any row matches nothing, refuse
and name the first three.
4. Every row's marks must equal the coded question's marks. If any disagree,
refuse and name them: the figures are keyed to a mark allocation, and max
is derived from it.
5. Where the coded question has avg, state / max * marks should land within
0.5 of it. This is the identity test, not the transcription test — a file
from another paper fails it almost everywhere. If more than a third of the
checkable rows are out, refuse and say the file appears to describe a
different paper.
On refusal load NOTHING — not the rows that passed — leave the tool exactly as
it was, and say which check failed and what to do about it. Never partly load a
file: it looks like it worked, and what survives is the rows that collided by
chance. A file with no meta.study was written before the stamp existed: say the
name could not be checked and that it passed on structure alone, rather than
reporting it as verified.
A file may also arrive with an empty rows array and a draft array instead. That
is a part-way file, saved by someone still typing. It is not cohort data: no view
may read it. Put it through checks 1 to 3 anyway — a draft from another study
must be refused too — then use it to fill the entry form, and say on the face of
the tool that the file is part-way rather than finished.
Then add the remaining views one per message. Each has its own prompt below — open it, copy it, paste it, check what comes back, then move to the next. Which ones you should build depends on your diagnostic result, and a view marked Restrict has that limit written into its prompt already.
This is a teacher's tool
Don't build a student view of it, and don't hand the file to a class. Students given a ranked likelihood list will study to it — they'll revise the top five and drop the rest, which is the opposite of what the analysis is for. Use it to decide what you teach and in what order; what reaches students is your teaching, and revision advice you've already filtered through your own judgement.
Step 8
Test it against things you already know
Four checks. You are the only instrument that can run them.
≈15 min
The rule for anything you can't trace
If the tool states a figure you can't trace, ask it which file and page it came from. Anything untraceable comes out — not softened, not caveated. Out.
Run all four checks first.
Step 9
Share it, and keep it honest
It's a single HTML file. Email it, drop it in a shared drive, or share the project so colleagues can regenerate it.
≈5 min
Each year, when VCAA publishes the new exam and report: add both to the project, code the new paper with the Step 5 prompt, and rebuild. Then compare the new coverage matrix against last year's — what rotated in and out is the most useful thing the tool will tell you.
Next year, and the year after
The annual re-verification is the step most likely to get skipped, because by then the build feels finished. It's also the easiest to share out: send the new paper's dataset and the exam to a colleague with the Tag Checker and they can do the whole check without a Claude account, in the time it takes to mark a set. Their corrections file goes into your project and the tool rebuilds. Half an hour a year is what keeps this honest.
Two things to say out loud when you hand it round
The predictions are pattern-reading, not forecasting. The whole study design is examinable every year and every paper is unique. The tool is for prioritising revision when time is short, not for narrowing what gets taught.
The analysis is only as good as the coding. If a colleague disagrees with a tag, they're probably right to check — send them back to the dataset rather than defending the interface.
The analysis is only as good as the coding. If a colleague disagrees with a tag, they're probably right to check — send them back to the dataset rather than defending the interface.