Student products — AI competency exam
LLMCE — the AI competency exam
Traditional exams test a fixed question paper. LLMCE tests the person — an LLM-powered, conversational examiner that grounds every assessment in what this specific learner actually studied and the target competencies a qualification is accredited to certify.
A note on naming: LLMCE is the market-facing packaging of Woolf existing Exam System and its RPLFC AI exam capabilities — a product wrapper around shipping technology, not a separate internal system.
Dual grounding: the whole point
Every exam is built from two inputs reconciled against each other — what the learner consumed, and what must be proven — mapping cleanly onto Knowledge, Skills, and Competence.
Questions drawn only from the extracted sections of the specific learning resource the student engaged with.
Knowledge-coverage configured by staff for that resource.
Topics extracted from all learning resources associated with the course.
EQF taxonomy per question group across Knowledge, Skills, and Competence, fed by the course intended learning outcomes (ILOs).
Questions generated from the student own evidence — CV, portfolio, certificates.
A target-degree payload — degree outcomes, course list, ILOs, ECTS, and EQF level — that questions must align to.
In each case questions are validated to be answerable from the learner material, while being interpreted from — never copied verbatim out of — the target outcomes.
Three exam kinds
One exam engine, three grounded modes — from a single resource quiz to a full recognition-of-prior-learning workflow.
Summative assignment
ImplementedA college-managed, course-level summative assessment grounded in the topics extracted from every learning resource on the course.
PRL exam
ImplementedA resource-grounded quiz — for example over a peer-reviewed publication — that proves a student engaged with a specific piece of learning.
RPLFC exam
ExperimentalPart of the Recognition-of-Prior-Learning-for-Credit workflow: generated from a student evidence and aligned to a target degree ILOs.
Offline
Question generation
A dedicated ML service builds the question bank. For a course it ingests metadata and ILOs to produce assessable topics. For a resource (PRL) it generates questions, graduated hints, and a Bloom level — and validates that questions, hints, and answers come only from the provided content, retrying with feedback until they do. For RPLFC it generates graduated-difficulty questions from the evidence and aligns them to the target degree ILOs, with an evaluation-and-regeneration loop.
Live
The exam coach (multi-agent)
The live exam runs on a Vertex AI reasoning engine as a multi-agent exam coach.
Conducts the conversational exam with the student in real time.
Decides pass or fail for each question, with explicit reasoning.
Offers progressive, penalty-bearing hints across levels 1 to 5.
Grading and competency mapping
Every score is explainable — a rubric, a hint penalty, and a best-session rule produce a defensible competency result.
Per question: the evaluator uses a coverage rubric — a pass requires roughly 70% of the weighted criteria to be covered, with no essential criterion entirely missing.
Hint penalty: each hint reduces the achievable score, and a level-5 hint drives the question contribution toward zero, so assistance is available but never free.
Per session: question scores sum to a session total capped at 100; the student best session — not their last — sets the exam score, judged against a configurable passing threshold.
Bounds: exams enforce per-question answer limits, per-exam attempt limits, and a timed auto-submit, so the assessment is finite and fair.
Downstream: an RPLFC exam result carries an examPassed flag into the credit-exemption workflow, where the academic team makes the final call.
Integrity and fairness
Grounding, alignment, and human review keep the exam rigorous, while penalties and answer secrecy keep it hard to game.
Grounding
For PRL, questions, hints, and answers must derive only from the provided content — validated automatically, with retries.
Alignment
For RPLFC, questions must match the target degree ILOs; difficulty is graduated and evaluated in a loop.
Anti-gaming hints
Hints cost score, and stronger hints cost more, so the exam cannot be gamed for free assistance.
Answer secrecy
Correct answers and hints are stripped from the client session state.
Human in the loop
For credit-bearing decisions, an academic reviewer approves the outcome.
Why it matters
The bridge between doing the work and meeting the standard
For learners
The exam meets learners where they are — grounded in what they studied or can prove — while still holding them to the accredited standard.
For colleges
Competency assessment that scales without writing a new exam paper for every cohort, with a rubric and audit trail behind every score.
For investors
LLMCE turns Woolf unique asset — fine-grained records of what each learner actually consumed — into rigorous, outcome-aligned assessment only a platform with this evidence layer can credibly offer.