Evaluation Methodology

Eight planes, one structured assessment

We assess every course across eight independent planes — from pedagogical design to data protection. The planes run separately and combine into a single traffic light. Every observation is a structured “finding” with a clear source and confidence level.

How to read a finding
id · plane · checkpoint · severity · description · recommendation · location · source · confidence
Severity
Critical — blocks quality
Medium — to improve
OK
Needs review — uncertain finding
Finding source
det rule engine
AI language model
vision image analysis
hybrid rules + AI
Confidence
Deterministic findings have a confidence of 1.0. AI findings go through two-phase verification — in the second round the model checks them against the data. The confidence threshold rises with depth.
Confidence thresholds by depth

An LLM finding receives a 0–1 confidence from two-phase verification. How many findings make it into the report and what gets flagged as “needs review” is governed by the depth:

DepthHide below“Needs review”Confident finding
Overviewall except > 0.75> 0.75
Standard< 0.40.4 – 0.7> 0.7
Detailed< 0.30.3 – 0.7> 0.7
Three depths — cumulative
Overview · free
Free Traffic Light
A per-plane traffic light + the 3–5 most critical findings. Shows whether the course is on fire.
Show a sample audit →
Standard
Full assessment
Per-plane scores and the complete list of findings with description, location and recommendation.
Show a sample audit →
Detailed
Dashboard
Everything in Standard + dependencies, student walkthrough, impact×effort matrix, action plan, re-audit.
Show a sample audit →
The eight evaluation planes
What the tool does not evaluate (boundaries)

These areas are outside the automated audit — the report marks them as “handled individually”:

In-depth content audit of multimediabranched H5P scenarios and long videos are not assessed for content.
Full WCAG accessibility auditthe tool performs only basic checks within planes 4 and 5.
Content quality of SCORM/H5P packagesplane 3 evaluates only the technical deployment and tracking.
Applying fixes in Moodlethe audit is an analysis and report, not repairs; those are a separate paid add-on.

The price is never computed by AI — it is set by a separate deterministic calculator. The report is indicative, not a legally binding assessment.

Reports never name specific AI models — each finding carries only its source (deterministic / AI / vision / hybrid) and a confidence level. All 8 planes are always evaluated (depth controls detail, not plane selection) — at the Free level only deterministic checks run; planes with in-depth AI analysis are fully evaluated in Standard and Detailed, and plane 7 only with a connected Moodle API.