This article explains how the AI scoring works, what it can and can't evaluate, and what to do if something seems off.

The QM Review Assistant Tool uses AI to analyze your course content and score it against the QM rubric. It's designed to give you a fast, detailed first-pass assessment  but there are important things to understand about how it works and where it has limits.

What the tool can evaluate

The tool reads content that is published and visible inside your Canvas course. This includes pages, modules, assignments, quizzes, discussions, and any uploaded files like a syllabus. The more complete and published your course is at the time of evaluation, the more accurate your results will be.

What the tool cannot evaluate

• Content that is unpublished or hidden from students in Canvas
• External content linked from your course (third-party tools, outside websites) that the tool cannot access
• Offline materials or resources not reflected in Canvas
• Subjective instructional quality decisions that require human judgment

If your course has significant content outside of Canvas, your score may not fully reflect the actual quality of your course design.

Scores can vary between runs

Because the tool uses AI, results may vary slightly between evaluations of the same course, especially if no course content has changed. This is a normal characteristic of AI-based analysis. If you see a meaningful difference between two runs on the same course, use the thumbs down feedback in the report and reach out to support with specifics.

Scores should be treated as directional, not definitive. They are most useful for identifying patterns and priorities, not for precise point-by-point benchmarking.

This tool is not a formal QM review

A score from the QM Review Assistant Tool does not constitute an official Quality Matters review or certification. QM-certified courses go through a formal peer review process conducted by trained QM reviewers. This tool is meant to help you prepare for that process not replace it.

If you have questions about what a score means in relation to QM standards, contact Quality Matters directly. For questions about the tool's scoring behavior, contact Noodle support.

If something seems wrong

• Use the Rate these AI results thumbs down in the report to flag inaccurate scoring
• Note the specific standard or sub-criterion where the score seems off
• Reach out to support with the course name, evaluation number, and a brief description of what you observed 

Need help? Contact us at qualitymatters@support.noodle.com