PROGRAM LEARNING OUTCOMES
Rubric for Assessing the Quality of Academic Program Learning Outcomes
Criterion Initial Emerging Developed Highly Developed
Comprehensive
List
The list of outcomes is
problematic: e.g., very incomplete,
overly detailed, inappropriate,
disorganized. It may include only
discipline-specific learning,
ignoring relevant institution-wide
learning. The list may confuse
learning processes (e.g., doing an
internship) with learning outcomes
(e.g., application of theory to real-
world problems).
The list includes reasonable
outcomes but does not specify
expectations for the program
as a whole. Relevant
institution-wide learning
outcomes and/or national
disciplinary standards may be
ignored. Distinctions between
expectations for
undergraduate and graduate
programs may be unclear.
The list is a well-organized set of
reasonable outcomes that focus on
the key knowledge, skills, and
values students learn in the
program. It includes relevant
institution-wide outcomes (e.g.,
communication or critical thinking
skills). Outcomes are appropriate
for the level (undergraduate vs.
graduate); national disciplinary
standards have been considered.
The list is reasonable, appropriate, and
comprehensive, with clear distinctions
between undergraduate and graduate
expectations, if applicable. National
disciplinary standards have been
considered. Faculty have agreed on
explicit criteria for assessing students’
level of mastery of each outcome.
Assessable Outcome statements do not Most of the outcomes indicate Each outcome describes how Outcomes describe how students can
Outcomes identify what students can do to
demonstrate learning. Statements
such as “Students understand
scientific method” do not specify
how understanding can be
demonstrated and assessed.
how students can demonstrate
their learning.
students can demonstrate learning,
e.g., “Graduates can write reports
in APA style” or “Graduates can
make original contributions to
biological knowledge.”
demonstrate their learning. Faculty have
agreed on explicit criteria statements,
such as rubrics, and have identified
examples of student performance at
varying levels for each outcome.
Alignment There is no clear relationship
between the outcomes and the
curriculum that students
experience.
Students appear to be given
reasonable opportunities to
develop the outcomes in the
required curriculum.
The curriculum is designed to
provide opportunities for students
to learn and to develop increasing
sophistication with respect to each
outcome. This design may be
summarized in a curriculum map.
Pedagogy, grading, the curriculum,
relevant student support services, and co-
curriculum are explicitly and intentionally
aligned with each outcome. Curriculum
map indicates increasing levels of
proficiency.
Assessment
Planning
There is no formal plan for
assessing each outcome.
The program relies on short-
term planning, such as
selecting which outcome(s) to
assess in the current year.
The program has a reasonable,
multi-year assessment plan that
identifies when each outcome will
be assessed. The plan may
explicitly include analysis and
implementation of improvements.
The program has a fully-articulated,
sustainable, multi-year assessment plan
that describes when and how each
outcome will be assessed and how
improvements based on findings will be
implemented. The plan is routinely
examined and revised, as needed.
The Student
Experience
Students know little or nothing
about the overall outcomes of the
program. Communication of
outcomes to students, e.g. in
syllabi or catalog, is spotty or
nonexistent.
Students have some
knowledge of program
outcomes. Communication is
occasional and informal, left to
individual faculty or advisors.
Students have a good grasp of
program outcomes. They may use
them to guide their own learning.
Outcomes are included in most
syllabi and are readily available in
the catalog, on the web page, and
elsewhere.
Students are well-acquainted with
program outcomes and may participate in
creation and use of rubrics. They are
skilled at self-assessing in relation to the
outcomes and levels of performance.
Program policy calls for inclusion of
outcomes in all course syllabi, and they
are readily available in other program
documents.
How Visiting Team Members Can Use the Learning Outcomes Rubric
Conclusions should be based on a review of learning outcomes and assessment plans. Although you can make some preliminary judgments
about alignment based on examining the curriculum or a curriculum map, you will have to interview key departmental representatives, such as
department chairs, faculty, and students, to fully evaluate the alignment of the learning environment with the outcomes.
The rubric has five major dimensions:
1. Comprehensive List. The set of program learning outcomes should be a short but comprehensive list of the most important knowledge, skills,
and values students learn in the program, including relevant institution-wide outcomes such as those dealing with communication skills, critical
thinking, or information literacy. Faculty generally should expect higher levels of sophistication for graduate programs than for undergraduate
programs, and they should consider national disciplinary standards when developing and refining their outcomes, if available. There is no strict
rule concerning the optimum number of outcomes, but quality is more important than quantity. Faculty should not confuse learning processes
(e.g., completing an internship) with learning outcomes (what is learned in the internship, such as application of theory to real-world practice).
Questions
. Is the list reasonable, appropriate and well-organized? Are relevant institution-wide outcomes, such as information literacy,
included? Are distinctions between undergraduate and graduate outcomes clear? Have national disciplinary standards been considered when
developing and refining the outcomes? Are explicit criteria – as defined in a rubric, for example – available for each outcome?
2. Assessable Outcomes. Outcome statements should specify what students can do to demonstrate their learning. For example, an outcome
might state that “Graduates of our program can collaborate effectively to reach a common goal” or that “Graduates of our program can design
research studies to test theories and examine issues relevant to our discipline.” These outcomes are assessable because faculty can observe
the quality of collaboration in teams, and they can review the quality of student-created research designs. Criteria for assessing student
products or behaviors usually are specified in rubrics, and the department should develop examples of varying levels of student performance
(i.e., work that does not meet expectations, meets expectations, and exceeds expectations) to illustrate levels. Questions
. Do the outcomes
clarify how students can demonstrate learning? Have the faculty agreed on explicit criteria, such as rubrics, for assessing each outcome? Do
they have examples of work representing different levels of mastery for each outcome?
3. Alignment. Students cannot be held responsible for mastering learning outcomes unless they have participated in a program that
systematically supports their development. The curriculum should be explicitly designed to provide opportunities for students to develop
increasing sophistication with respect to each outcome. This design often is summarized in a curriculum map—a matrix that shows the
relationship between courses in the required curriculum and the program’s learning outcomes. Pedagogy and grading should be aligned with
outcomes to foster and encourage student growth and to provide students helpful feedback on their development. Since learning occurs within
and outside the classroom, relevant student services (e.g., advising and tutoring centers) and co-curriculum (e.g., student clubs and campus
events) should be designed to support the outcomes. Questions
. Is the curriculum explicitly aligned with the program outcomes? Do faculty
select effective pedagogy and use grading to promote learning? Are student support services and the co-curriculum explicitly aligned to
promote student development of the learning outcomes?
4. Assessment Planning. Faculty should develop explicit plans for assessing each outcome. Programs need not assess every outcome every
year, but faculty should have a plan to cycle through the outcomes over a reasonable period of time, such as the period for program review
cycles. Questions.
Does the plan clarify when, how, and how often each outcome will be assessed? Will all outcomes be assessed over a
reasonable period of time? Is the plan sustainable, in terms of human, fiscal, and other resources? Are assessment plans revised, as needed?
5. The Student Experience. At a minimum, students should be aware of the learning outcomes of the program(s) in which they are enrolled;
ideally, they should be included as partners in defining and applying the outcomes and the criteria for levels of sophistication. Thus it is
essential to communicate learning outcomes to students consistently and meaningfully. Questions
: Are the outcomes communicated to
students? Do students understand what the outcomes mean and how they can further their own learning? Do students use the outcomes and
criteria to self-assess? Do they participate in reviews of outcomes, criteria, curriculum design, or related activities?