Definitions

Assessment plan

the outcomes a program plans to assess and the methods it will use to assess them in a given academic year. Other supporting information should be provided as well

Assessment results

include data from completed outcomes assessments and thoughtful analysis of their context and meaning, usually from the prior academic year.

Outcomes

Outcomes are specific, realistic, and measurable statements of what will happen as a result of the program or unit’s activities.

Assessment Methods

Definitions and Expectations:

Assessment methods are the specific tools used to discern whether an outcome was attained. Common methods of assessment for academic programs include exams and rubrics.

Methods should align with outcomes. For example, an outcome centered on students’ oral communication skills cannot be assessed through a writing exercise.

Targets are a program or unit’s definition of success for a particular outcome and method. Targets are directly related to the method being used to assess the outcome and are usually result-oriented. For example, an academic program that is assessing an outcome through an exam might set a target of 75% or more students scoring a C or better on the exam.

Direct vs. indirect methods

After faculty members agree on program student learning outcomes, they choose a data collection method that will help them assess the outcome and provide information to aid in decision making. Data collection methods for assessment purposes typically fall into two categories: direct and indirect.

SACS identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of seeking improvement based on analysis of the results in student learning outcomes for each of its educational programs.

Direct methods

provide for the direct examination or observation of student knowledge or skills against measurable learning outcomes. Faculty conduct direct assessments of student learning throughout a course using such techniques as exams, quizzes, demonstrations, and both oral and written reports. These techniques provide feedback on what students know and/or can do and provide strong evidence of student learning. This type of assessment most often takes place in courses, but also can occur in portfolios, capstone projects, internships, or standardized exams.

Indirect methods

are not as effective in assessing learning as direct methods because assumptions must be made about how to interpret an indirect report. If students report they have attained a specific learning outcome, how can that report be validated? An indirect assessment is useful in that it can be used to identify certain implicit qualities of student learning, such as values, perceptions, and attitudes, from a variety of perspectives. However, in the absence of direct evidence, assumptions must be made about how well student perceptions match the reality of actual achievement of student learning. The most common form of indirect assessment is surveys, though interviews and focus groups are sometimes used as well.

Both are important, but indirect evidence by itself is insufficient for assessing learning. Direct evidence is required. Ideally, a program collects both types. Direct evidence, by itself, can reveal what students have learned and to what degree, but it does not provide information as to why the student learned or did not learn. The why is valuable because it can guide faculty members in how to interpret results and make improvements. Programs can collect both direct and indirect evidence of student learning to gain a better picture of their students.

Student learning outcomes:

Student learning outcomes (SLOs) are explicit statements of skills, abilities, and competencies a student is expected to acquire and demonstrate as a result of completing a course, a coordinated set of courses, or other activities, including co-curricular experiences.

Planned Strategies to Achieve Outcome

Planned Strategies to Achieve Outcome are specific actions that increase the likelihood that students achieve an outcome. It is a required field in Nuventive/TracDat and must be completed for each outcome.

For example, if an academic program seeks to foster students’ oral communication skills, the report could describe the learning activities and experiences that faculty will provide to support attainment of the learning outcome.

Student learning outcomes vs. program/unit outcomes

Student learning outcomes are distinguished from general program outcomes by their emphasis on the skills, abilities, and competencies students develop and demonstrate in a course, a coordinated set of courses, or other activities that are part of a formal academic experience. This may include activities such as internships, research experiences, or practica if they are required elements of the program.

Program outcomes include broader goals or objectives of the program or unit and relate to the program or unit’s impact on students or stakeholders.

They are often more operational than learning outcomes. Program outcomes are measured by establishing specific actions that will provide data that inform the faculty or program/unit staff of the progress they are making toward achieving the goal.

Embedded Assessments

Embedded assessments are direct assessment methods conducted when faculty members evaluate students’ work in a course. They may include pre- course assessments, papers, projects, experiences, exams, or evaluations. An embedded assessment may be a single exam or assignment or a set of exams or assignments. In all cases, however, the exam or assignments are aligned with specific program learning outcomes.

Some embedded assessments that are course-related can be more comprehensive than single assignments. For example, an assessment can be conducted of the body of work in a capstone course or in student portfolios.

Prohibition on Course Grades as Assessment Methods

When program-level student learning outcomes (SLOs) are assessed in a course, it is tempting to conclude that the course grade is an acceptable measure of the program SLO. However, course grades often are, appropriately, a composite representation of multiple direct assessments and grading criteria. Because the calculation of a course grade often involves a combination of various information sources that include data unrelated to the SLO (e.g., attendance, participation, assessments of other objectives), the course grade does not directly represent the student’s achievement of the SLO. This weakens the validity of the course grade as a measure of the SLO because appropriate evidence to support the interpretation of the course grade for its intended use as an SLO achievement measure cannot be adequately established. Therefore, overall course grades cannot be used as SLO measures. There are no exceptions to this rule.

Results

Results are narrative descriptions of the results of an assessment, including how the data were collected and analyzed. In most cases, the description is related to assessment methods identified in the planning phase of the assessment process. This information is being provided to assist with assessment planning.

Guidance and Training

Academic Affairs maintains two self-enrolling Canvas sites to support degree program assessment.

The Assessment Center is a resource site with guides, templates, video recordings of workshops and training sessions, and a link to the Nuventive/TracDat assessment management system.
Foundations of Student Learning Outcomes Assessment is a professional development course designed to equip UT personnel with essential knowledge and skills for assessing and improving student learning.

Results in Nuventive/TracDat are submitted in two parts.

Description of Results contains a summary of the results obtained by the assessment method. The program must also indicate whether the predefined assessment method Target was met. Whenever possible, the description should include:

The total number of students assessed.
A breakdown of their performance on the assessment – e.g., the percent that earned a grade of A, B, C, or less on the assignment used in the assessment.
The criteria used in the assessment – e.g., the rubric in the case of a written assignment.

Analysis of Results describes how results were analyzed and documents the program’s conclusions about them. Questions to consider in your analysis include, but are not limited to:

What do the assessment results say about students' mastery of program student learning outcomes?
What do the results say about students' progress in meeting non-learning program outcomes?
Are there areas where your students are outstanding? Do the data suggest weakness in any particular skills?
Are there any changes that need to be made to the program or course? What additional information is needed?