Sociology SMALCs Reporting Tool on Qualtrics
Each year we report program outcomes to satisfy the requirements of our accrediting organization (SACS) and the State of Florida. For instructors who are teaching designated courses (generally, 4000-level courses), this page describes how to gather and report these data before the end of the semester.
Questions can be directed to Kathi Tillman, Chair of the department, or a colleague who has taught the same course in the recent past.
Here are some things to keep in mind:
- Design a specific assignment (paper, exam question, other exercise) to measure the outcome(s) being assessed in your class.
- This assessment should be designed to be both reliable and valid in your judgment.
- The assessment should be required of all students and count in their grade (like an essay on the final exam).
- The assessment should be given late in the semester to evaluate learning, not diagnose limitations.
- If a paper is used, the grade should reflect the outcome of interest rather than issues like grammar.
- The assessment should be something all students take seriously (not an optional exam or optional exercise).
- It will be helpful to distinguish between missing values (e.g., student is absent) from a true zero as an indicator of student knowledge or skill.
- The student learning goals should be taken seriously as a course objective (what you want the students to know when the course is over) and the assessment exercise should also be taken seriously to document that they have learned it in a methodologically rigorous way.
IF YOU ARE TEACHING A (3 CREDIT-HOUR) 4000-LEVEL COURSE THIS SEMESTER BESIDES THEORY (SYA4010), METHODS (SYA4300), OR STATISTICS (SYA4400).
For your 4000-level course (that is NOT theory, methods, or stats), you are responsible for gathering data on one outcome: “TLO3 Demonstrate knowledge of inequality.”
Defined Outcome: Upon completion of the course of instruction, the student will be able to demonstrate knowledge of the primary axes of social inequality, and their dynamics, and effects, in contemporary society.
Assessment and Evaluation Process: This will result in 66% of the students for the 2014-15 academic year enrolled in all sections of 4000-level sociology courses (other than theory, methods and statistics) producing a written report on social inequality on which they score 80% or better.
Method: Course Embedded Assignment (often in tandem with exam question bank) and Written Report or Essay.
(1) Conduct your course assessment on demonstrating knowledge of inequality.
(2) Keep a copy of that course assessment in your electronic records for this course.
(3) Put the results in an Excel spreadsheet that includes the maximum possible points a student could make at the top of the spreadsheet and all of the individual student scores on the assessment below the maximum. E.g., if my assessment was worth a total of 10 points in a class of 12 people, my spreadsheet might look like this
Max = 10
Scores = 9, 3, 8, 9, 8, 10, 7, 4, 9, 7, 8, 10
[Note, it is a good idea to sort these scores.]
(4) Count the total number of students who took the assessment. In this case, 12.
(5) Count the total number of students who scored at least 80% on the assessment. In this case, everyone with an 8 or higher met the target. That equals 8 students in this example.
(6) Calculate the percent who got at least 80% correct. In this case, 8/12 = 67%.
(7) Upload your spreadsheet and report your calculations on Qualtrics.
Submit your assessment numbers for your 4000-level class.
[1] Total possible points on the assessment
[2] Number of students who took the assessment
[3] Number of students who got at least 80% of the total possible pts
[4] Percent of students who got at least 80% of the total possible pts
Record whether this is this an online or face-to-face class.
Upload your spreadsheet containing the total possible points, and the scores for each of the students who completed the assessment. Do NOT include student names or ids.
IF YOU ARE TEACHING “SOCIOLOGICAL THEORY” (SYA4010) THIS SEMESTER.
You are responsible for gathering data on two outcomes, “TLO4 Define and apply sociological concepts” and “TLO7 Compare sociological theories.”
TLO4 Define and apply sociological concepts
Define Outcome: Upon completion of the course of instruction, the student will be able to define and apply core sociological concepts to an understanding of social life.
Assessment and Evaluation Process: This will result in 66% of students enrolled in SYA 4010: Sociological Theory during academic year 2014-15 earning a score of 80% or higher on course embedded assignments or a written report or essay.
Method(s): Course Embedded Assignment (often in tandem with exam question bank) and Written Report or Essay.
TLO 7 Compare sociological theories
Define Outcome: Upon completion of the course of instruction, the student will be able to accurately compare and contrast two sociological theories (including their proponents) and their implications for explaining social phenomena, situations, or events.
Assessment and Evaluation Process: This will result in 66% of students for the 2014-15 academic year in all sections of SYA 4010: Sociological Theory producing a written report on which they score 80% or higher.
Method(s): Written Report or Essay.
(1) Conduct your course assessments on TLO4 (soc concepts) and TLO7 (compare theories).
(2) Keep a copy of that course assessment in your electronic records for this course.
(3) Put the results in an Excel spreadsheet that includes the maximum possible points a student could make at the top of the spreadsheet and all of the individual student scores on the assessment below the maximum. E.g., if my assessment was worth a total of 10 points in a class of 12 people, my spreadsheet might look like this
Max = 10
Scores = 9, 3, 8, 9, 8, 10, 7, 4, 9, 7, 8, 10
[Note, it is a good idea to sort these scores.]
(4) Count the total number of students who took the assessment. In this case, 12.
(5) Count the total number of students who scored at least 80% on the assessment. In this case, everyone with an 8 or higher met the target. That equals 8 students in this example.
(6) Calculate the percent who got at least 80% correct. In this case, 8/12 = 67%.
(7) Upload your spreadsheet and report your calculations on Qualtrics.
Submit your assessment numbers for “TLO4 Define and apply sociological concepts.”
[1] Total possible points on the assessment
[2] Number of students who took the assessment
[3] Number of students who got at least 80% of the total possible pts
[4] Percent of students who got at least 80% of the total possible pts
Submit your assessment numbers for “TLO7 Compare sociological theories.”
[1] Total possible points on the assessment
[2] Number of students who took the assessment
[3] Number of students who got at least 80% of the total possible pts
[4] Percent of students who got at least 80% of the total possible pts
Record whether this is an online or face-to-face class.
Upload your spreadsheet containing the total possible points, and the scores for each of the students who completed the assessment. Do NOT include student names or IDs.
IF YOU ARE TEACHING “METHODS OF SOCIAL RESEARCH” (SYA4300) THIS SEMESTER.
You are responsible for gathering data on two outcomes, “TLO2 Conduct research and report results” and “TLO6 Evaluate research designs.”
TLO2: Conduct research and report results
Define Outcome: Upon completion of the course of instruction, the student will construct a research question, analyze and interpret data appropriate to answering the question, and write results in jargon-free English.
Assessment and Evaluation Process: This will result in 66% of students for the 2014-15 academic year in all sections of SYA 4300: Methods of Social Research producing a written report for a research project on which they score 80% or better.
Method(s): Project Evaluation and Written Report or Essay.
TLO6: Evaluate research designs
Define Outcome: Upon completion of the course of instruction, the student will be able to identify the advantages and disadvantages of alternative research designs for conducting sociological research.
Assessment and Evaluation Process: This will result in 66% of the students for the academic year 2014-15 enrolled in all sections of SYA 4300: Methods of Social Research scoring 80% or higher on course embedded assignments aimed at testing this knowledge.
Method(s): Course Embedded Assignment (often in tandem with exam question bank).
(1) Conduct your course assessments on TLO2 (conduct research) and TLO6 (evaluate research designs).
(2) Keep a copy of that course assessment in your electronic records for this course.
(3) Put the results in an Excel spreadsheet that includes the maximum possible points a student could make at the top of the spreadsheet and all of the individual student scores on the assessment below the maximum. E.g., if my assessment was worth a total of 10 points in a class of 12 people, my spreadsheet might look like this
Max = 10
Scores = 9, 3, 8, 9, 8, 10, 7, 4, 9, 7, 8, 10
[Note, it is a good idea to sort these scores.]
(4) Count the total number of students who took the assessment. In this case, 12.
(5) Count the total number of students who scored at least 80% on the assessment. In this case, everyone with an 8 or higher met the target. That equals 8 students in this example.
(6) Calculate the percent who got at least 80% correct. In this case, 8/12 = 67%.
(7) Upload your spreadsheet and report your calculations on Qualtrics.
Submit your assessment numbers for “TLO2 Conduct research and report results.”
[1] Total possible points on the assessment
[2] Number of students who took the assessment
[3] Number of students who got at least 80% of the total possible pts
[4] Percent of students who got at least 80% of the total possible pts
Submit your assessment numbers for “TLO6 Evaluate research designs.”
[1] Total possible points on the assessment
[2] Number of students who took the assessment
[3] Number of students who got at least 80% of the total possible pts
[4] Percent of students who got at least 80% of the total possible pts
Record whether this is an online or face-to-face class.
Upload your spreadsheet containing the total possible points, and the scores for each of the students who completed the assessment. Do NOT include student names or IDs.
IF YOU ARE TEACHING “SOCIAL STATISTICS” (SYA4400) THIS SEMESTER.
You are responsible for gathering data on two outcomes, “TLO1 Perform statistical analysis” and “TLO5 Interpret sociological evidence.”
TLO1: Perform statistical analyses
Define Outcome: Upon completion of the course of instruction, the student will be able to calculate and interpret basic statistics such as percentages, chi-square, cross-tabulations, and bivariate regression.
Assessment and Evaluation Process: This will result in 66% of the students for the 2014-15 academic year in all sections of SYA 4400: Social Statistics, scoring 80% or better on problem-solving exercises.
Method(s): Problem-Solving Exercise.
TLO5: Interpret sociological evidence
Define Outcome: Upon completion of the course of instruction, the student will be able to interpret and weigh evidence as to whether asserted conclusions about social conditions or processes are warranted.
Assessment and Evaluation Process: This will result in 66% of the students in the 2014-15 academic year in all sections of SYA 4400: Social Statistics scoring 80% or better on course embedded assignments that test for these sociological skills.
Method(s): Course Embedded Assignment (often in tandem with exam question bank).
(1) Conduct your course assessments on TLO1 (do stats) and TLO5 (interpret evidence). (2) Keep a copy of that course assessment in your electronic records for this course.
(3) Put the results in an Excel spreadsheet that includes the maximum possible points a student could make at the top of the spreadsheet and all of the individual student scores on the assessment below the maximum. E.g., if my assessment was worth a total of 10 points in a class of 12 people, my spreadsheet might look like this
Max = 10
Scores = 9, 3, 8, 9, 8, 10, 7, 4, 9, 7, 8, 10
[Note, it is a good idea to sort these scores.]
(4) Count the total number of students who took the assessment. In this case, 12.
(5) Count the total number of students who scored at least 80% on the assessment. In this case, everyone with an 8 or higher met the target. That equals 8 students in this example.
(6) Calculate the percent who got at least 80% correct. In this case, 8/12 = 67%.
(7) Upload your spreadsheet and report your calculations on Qualtrics.
Submit your assessment numbers for “TLO1 Perform statistical analyses.”
[1] Total possible points on the assessment
[2] Number of students who took the assessment
[3] Number of students who got at least 80% of the total possible pts
[4] Percent of students who got at least 80% of the total possible pts
Submit your assessment numbers for “TLO5 Interpretation sociological evidence.”
[1] Total possible points on the assessment
[2] Number of students who took the assessment
[3] Number of students who got at least 80% of the total possible pts
[4] Percent of students who got at least 80% of the total possible pts
Record whether this is an online or face-to-face class.
Upload your spreadsheet containing the total possible points, and the scores for each of the students who completed the assessment. Do NOT include student names or IDs.
IF YOU ARE TEACHING THE GRADUATE COURSE “INTRODUCTION TO RESEARCH METHODS” (SYA5305) THIS SEMESTER.
You are responsible for gathering two outcomes, described below. This refers only to the Applied Masters Students in your class, not all the students. Contact Jamie if you need help identifying these students.
Outcome 54413. Data analysis skills
Define Outcome: Upon completion of the course of instruction, the student will be able to demonstrate proficiency in data analysis using multiple regression and other techniques.
Assessment and Evaluation Process: This will result in 75% of the students scoring 80% or better as determined by an instructor constructed exam in master’s statistics course, SYA 5305, a milestone course.
Method(s): Instructor Constructed Exam.
Outcome 54414. Professional preparation
Define Outcome: Upon completion of the course of instruction, the student will be able to demonstrate competence in conducting empirical research of the type that will be required in professional employment settings.
Assessment and Evaluation Process: This will result in 80% of the students scoring 85% or better as determined by a course project required in SYA 5305 (Research Methods), a milestone course.
Method(s): Project Evaluation.
(1) Conduct your course assessments on Outcome 54413 (data analysis skills) and Outcome 54414 (professional prep).
(2) Keep a copy of the course assessments in your electronic records for this course.
(3) Count the total number of MASR students in SYA 5305.
(4) Count the number of MASR students who scored at least 80% on the data analysis test, and also the number of MASR students who scored at least 85% on the course project.
(5) Calculate the percent who got at least 80% correct on the data analysis exercise and the percent who got at least 85% on the course project.
(6) Report your calculations on Qualtrics.
Submit your assessment numbers for “Outcome 54413 Data analysis skills.”
[1] Number of MASR students
[2] Number of students who got at least 80% of the total possible pts
[3] Percent of students who got at least 80% of the total possible pts
Submit your assessment numbers for “Outcome 54414 Professional preparation.”
[1] Number of MASR students
[2] Number of students who got at least 85% of the total possible pts
[3] Percent of students who got at least 85% of the total possible pts