Consensus Metric Materials

This repository contains data and notebooks for exploring and evaluating a consensus metric on course-evaluation data.

Repository Contents

• example_calculate_consenus.ipynb: Example workflow for calculating and inspecting consensus-style metrics on course data.
• simulation_study.ipynb: Simulation workflow used to study metric behavior under different settings.
• ./data/student_responses.csv: Row-level (student-level) survey responses.
• ./data/course_data.csv: Aggregated (course-section-level) dataset derived from responses.

Quick Start

1. Create a Python environment

Windows PowerShell:

python -m venv .venv
.\.venv\Scripts\Activate.ps1

2. Install dependencies

pip install jupyter numpy pandas seaborn matplotlib

3. Open and run notebooks

Run either notebook in VS Code or Jupyter Lab/Notebook:

jupyter notebook

Then open:

• example_calculate_consenus.ipynb
• simulation_study.ipynb

Run cells from top to bottom.

Notebook Notes

example_calculate_consenus.ipynb

This notebook includes Google Colab-specific setup lines (for example from google.colab import drive, drive.mount(...), and %cd ...).

If you run locally, skip or comment those Colab-only cells and keep your working directory at the repository root so relative CSV paths resolve correctly.

simulation_study.ipynb

This notebook runs simulations and may generate result CSV files such as:

• result_n30_7.csv
• result_n50_k3.csv
• result_n70_k6.csv

These outputs are created in the current working directory unless paths are changed.

Data Dictionary

student_responses.csv (student-level)

Each row represents one student's record for a specific course section in a term.

Important column groups:

• Identifiers and context:
  • YEAR_SEMESTER, Course_CRNadjusted, InstructorID_PIDMadjusted, StudentID_PIDMadjusted
  • Dept, Dept_subject, College_new, cname
• Student/course metadata:
  • GENDER, USER_LEVEL, COURSE_LEVEL, TEACHING_METHOD, ENROLLMENT
  • Q1_CourseType_CourseType, Q2_GradethatIexpectinthiscourse_GradethatIexpectinthiscourse
• Per-question rating responses:
  • Instructorsabilitytocom, Instructorsencouragemen, Instructorspromptnessan, Instructorsavailability
  • Instructorspromptnessin, Instructorsfairnessandc, Instructorsknowledgeoft, Overallteachingeffectiv
  • Additional course-evaluation items such as difficulty/materials/value questions (Q3, Q4, Q5 columns)
• Demographic adjustments:
  • race_adjusted, gender_adjusted

Notes:

• -1 appears in some survey fields and is typically used as a missing or not-applicable code in this dataset.

course_data.csv (aggregated course-section level)

Each row represents one course section (per instructor/term context) with aggregated statistics.

Important column groups:

• Keys and context:
  • course_id, section_id, semester, course_level, dept, dept_subject, college, cname
• Aggregated question metrics:
  • Questions with no modifiers (e.g., Instructorsabilitytocom) show the consensus
  • For each core teaching item, columns include a score plus _average and _median
  • Example pattern: Instructorsabilitytocom, Instructorsabilitytocom_average, Instructorsabilitytocom_median
  • Similar patterns exist for the other instructor/overall teaching fields
• Normalized metrics:
  • Columns ending in _norm_average (normalized averages)
• Importance/category labels:
  • Columns ending in _importance_level
• Enrollment/response summary:
  • number_students, enrollment, response_rate
• Additional course attributes:
  • course_type_majority, expected_grade_majority,
  • difficulty_median, difficulty_average,
  • overall_educational_value_median, overall_educational_value_average,
  • course_materials_quality_median, course_materials_quality_average

Reproducibility Tips

• Keep the notebook working directory as this repository root.
• Run all cells in order after kernel restart.
• If you adapt file locations, update pd.read_csv(...) and to_csv(...) paths accordingly.