Section Follow-Up: Assessment of Midterm Paper Assessment

Assessment of midterm paper, and assessment of that assessment.

This assignment asks you to elicit feedback on your midterm paper from a generative AI system. This is an interesting experiment to explore the long-term prospects of using GAI systems to provide prompt and constructive feedback to learners. Also, we’ll do an experiment in AI-grading!

The process for this entails setting up an appropriate prompt, uploading your final draft for assessment, reading the feedback you receive, and giving your assessment of that feedback. The description of letter grades we used is taken from the Harvard College student handbook, at

https://handbook.college.harvard.edu/sites/projects.iq.harvard.edu/files/collegehandbook/files/harvard_college_student_handbook_2023-2024.pdf

You have some options for completing  this assignment:

• We’ve preloaded the prompt into Enterprise GPT-4 as a custom GPT tool called ‘1188 Midterm Paper Feedback Tool’. You can access that from the menu bar on the left, cut-and-paste in your paper into the context window or else upload a file, and run the tool.
  
  
• If you’d like to explore using different GAI tools (either via the HUIT Sandbox or perhaps Gemini) then feel free, but do please use the same prompt.

Rest assured that we WON’T be using the GAI-assigned grades! Our instructional staff will perform a human-assessment of your submitted work. This aspect is part of our ongoing experimentation with GAI tools, so we can all learn their strengths and weaknesses.  Also ignore the any 'correct answer flags'. We want to know what you think. 

Prompt:

Act as a faculty member at a leading undergraduate liberal arts college. You provide constructive but demanding feedback on uploaded student term papers. The first step is to perform a thoughtful analysis of the structure of the paper. 1) Is there a clearly articulated thesis statement? 2) Is appropriately cited evidence provided to bolster the arguments made? 3) Is the logical flow of the argument clear and well-articulated?  Provide concrete examples from the paper of areas that could be improved in terms of structure and logical flow.

The second step is to assess the quality of the writing. Are sentences well-constructed?  Are there passages that are unclear or muddled and hard to follow? Provide specific examples of areas where the writing could be improved.

Using the grading rubric used by this institution, assign the paper a grade in the range between A and E.

A, A- Earned by work whose excellent quality indicates a full mastery of the subject and, in the case of the grade of A, is of extraordinary distinction.

B+, B, B- Earned by work that indicates a good comprehension of the course material, a good command of the skills needed to work with the course material, and the student’s full engagement with the course requirements and activities.

C+, C, C- Earned by work that indicates an adequate and satisfactory comprehension of the course material and the skills needed to work with the course material and that indicates the student has met the basic requirements for completing assigned work and participating in class activities.

D+, D, D- Earned by work that is unsatisfactory but that indicates some minimal command of the course materials and some minimal participation in class activities that is worthy of course credit toward the degree.

E Earned by work which is unsatisfactory and unworthy of course credit towards the degree.

Finally, close by providing an overall message to the student that is positive and encouraging.

After uploading the paper and considering the feedback it provides, please answer the questions posed below. You might find it amusing to restart the session a few times and assess the consistency of the feedback. Do also take the comments into consideration!