Assessment and Evaluating Student Work

It's ironic, given it's centrality, how little that's sensible and defensible has been said about the relation between grading and assessment.  To my mind, it's a lost opportunity to offer constructive criticism of grading in general as well as a failure on the part of the assessment industry to demonstrate and convey clear thinking and to develop useful tools for teachers.

And so here is part one of working through a relationship between grading and assessing.

My students always want to know "how much does this count for" and so my syllabus always says something like

Exam 1	30%
Problem Sets	20%
Final Essay	40%
Participation	10%

When I grade any particular item -- assignment or paper -- I make use (at least implicitly) of a similar decomposition of the grade.  If it is an essay I may be evaluating the quality of the writing, the use of evidence, the structure of the argument, the use of sources, and so on.  If it is an exam, the questions can usually be separated into a finite number of groups, each "testing" a particular skill or understanding of a particular concept (but see fn 1 below).  Let's imagine a class in which five skills or concepts, J,K,L,M, and N, make up the content.  And let's imagine my graded activities from above can be described this way

Exam 1	25%J	25%K	25%L	25% critical thinking
Problem Sets	20%J	20%K	20%L	20%M	20%N
Final Essay	20% Writing	30% J-N	30% Argument	20% Scholarly conventions
Participation	33%  J-N	33% Staying up with material in course	33% Poise,  verbal skills, etc.

Now let's look at how all of the things I've graded fit together.  In the table below, the rows represent skills or learning outcomes that I want students to demonstrate.  The columns show me the evaluative tools I've used and which of these each one included.

Substance	Exam 1	Problem Sets	Final Essay	Participation
Concept J	+	+	+	+
Concept K	+	+	+	+
Concept L	+	+	+	+
Concept M	-	+	+	+
Concept N	-	+	+	+
Writing	-	-	+	-
Argument	-	-	+	-
Scholarly Conventions	-	-	+	-
Critical Thinking	+	-	+	-
Keeping Up	-	-	-	+
Verbal Skills	-	-	-	+

Next let's suppose that I graded each of these items on an A-F scale and that I've made some attempt to put on paper how I "operationalize" the grades "excellent," "good," "satisfactory," etc. I might, for example, have let students know that I consider an excellent use of concepts in the final essay to be when

Essay employs 3 or more of main concepts from the course in a manner that's appropriate to the subject at hand and that demonstrates a strong understanding of what they mean and how they can be useful.

And finally, let's assume that my program goals include concepts K and M and the school as a whole includes writing and critical thinking as goals.

I can simply take scores on concepts K from all four evaluations and M from the last three and then take the writing score from the final essay and the critical thinking scores from the first exam and the final essay and, oila, I've got my assessment.

fn 1  Two things require mention: 1) not every skill/concept that we expect to be learned is measured on every exam/exercise -- exams are samples; 2) many "items" will depend on more than one skill or concept.  More on these issues later.