Issues Related To Scoring Of Essay Type Test Items On Subject-Verb

Charles Champlin (2006), a journalist for Time and Life magazines, describes his experience of taking essay tests as a student at Harvard:

“The worst were the essay questions (which seemed only distantly related to whatever you’d read or heard in lectures).  They made a statement and then simply said, ‘Discuss.’  O terrifying word, ‘Discuss.’  Nothing so simple as tossing in a few facts retained from all-night cramming.  It was meaning that was sought – which was, as I’d already begun to appreciate, the way it should be.  But it was a strained step up from the exams I’d known before, when memory, regurgitated, would get you around almost any corner.”

Champlin’s reminiscence reveals some of the strengths and dangers associated with essay questions.  They are a wonderful way to test higher-level learning, but they require careful construction to maximize their assessment effectiveness.

I.  Strengths Associated with Essay Examinations

Among the strengths of essay examinations, faculty who use them find they are a valuable means to measure higher-order learning and a wonderful way, when scored properly, to further student learning.  Given these strengths, essay tests require careful preparation and scoring.

1. Essay Questions Test Higher-Level Learning Objectives

Unlike objective test items that are ideally suited for testing students’ broad knowledge of course content in a relatively short amount of time, essay questions are best suited for testing higher-level learning.  By nature, they require longer time for students to think, organize and compose their answers.

In the table below, appropriate testing strategies are associated with Bloom’s hierarchy of learning. The action verbs under each domain illustrate the kinds of activities that a test item might assess.  Use the verbs when constructing your essay questions so that students know what you expect as they write.  While essay questions can assess all the cognitive domains, most educators suggest that due to the time required to answer them, essay questions should not be used if the same material can be assessed through a multiple-choice or objective item.  Reserve your use of essay questions for testing higher-level learning that requires students to synthesize or evaluate information.

2. Essay Questions When Scored Properly Can Further Learning

Teachers score essay exams by either the holistic approach or the analytic approach.

Holistic Scoring
The holistic approach involves the teacher reading all the responses to a given essay question and assigning a grade based on the overall quality of the response. Some teachers use a holistic approach by ranking students’ answers into groups of best answers, average answers and poor answers and subdividing the groups to assign grades.

Holistic scoring works best for essay questions that are open-ended and can produce a variety of acceptable answers.

Analytic Scoring
Analytic scoring involves reading the essays for the essential parts of an ideal answer.  In this case, you will need to make a list of the major elements that students should include in an answer.  You will grade the essays based on how well students’ answers match the components of the model answer.

Whichever method, holistic or analytic, that you use to score the exam, you should write comments on the students’ papers to enhance their learning.  Your comments will help students write better essays for future classes and reinforce what students know and need to learn.  Your comments are also a good reminder for yourself if students come to you with questions about their grades.

II.  Dangers to Consider When Giving and Grading Essay Examinations

1.  Establish limits within the essay question

The example of Charles Champlin’s experience at Harvard where his teachers gave a statement and then simply said, ‘Discuss,’ shows a danger in using essay questions.  Instructors should build limits into questions in order to save needless writing due to vague questions:  “With some essay questions, students can feel like they have an infinite supply of lead to write a response on an indefinite number of pages about whatever they feel happy to write about. This can happen when the essay question is vague or open to numerous interpretations. Remember that effective essay questions provide students with an indication of the types of thinking and content to use in responding to the essay question” (Reiner, 2002).

Another good way to prevent students from spending excessive time on essays is to give them testing instructions on how long they should spend on test items.  McKeachie (2002) gives the following advice: “As a rule of thumb I allow about 1 minute per item for multiple-choice or fill-in-the-blank items, 2 minutes per short-answer question requiring more than a sentence answer, 10 to 15 minutes for a limited essay question, and half-hour to an hour for a broader question requiring more than a page or two to answer.”

2. Remember that essays require more time to score

While essay exams are quicker to prepare than multiple-choice exams, essay exams take much longer to score.  You should plan sufficient time for scoring the essays to prevent finding yourself crunched to report final grades.

3. Avoid scoring prejudices

Essay exams are subject to scoring prejudices.  Reading all of an individual’s essays at the same time can cause either a positive or a negative bias on the part of the reader.  If a student’s first essay is strong, the examiner might read the student’s remaining essays with a predisposition that they are also going to be strong.  The reverse is also true.  To prevent this scoring prejudice, educators suggest reading all the answers to a single essay question at one time.

Resources:

Champlin, C. (2006). A life in writing: the story of an American journalist. Syracuse: Syracuse University.

McKeachie, W. (2002). McKeachie’s teaching tips (11th. ed.) New York: Houghton Mifflin.

Reiner, C., Bothell, T., Sudweeks, R., & Wood, B. (2002). Preparing effective essay questions. (http://testing.byu.edu/info/handbooks/WritingEffectiveEssayQuestions.pdf).

Constructing Essay Exams

What happens: Learner

  • Hears and reads instructions
  • Interprets the question
  • Recalls relevant information
  • Prepares a response according to the verbal directive,
    either mentally or written, either outlined or "mapped",
  • Writes response
  • Reviews and edits if time permits

Essay tests can evaluate more complex cognitive or thinking skills
assuming that rote memory and recall tasks are assessed more appropriately through objectives tests as true-false and multiple choice questions. These cognitive challenges are reflected in the verbs of the questions themselves, from simple to complex (c.f. lists of verbs in objects...)

  1. Knowledge: recall, define, arrange, list, label, identify, match, reproduce
  2. Comprehension: describe, explain, recognize, restate, review, translate, classify; give examples; (re)state in own words
  3. Application: apply, illustrate, interpret, operate, solve, predict, utilize
  4. Analysis: analyze, compare, contrast, distinguish, examine, experiment, diagram; outline
  5. Synthesis: design, develop, formulate, propose, construct, create, reorganize, integrate, model, incorporate, plan
  6. Evaluation: evaluate, argue, assess, compare, contrast, conclude, defend, judge, support, interpret, justify

(for a complete listing of verbs in these categories, see Essay terms and directives)

Advantages:

  • Require students to demonstrate critical thinking
    in organizing and producing an answer beyond rote recall and memory
  • Empower students to demonstrate their knowledge
    within broad limits beyond the restraint of objective tests (true false, multiple choice)
  • Allows learners to demonstrate originality and creativity
  • Reduces preparation time in developing,
    as well as distributing, a test, especially for small number of students
  • Presents more possibilities for diagnosis

Disadvantages:

  • Grading is often subjective and not consistent, colored by
    preconceptions of student, prior performance, time of day, neatness and handwriting, spelling and grammar, and where the actual test falls in
  • Can be a limited sampling of content
  • Good writing requires time to think,
    organize, write and revise
  • Time consuming to correct
  • Advantageous for students with good writing and verbal skills
    as opposed to those who have alternative learning styles (visual and kinesthetic)
  • Essay questions are not always properly developed
    to assess higher thinking skills (often only test for recall and style)
  • Advantageous for students who are quick,
    as opposed to those who take time to develop an argument or may suffer from writers block

Mechanics:

  • Clearly state questions
    not only to make essay tests easier for students to answer,
    but also to make the responses easier to evaluate
  • Include a relatively larger number of questions
    requiring shorter answers in order to cover more content
  • Guard against having too many test items
    for the time allowed
  • Indicate an appropriate response length
    for each question
  • Set time limits if necessary
  • Note graded weights to questions

Ideal test items:

  • Integrate course objectives into the essay items
  • Specify and define what mental process you want the students to perform
    (e.g., analyze, synthesize, compare, contrast, etc.).
    Does not assume learner is practiced with the process
  • Start questions with an active verb
    such as "compare", "contrast", "explain why";
    Offer definitions of the active verb, and even practice beforehand.
  • Avoid writing essay questions that require factual knowledge,
    as those beginning questions with interrogative pronouns
    (who, when, why, where)
  • Avoid vague, ambiguous, or non-specific verbs
    (consider, examine, discuss, explain)
    unless you include specific instructions in developing responses
  • Have each student answer all the questions
    Do not offer options for questions
  • Structure the question to minimize subjective interpretations

Directions:

  • Present the assignment both verbally and in writing.
    The initial oral plus written presentation to promote and inspire thought;
    written for reference within the test
  • Provide evaluation criteria
  • Focus on the mental activity to avoid rote answers,
    and/or repeating examples from the text
  • Teach students how to write an essay (test)
    explaining definitions of cognitive verbs
  • Teach the difference
    between presenting a position as opposed to presenting an opinion
  • Define requirements clearly
    State the number of points each question is worth
  • Warn students of possible pitfalls
    especially if you have strong ideas of what you do and do not want
  • Inform the students about how you evaluate
    misspelled words, neatness, handwriting, grammar, irrelevant material (bluffing)

Correcting:

  • Develop a model answer
    that contains all necessary points
  • Note additional content for extra points
  • Conceal or ignore students' names in the correcting process
  • Read through the answers to one test item at a time
    without interruption
  • Sequence best through worst responses
    for verification if time permits
  • Write comments on the students’ answers,
    both affirming and correcting
  • Do not give credit for irrelevant material
  • Mix or shuffle papers to vary subject's location
    before assessing the next test item
Curricular guides and resources:

Using feedback in the classroom | Teaching critical thinking | Bloom's taxonomy |
Teaching with questioning | Preparing guided notes |
A curricular idea! | Curricular resources and guides |
Learning Exercises & Games | Exploring learning styles |
Constructing true/false tests | Constructing multiple choice tests |
Constructing essay exams | Cross language resources including digital translators |
Online Learning/eLearning books and resources for teachers

Categories: 1

0 Replies to “Issues Related To Scoring Of Essay Type Test Items On Subject-Verb”

Leave a comment

L'indirizzo email non verrĂ  pubblicato. I campi obbligatori sono contrassegnati *