It’s 8 a.m. on a Thursday, nearly three weeks after you took the SAT.
You nervously access your College Board account to get your scores. You’re familiar with the 800-point scale, but you see that your essay received 8 out of a possible 12 points. Isn’t that 67 percent, a C?
In a word: no. Actually, your essay was scored holistically. The readers believed you showed “adequate mastery” but also displayed “lapses in quality.” You may still wonder, however: What exactly goes into holistic scoring, and what can you do to drive up your essay scores this school year?
Holistic Essay Scoring: What it Means
The word holistic is widely used in education. For example, you may have heard that college admissions officers review candidates holistically; that is, they form an overall impression of the candidate by considering many attributes (e.g., grades, test scores, talent, ability) before deciding whether to admit the individual.
Holistic reviews of essays are similar. A rater reads through the essay to get an overall sense of its merit on a given scale (e.g., 1 to 6), and then adjusts that score based on how well the writing displays particular characteristics. What we commonly refer to as holistic scoring is actually criterion-based holistic scoring. This type of evaluation began in the late 1980s and became popular in the 1990s, especially when schools began to assess writing using prompts in timed situations. That’s also when the word rubric became widely used in academia to refer to the scoring instrument that assigns points across specific criteria.
Advocates for criterion-based essay scoring believe that it eliminates bias and allows educators, families, and students to compare relative performance across testing populations over time — even though the writing task may change. According to a New Jersey Department of Education handbook, “Criterion-based holistic scoring brings uniformity to the evaluation of writing across contents and settings by specifying salient features of writing quality and levels of writing proficiency.”
The Writer’s Task: How to Write for Your Audience
When teaching language arts, I found rubrics to be effective instruments for scoring essays and research papers because they clarified my expectations before students wrote and submitted the assignment. In other words, criterion-based holistic scoring makes it easy for students to deliver excellent work. To illustrate this point, let’s take a look at rubrics for two high-stakes exams:
Example: The 2015–2016 ACT
Effective this fall, the ACT will introduce what is believed to be a more difficult essay prompt. Test-takers will be asked to evaluate three perspectives on an issue, offer their own viewpoint, and compare their personal position to those provided — as in this sample prompt. As in previous years, each of two raters awards a score of 6, for a total out of 12 points — and if two raters differ by more than one point, a third casts the deciding vote.
This school year, ACT raters will use a new rubric to evaluate the essay. It clearly presents the expectations at each score point level across four categories: Ideas and Analysis (the complexity of an essay’s argument); Development and Support (expression of ideas in a logical fashion and confirmed with apt reasoning and examples); Organization (essay is unified by a central concept or focus and structured to successfully demonstrate relationships between ideas); and Language Use (appropriate word choice and tone that accurately conveys the intended meaning). The raters, experienced teachers, are thoroughly trained in using model essays, known as anchor papers, at each score point level.
Example: AP English Language and Composition
Any student preparing for an AP exam should be familiar with the criteria used to come up with the final score. That way, she can adjust her test-taking strategy accordingly. A student can be a prolific writer, but that does not guarantee success on “AP Lang.” Success, rather, depends on how well the student adheres to the rubric, or scoring guidelines.
Take a look at the scoring guidelines for the 2013 essay. The College Board explains:
All essays, even those scored 8 or 9, may contain occasional lapses in analysis, prose style, or mechanics. Such features should enter into a holistic evaluation of an essay’s overall quality. In no case should an essay with many distracting errors in grammar and mechanics score higher than a 2.
What does this mean? In a nutshell, you can still earn a high score if you make occasional mistakes in spelling or sentence construction (maybe you wrote “if I was” instead of “if I were,” or misspelled “misspell”) — but if your essay is riddled with errors, your score will reflect that.
Holistic scoring is, after all, intended to assess your ideas, analysis, ability to structure an essay, and, finally, your writing skills. But infrequent lapses in one of these areas need not preclude you from earning an overall high score.
Timed Writing: What is Expected
Depending on your level of preparedness and ability to perform under pressure, you may actually prefer timed writing to essays written at home. When writing is timed, the rater has to allow for some errors that would occur in a first draft. The ACT guidelines for a 6.0 (the highest score) in Language clearly state: “While a few minor errors in grammar, usage, and mechanics may be present, they do not impede understanding.” Contrast that with a research paper or take-home essay, in which there is not much, if any, tolerance of such errors.
The College Board agrees:
The essay score should reflect the essay’s quality as a whole. Remember that students had only 15 minutes to read the sources and 40 minutes to write; the paper, therefore, is not a finished product and should not be judged by standards appropriate for an out-of-class assignment. Evaluate the paper as a draft, making certain to reward students for what they do well.
In other words, you want to take care with your writing — but you needn’t (shouldn’t) perseverate over every grammatical detail, because occasional mistakes won’t matter, and you’ll need to use the little precious time you have to develop a thoughtful, clear essay.
Next Step: How to Drive Up that Score!
Now that you know what holistic scoring entails, you can follow these six strategies for improving your scores.
1. Know what matters to your grader. Always review a rubric at the beginning of a writing assignment. For holistically-scored essays on standardized tests, be familiar with the rubric well in advance of the test date.
2. Keep track of whether you’re meeting your goals. If necessary, create your own checklist when you brainstorm your answers to ensure you have addressed all required criteria, such as point of view; organization and focus; use of language; sentence structure; grammar and usage. Additionally, when you brainstorm, write down the categories identified in the prompt. For example, for the SAT essay, students are asked to “Support your position with reasoning and examples taken from your reading, studies, experience, or observations.” Make a box for each category — reading, studies, experience, observations — and jot down examples to reference as you write.
3. Check your work — especially when you have time. For at-home assignments, be sure to revisit the rubric after writing the essay and before handing it in.
4. Get feedback. When appropriate, seek help from your teacher to see if you’re on the right track.
5. Learn from past performance. Go back to your previous essays to see how they were holistically scored and why; work to improve where necessary. If you received an 8.0 on your SAT essay, for instance, trying using stronger examples, think more carefully about the structure of the essay, use more varied vocabulary and sentence structure, and proofread to catch errors in grammar and usage.
6. Try your hand at holistic assessments. If examples of essays at different score points are available, such as those attached to AP English Language and Composition, read them to get a greater understanding of what’s required. You can also cover up the score of a sample essay, read the essay, and then try to guess the score awarded. The College Board and ACT make sample essays available. Also check out the College Board essay scoring guide, in effect until March 2016.
During your time as a student, you can expect holistic scoring to continue to play a role in your assessments. Remember to scrutinize the rubric, develop a thesis, and support it with eloquent examples. Then, when you receive your test scores, you can be confident in your success!
For more information, check out this article on how SAT essay portion will change in 2016.
You can also find more expert advice from Nina Berler and others on test-taking strategies and applying to college.
Red Rubric Marker
Instructors have many tasks to perform during the semester. Among those is grading, which can be subjective and unstructured. Time spent constructing grading rubrics while developing assignments benefits all parties involved with the course: students, teaching assistants and instructors alike. Sometimes referred to as a grading schema or matrix, a rubric is a tool for assessing student knowledge and providing constructive feedback. Rubrics are comprised of a list of skills or qualities students must demonstrate in completing an assignment, each with a rating criterion for evaluating the student’s performance. Rubrics bring clarity and consistency to the grading process and make grading more efficient.
Rubrics can be established for a variety of assignments such as essays, papers, lab observations, science posters, presentations, etc. Regardless of the discipline, every assignment contains elements that address an important skill or quality. The rubric helps bring focus to those elements and serves as a guide for consistent grading that can be used from year to year.
Whether used in a large survey course or a small upper-level seminar, rubrics benefit both students and instructors. The most obvious benefit is the production of a structured, consistent guideline for assigning grades. With clearly established criteria, there is less concern about subjective evaluation. Once created, a rubric can be used every time to normalize grading across sections or semesters. When the rubric for an assignment is shared with teaching assistants, it provides guidance on how to translate the instructor’s expectations for evaluating student submissions consistently. The rubric makes it easier for teaching assistants to give constructive feedback to students. In addition, the instructor can supply pre-constructed comments for uniformity in grading.
Some instructors supply copies of the grading rubric to their students so they can use it as a guide for completing their assignments. This can also reduce grade disputes. When discussing grades with students, a rubric acts as a reminder of important aspects of the assignment and how each are evaluated.
Below are basic elements of rubrics, with two types to consider.
I. Anatomy of a rubric
All rubrics have three elements: the objective, its criteria, and the evaluation scores.
Before creating a rubric, it is important to determine learning objectives for the assignment. What you expect your students to learn will be the foundation for the criteria you establish for assessing their performance. As you are considering the criteria or writing the assignment, you may revise the learning objectives or adjust the significance of the objective within the assignment. This iteration can help you hone in on what is the most important aspect of the assignment, choose the appropriate criteria, and determine how to weigh the scoring.
When writing the criteria (i.e., evaluation descriptors), start by describing the highest exemplary result for the objective, the lowest that is still acceptable for credit, and what would be considered unacceptable. You can express variations between the highest and the lowest if desired. Be concise by using explicit verbs that relate directly to the quality or skill that demonstrates student competency. There are lists of verbs associated with cognitive categories found in Bloom’s taxonomy (Knowledge, Comprehension, Application, Evaluation, Analysis, and Synthesis). These lists express the qualities and skills required to achieve knowledge, comprehension or critical thinking (Google “verbs for Bloom’s Taxonomy”).
The evaluation score for the criterion can use any schema as long as it is clear how it equates to a total grade. Keep in mind that the scores for objectives can be weighted differently so that you can emphasize the skills and qualities that have the most significance to the learning objectives.
II. Types of rubrics
There are two main types of rubrics: holistic (simplistic) and analytical (detailed).
Selecting your rubric type depends on how multi-faceted the tasks are and whether or not the skill requires a high degree of proficiency on the part of the student.
A holistic rubric contains broad objectives and lists evaluation scores, each with an overall criterion summary that encompasses multiple skills or qualities of the objective. This approach is more simplistic and relies on generalizations when writing the criteria.
The criterion descriptions can list the skills or qualities as separate bullets to make it easier for a grader to see what makes up an evaluation score. Below is an example of a holistic rubric for a simple writing assignment.
An analytical rubric provides a list of detailed learning objectives, each with its own rating scheme that corresponds to a specific skill or quality to be evaluated using the criterion. Analytical rubrics provide scoring for individual aspects of a learning objective, but they usually require more time to create. When using analytical rubrics, it may be necessary to consider weighing the score using a different scoring scale or score multipliers for the learning objectives. Below is an example of an analytical rubric for a chemistry lab that uses multipliers.
It is beneficial to view rubrics for similar courses to get an idea how others evaluate their course work. A keyword search for “grading rubrics” in a web search engine like Google will return many useful examples. Both Blackboard and Turnitin have tools for creating grading rubrics for a variety of course assignments.
Teaching Professor, Chemistry, JHU
Louise Pasternack earned a Ph.D. in chemistry from Johns Hopkins. Prior to returning to JHU as a senior lecturer, Louise Pasternack was a research scientist at the Naval Research Laboratory. She has been teaching introductory chemistry laboratory at JHU since 2001 and has taught more than 7000 students with the help of more than 250 teaching assistants. She became a teaching professor at Hopkins in 2013.
Image sources: © 2014 Reid Sczerba