We have no idea of the validity or reliability of the test. While McGraw Hill and the Colorado Student Assessment Program are the sole means for auditing schools, the test has never been independently evaluated for accuracy, validity, or reliability.

As a fourth grade teacher, I first administered the CSAP in 1997. Initially I was enthusiastic. I had heard that criterion referenced tests went beyond multiple choice and singular linear solutions and allowed for every child to be judged proficient. Despite the hype, CSAP isn't much different than the ITBS or Terra Nova tests of earlier generations. While CSAP developers don't follow the traditional "norming" bell curve, the criteria is determined according to a distribution schedule.

Currently Kelly Services is contracted to score these responses. The only requirement is a college degree - which can be in culinary arts or personal training. Graders are trained over an evening. They are then assigned a group of questions to evaluate. It is not uncommon for these temp workers to score hundreds of responses over the course of one shift.

Consider question number 23: "The speaker of the poem states, ""I watched in silence, till my heart was full of clear dark water.’ Write an explanation to tell what the speaker means." There is no "correct" answer here. The whole point of literature is interpretation. Graders in this instance are trained to look for specific words and correct sentence structure (note the sentence structure of the poem).

I first began investigating the Colorado Student Assessment Program after my students’ test scores were returned. Not only were the results vague and general enough to not be useful, but in many instances, my students scores were completely inaccurate. The best writing student in my class that year was graded "below proficient." The essay prompt on the test that year was to write about an alien landing on a hill.