Scantron Testing at Texas Tech

I set up the Scantron machines for forms like the 5-response Texas Tech custom form:

5-response This form is double sided, up to 240 responses.

We also have a 10-response form, it's red.  It has 120 responses per sheet.

Tech designed the custom form.  We sell it for 10 cents, bookstores resell for whatever they get.  A bit less than 5 cents is what it co$ts us to have the form printed & delivered, usually ordering 180,000 every six months (just under $5,000.00 each time).  The nickel profit goes to maintenance of the equipment, a $21,000 yearly service contract for the two machines below.

In 2009, the form was changed to the current one, the previous one was set-up for a social security number, we now use a "R-number" assigned to a student.  The form also had a department, course and section area.  This was changed to a CRN (Course Request Number), a 5-digit number that signifies the department/course/section.  

There's another form, run twice a year, student evaluation of instructor and class, a mandatory run that we do for Institutional Management/Institutional Research.  All we do is scan and give them the text output from the scanners.

We have NCS Pearson Model 10's.

Scanners The scanners are about 10 years old, the high speed runs about 190,000 sheets per year, the low speed runs about 30,000 sheets per year.

I took my ScanTools class in California about 10 years ago.   Time flies.

The Scantrons do NOT have the incorrect answers marked on the student sheet.

  Instructors can use our testing service without charge.  Students buy the sheets we sell (copyrighted) to the bookstores and to departments.  Surveys can be done with the sheets, we can "grade" the sheets without an identification number, but to have student names associated with a test, a valid CRN for the semester of grade and valid RNumber for a student must be bubbled in.  If a student fails to register a correct RNumber, one is assigned starting with 0000001.  The test will grade, but show the student for the test as "absent" and this grade assigned to R0000001.  If a CRN is incorrect, then all students will show "absent" by name, and the RNumber will have a grade beside it.

Test Input Once the scanner has been loaded, we run a batch job that starts up ScanTools II which senses the bubbles on a Scantron.  This input file is saved as a "generation", an assigned number that's next in the series.  It's format is g0001v00 and is a text file, without an extension.

Texas Tech Technology Operations & Systems Management has three PCs which have SAS 9.2 installed on them.  My two employees who work with the scanners can take the generation number, instructor name and Course Request Number(s) as the input for the SAS job to run the tests.  

All an instructor needs to do is to complete the Test Request form (below) and submit it with his Scantrons.

We give 15 minute (or less) turn-around from the time the test is submitted until the printout from the program is finished.  Most instructors (or their teaching assistants) stay for the results.

Once the test is run, we e-mail the results to the addresses left on the Test Request form.
SAS Program

Test Request Since a CRN is one section of a course, multiple sections can be graded if the instructor includes the CRNs of each section.  We have had up to 24 sections graded at one time.

The SAS program generates output and statistics based on an instructor's choosing.  The report part of the output is sent to the printer below, the file is then e-mailed to the instructor, often receiving the e-mail before they get back to their office.

ALL BETS are OFF during FINALS.  Usually, we can have output turnaround close to the 15-minute target.  Sometimes during finals, we'll have 10 tests submitted at a time.
SAS output
IBM Infoprint 40 We can direct print to multiple printers but usually one will suffice.  It has duplex capabilities but for tests, they are printed one-sided in landscape orientation on 3-hole paper.  

The test output along with the Scantrons submitted are bundled and kept until the instructor picks them up.  After 30 days, the bundle is shredded if not picked up. Output

Response Analysis Response Analysis output in old-fashioned ASCII format.  To change the format might be too much of the idea that the results are not accurate.

Item Analysis has become an automatic, no longer an option an instructor has to code. Item Analysis

Test Output SAS is a very capable programming language to handle the data with the numerous options.

Names have been obliterated to protect the student.

The text file is created by the SAS program.  Student names are in last name/first name sorted alphabetical order listing.  If an instructor has two sections graded at the same time, then the report is sorted by CRN (smaller) alpha A-Z (last name) followed by then next.  Students with a "bad" RNumber are listed at the bottom of the report.  It is possible that that student is someone who is "absent".  RAW score is the number of questions the student got right, % is the grade based on a 100 percentile.  With up to 240 responses on the Orange sheet, the raw score could be 240 but the percent score will never exceed 100%.  

The "key" sheet has the instructor's correct answers, if an instructor wants to skip or exclude a question, the key sheet has a blank for that question number and the question is skipped.

The instructor has the option to have the test graded with seven (7) methods and the test will accomodate the "straight-up" answer sheet plus one of the seven "alternate forms" of the test.

 To wit:
Method 1:  Direct reversal.  7 questions, #7=#1, #6=#2, #6=#3, #5=#4, #4=#5 ...
Method 2:  Odd-Even Switch.  7 questions, #1=#2, #2=#1, #3=#4, #4=#3...
                (if an odd number of questions, then the last question doesn't switch, it remains.)
Method 3:  Split Half-Switch.  7 Questions, #1, #2, #3 become #4, #5, #6
                (if an odd number of questions, then the last question retains it's position.)
Method 4:  Combination of Method 2 and Method 3 IN THAT order.
Method 5:  Combination of Method 3 and Method 1 IN THAT order.
Method 6:  Combination of Method 3 and Method 2 IN THAT order.
Method 7:  Conbination of  Methods 3, 2 and 1 IN THAT order.

The instructor needs to make sure form 2 (the student's instructions/question sheet) of this test has the questions in the order of the alternate form method.

The "weight" sheet can individually weight each question from 1 to 5 points.  If this is the choice, then each question needs to be weighted.  If the first response is bubbled in "A", then ALL questions are equally weighted "1" point, "B" equals 2 points ... and "E" equals 5 points.
Text File Output

Excel output Here's the Excel file of the above output.  Out of the "goodness of our heart", we use the Excel 3-step wizard to provide this service for the instructor.  They COULD do it, but we know how busy those folks are and we have plenty of time and resources (and knowledge) to do it for them...

Student names were mucked up to protect the Students along with their RNumbers and the CRN. The grades are real.

Me This was me <--- before this project and me ---> after.
Always, always have a before & after.
Bill the Cat

Return to TOP

Go to Main Page