This online tool provides basic analysis of multiple-choice tests. The analysis includes a detailed view of each examinee's responses with correct answers indicated, each examinee's overall average score, and a distractor analysis that shows the percent of examinees that selected each possible answer. The difficulty index (percent of examinees that correctly answered each item) is indicated in the distractor analysis.
Contents
Data for an exam to be analyzed must be saved in a plain ASCII text file that follows this specification. The first two lines of the data file define the test key and the number of distractors for each item. The remainder of the data file contains each examinee's answers, one row per examinee. For purposes of this description, the term "record" will refer to a single row in the data file, and the term "field" will refer to a single data item (e.g., test answer or student ID number) within a row. The ASCII file must be tab-delimited; that is, each field in each record must be separated by a tab space.
Test Definition Records (rows 1-2)
The first record (row) in the data file contains the key (correct answers) for the test. The first field in this record must be the word "KEY." A tab space separates this field from the correct answer for the first question, which is placed in field 2.
The second record in the data file contains the "scale" (number of distractors used) for each item. The first field in this record must be the word "SCALE." The scale definitions for each item in the test then follow, separated by tab spaces.
The following example shows the layout of test definition records for a five-item test:
KEY A B A C D SCALE 5 4 4 5 5
In the above example, the correct answer for the first question is "A," and this item has five possible answers. The correct answer for the second question is "B", and that item has four possible answers. A "yes/no" or "true/false" question would have "2" in the appropriate field of the SCALE record.
Note: As currently configured, the analysis tool is restricted to five possible distractors (i.e., "5" is the maximum that can be in the SCALE field for any question), and answers are set up on an A-E scale.
Examinee Answer Records (rows 3...)
Examinee "answer sheets" begin with the third record in the data file. The first field in each record must be a unique student identifier. The identifier may be alphanumeric; names are permissible but not advised. Continuing with the example above, five student records are added to the data file:
KEY A B A C D SCALE 5 4 4 5 5 001 A B B C D 002 B B A C D 003 A C A B C 004 A B B B D 005 C D D C E
Data files for small tests with few students may be easily prepared using a text editor such as Windows Notepad or Wordpad. The only critical formatting is the tab-delimitation between fields in each record. Data files should be saved with a filename extension of ".dat" (e.g., datafile.dat). To avoid confusion later, it is a good idea to name the data file with some indicator of the class and date, but this is not required by the system.
For larger data sets, spreadsheet software such as Microsoft Excel may be used to build the data file. Enter each field in a separate column and each record in a separate row as shown below:
Then, highlight only the block in the spreadsheet that contains the data:
Next, use either the menu option File|Copy or the keyboard shortcut Ctrl+C to copy the highlighted block to the clipboard. Open the text editor and use File|Paste or Ctrl+V to paste the data into the text editor. Tab spaces will automatically be formatted when pasting from Microsoft Excel. Save the file that has been created in Notepad with a ".dat" filename extension.
For very large tests, optical scanning is often used for scoring the exam. If the optical scanner has an option for creating an ASCII data file, this may be used to produce the data file. However, the scanned data will have to be edited using a text editor. (Wordpad will probably be necessary, as Notepad has a rather low limit on the size of files it can load; when using Wordpad be sure that the file is saved in ASCII text).
Optical scanners generally produce fixed-field data files. These must be "parsed" to insert the tab delimiters between fields. Again, this can be accomplished using Microsoft Excel.
First, save the fixed field data file from the optical scanner where it can be easily located. It is a good idea to use a ".txt" filename extension; this keeps the fixed field data separate from data that has already been formatted for analysis, and it works more easily with Excel.
Start Excel and select File|Open. Navigate to the location where the fixed field data file has been saved. In the "Files of Type" dropdown list at the bottom of the "Open" dialog box, select "Text Files (*.prn; *.txt; *.csv)" and the fixed field data file should appear in the file list box:
Click "Open," and Excel will display the "Text Import Wizard." The first few lines of the fixed field data file will appear in a small preview box near the bottom of the wizard display. Ensure that "Fixed Width" is checked at the top, and click "Next."
In the second "Text Import Wizard" box, click on the ruler above the preview window to divide the fixed field file into separate data fields. Divisions may be removed by double-clicking or relocated by drag-and-drop. Note that no division is placed after the last (right-most) data field. When the fields have been properly defined, click "Next."
In the third step, define the format for each field as text. This is very important! By default, Excel will use a "General" formatting method that automatically defines fields as numeric or text based on the content of the first row. This can mess up the student ID field, particularly if longer numbers are used. ("General" formatting may impose scientific notation on a social security number, for example.) Err on the side of caution: select each field in the preview window and check "Text" under "Column Data Format."
Click "Finish," and the data will be split into proper columns in a regular Excel spreadsheet. Select the first two rows by clicking, holding, and dragging down on the numbered row labels on the left side of the rows:
Right-click over the left-hand numbered row labels to display the pop-up menu and select "Insert" to place two empty rows above the first examinee data record. Add KEY and SCALE definitions in these new rows:
Save your work in Excel format. Be sure to check the "Save as Type" to specify Excel format. Then, follow the steps outlined above to copy and paste the data into a text editor. Save the text file with a ".dat" filename extension.
From the Control Panel, select "Upload new test data." Up to three data files can be uploaded at a time. Click the "Browse" button beside each filename box to locate the data file(s) to upload. Then, click "Upload!" to transfer the data files to the server. If the upload file has the same filename as data that is already on the server, the system will add an incremental number to the end of the new filename (e.g., "datafile.dat" would be renamed "datafile1.dat"). After the file upload process is finished, the Control Panel is displayed again.
Select "Analyze a test" from the Control Panel. A list of data files that have been uploaded to the server will appear. Note that only data files ending in ".dat" will be displayed in the list.
Select the data file for analysis and click "Analyze It!" If the data file is correctly formatted, nice results will appear. If the data file is not properly set up, the results will be trash. Garbage in, garbage out.
The output may be saved for future reference. Use File|Save As in the web browser menu to do this. Be sure to rename the file with an ".htm" or ".html" filename, since it will have a ".pl" or ".cgi" file extension by default.