Item request has been placed! ×
Item request cannot be made. ×
loading  Processing Request

WordBytes: Exploring an Intermediate Constraint Format for Rapid Classification of Student Answers on Constructed Response Assessments

Item request has been placed! ×
Item request cannot be made. ×
loading   Processing Request
  • Source:
    Journal of Educational Data Mining, v9 n2 p45-71 2017. 27 pp.
  • Accession Number:
    https://jedm.educationaldatamining.org/index.php/JEDM/article/view/209
  • Language:
    English
  • Publication Type:
    Journal Articles; Reports - Research
  • Additional Information
    • Author(s):
    • Availability:
      International Working Group on Educational Data Mining. e-mail: jedm.editor@gmail.com; Web site: http://www.educationaldatamining.org/JEDM/index.php/JEDM/index
    • Peer Reviewed:
      Y
    • ISSN:
      2157-2100
    • Subject Terms:
    • Abstract:
      Computerized classification of student answers offers the possibility of instant feedback and improved learning. Open response (OR) questions provide greater insight into student thinking and understanding than more constrained multiple choice (MC) questions, but development of automated classifiers is more difficult, often requiring training a machine learning system with many human-classified answers. Here we explore a novel intermediate constraint question format called WordBytes (WB) where students assemble one-sentence answers to two different college evolutionary biology questions by choosing, then ordering, fixed tiles containing words and phrases. We found WB allowed students to construct hundreds to thousands of different answers (= 20 tiles), with multiple ways to express correct and incorrect answers with different misconceptions. We found humans could specify rules for an automated WB grader that could accurately classify answers as correct/incorrect with Cohen's kappa = 0.88, near the measured intra-rater reliability of two human graders and the performance of machine classification of OR answers (Nehm et al., 2012). Finer-grained classification to identify the specific misconception had lower accuracy (Cohen's kappa < 0.75), which could be improved either by using a machine learner or revising the rules, but both would require considerably more development effort. Our results indicate that WB may allow rapid development of automated correct/incorrect answer classification without collecting and hand-grading hundreds of student answers.
    • Abstract:
      As Provided
    • Number of References:
      35
    • Sponsoring Agency:
      National Science Foundation (NSF)
    • Contract Number:
      1227245
    • Physical Description:
      27
    • Education Level:
      Higher Education
    • Journal Code:
      APR2018
    • Publication Date:
      2018
    • Accession Number:
      EJ1166127
  • Citations
    • ABNT:
      KIM, K. J. et al. WordBytes: Exploring an Intermediate Constraint Format for Rapid Classification of Student Answers on Constructed Response Assessments. Journal of Educational Data Mining, [s. l.], v. 9, n. 2, p. 45–71, 2017. Disponível em: http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1166127&custid=s8280428. Acesso em: 30 mar. 2020.
    • AMA:
      Kim KJ, Meir E, Pope DS, Wendel D. WordBytes: Exploring an Intermediate Constraint Format for Rapid Classification of Student Answers on Constructed Response Assessments. Journal of Educational Data Mining. 2017;9(2):45-71. http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1166127&custid=s8280428. Accessed March 30, 2020.
    • APA:
      Kim, K. J., Meir, E., Pope, D. S., & Wendel, D. (2017). WordBytes: Exploring an Intermediate Constraint Format for Rapid Classification of Student Answers on Constructed Response Assessments. Journal of Educational Data Mining, 9(2), 45–71.
    • Chicago/Turabian: Author-Date:
      Kim, Kerry J., Eli Meir, Denise S. Pope, and Daniel Wendel. 2017. “WordBytes: Exploring an Intermediate Constraint Format for Rapid Classification of Student Answers on Constructed Response Assessments.” Journal of Educational Data Mining 9 (2): 45–71. http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1166127&custid=s8280428.
    • Harvard:
      Kim, K. J. et al. (2017) ‘WordBytes: Exploring an Intermediate Constraint Format for Rapid Classification of Student Answers on Constructed Response Assessments’, Journal of Educational Data Mining, 9(2), pp. 45–71. Available at: http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1166127&custid=s8280428 (Accessed: 30 March 2020).
    • Harvard: Australian:
      Kim, KJ, Meir, E, Pope, DS & Wendel, D 2017, ‘WordBytes: Exploring an Intermediate Constraint Format for Rapid Classification of Student Answers on Constructed Response Assessments’, Journal of Educational Data Mining, vol. 9, no. 2, pp. 45–71, viewed 30 March 2020, .
    • MLA:
      Kim, Kerry J., et al. “WordBytes: Exploring an Intermediate Constraint Format for Rapid Classification of Student Answers on Constructed Response Assessments.” Journal of Educational Data Mining, vol. 9, no. 2, Jan. 2017, pp. 45–71. EBSCOhost, search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1166127&custid=s8280428.
    • Chicago/Turabian: Humanities:
      Kim, Kerry J., Eli Meir, Denise S. Pope, and Daniel Wendel. “WordBytes: Exploring an Intermediate Constraint Format for Rapid Classification of Student Answers on Constructed Response Assessments.” Journal of Educational Data Mining 9, no. 2 (January 1, 2017): 45–71. http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1166127&custid=s8280428.
    • Vancouver/ICMJE:
      Kim KJ, Meir E, Pope DS, Wendel D. WordBytes: Exploring an Intermediate Constraint Format for Rapid Classification of Student Answers on Constructed Response Assessments. Journal of Educational Data Mining [Internet]. 2017 Jan 1 [cited 2020 Mar 30];9(2):45–71. Available from: http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1166127&custid=s8280428