portrait
Alok Baikadi
Artificial Intelligence, Education, and Games Researcher
Affiliations

About Me

Research Interests
My research interests all in the intersection of Artificial Intelligence, Education, and Narrative. I have over a decade of experience building novel educational experiences based in artificial intelligence for both K-12 and Higher Ed classrooms.
Research History
I received my Ph.D in 2014 from North Carolina State University, where I studied under Dr. James Lester. During the course of my studies, I worked on several projects to support middle school students in both science and creative writing classrooms. Within these projects, I participated in all phases of the research, including initial software development, ai capability design & development, in-class observational studies, data extraction and curation, and final inferential data analysis and evaluation.

After finishing my Ph.D, I started a post-doctoral position at the Learning Research and Development Center, at the University of Pittsburgh. The focus of my post-doctoral research was research and development for the SWoRD Peer-Review writing support system. We developed a revision planning intervention which allowed students to collect the feedback they were receiving and make concrete decisions about how to incorporate the feedback into their drafts. As with my graduate work, I developed softwware, participated in in-class observational studies, extracted and curated data, and performed the final inferential data analysis and evaluation. I received a second post-doctoral research associate positiont with the Center for Instructional Design and Distance Education. This second project was focused on behavioral analysis of student interactions within PITT Online and Coursera courses.

Since leaving academia and entering industry, I've worked as a Data Scientist at Pearson, and then ETS. I've been involved in the end-to-end development of capabilities and prototypes of novel AI-enabled learning experiences, focusing on writing and language learning. I've lead multi-disciplinary teams of software engineer, data scientists, data engineers and data scientists, to build prototypes, develop capabilities, and design measurements which can lead to actionable insights.

Research Projects

Active Learning in Essay Scoring

An essay scoring prototype which collaborates with an instructor to build an essay scoring model over the course of a semester using online active learning. Contributions include an active learning algorithm for suggesting the order in which the instructor should grade the essays and a UX framework for communicating online learning flow and status to a non-ML fluent user.

UIBK 2019 LAS 2019

P e n d i n g

MOOC Data Analytics

An analysis of student interactions, including video interactions, in-class quiz interactions, offline assignments completed, and forum posts.

ILS 2018 EDM 2016

Revision in Peer Review

A revision planning intervention that allows students to consider how to respond to the feedback received and lessons learned from their reviewing tasks.

IS 2017 EDM Workshops 2016 EDM 2015

Goal Recognition in Narrative Games

Modeling student problem solving processes within an intelligent game-based learning environment for middle-grade science education.

AIIDE 2016 UMAP 2014 INT 2013 INT 2012 CMN 2012

JavaTutor

A system for selecting dialogue utterances based on features of the problem progress and task for an intelligent tutorial dialogue system for introductory Java concepts.

Narrative Theatre

An intelligent creativity-enhancement environment for 6th grade writing education. Students are guided through the process of planning and writing a fable, which is then visualized in a storyboard to aid in the creative revision process.

Action Representation in Description Logics

Developed a description logic framework for representing hierarchical actions for both recognition and planning.

Publications & Patents

Capstones & Dissertations

  • Ph.D Dissertation Discovery-Based Goal Recognition in Interactive Narrative Environments Department of Computer Science, NC State University

    Advisor: Dr. James Lester

  • Senior Thesis Partial Recognition and Reasoning about Actions Department of Computer Science, University of Illinois

    Advisor: Dr. Eyal Amir

Journal Articles

  • Alok Baikadi, Carrie Demmans Epp, Christian D. Schunn. (2018) Participating by activity or by week in MOOCs. Information and Learning Science 119(9/10), 572-585.

  • Fuhui Zhang, Christian D. Schunn, Alok Baikadi. (2017) Charting the routes to revision: An interplay of writing goals, peer comments, and self-reflections from peer reviews. Instructional Science 45(5), 679-707.

Conference Publications

  • Scott Hellman, Mark Rosenstein, Andrew Gorman, William Murray, Lee Becker, Alok Baikadi, Jill Budden, Peter W. Foltz. Scaling Up Writing in the Curricilum: Batch Mode Active Learning for Automated Essay Scoring. Sixth Conference on Learning @ Scale, Chicago, IL. 2019

  • Wookhee Min, Alok Baikadi, Bradford Mott, Jonathan Rowe, Barry Liu, Eun Ha, James Lester.. A Generalized Multidimensional Evaluation Framework for Player Goal Recognition.. To Appear In Proceedings of the Twelfth Annual AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Burlingame, California. 2016

  • Alok Baikadi, Christian Schunn, Carrie Demmans-Epp, Yanjin Long.. Redifining “What” In Analyses of Who Does What in MOOCs. In Proceedings of the Ninth International Conference on Educational Data Mining, Raleigh, NC. pp. 569-570. 2016

  • Alok Baikadi, Christian Schunn, Kevin Ashley. Understanding Revision Planning in Peer-Reviewed Writing. In Proceedings of the Eight International Conference on Educational Data Mining, Madrid, Spain. pp. 544 – 548. 2015 [PDF]

  • Alok Baikadi, Jonathan Rowe, Bradford Mott, James Lester.. Generalizability of Goal Recognition Models in Narrative-Centered Learning Environments. In Proceedings of the Twenty First International Conference on User Modeling, Adaptation and Personalization, Aalborg, Denmark. pp. 278 – 289. 2014 [PDF]

  • Eun Young Ha, Alok Baikadi, Carlyle J. Licata, Bradford W. Mott, James C. Lester.. Exploring the Effectiveness of Lexical Ontologies for Modeling Temporal Relations with Markov Logic. In Proceedings of the Eleventh Annual SIGDIAL Meeting on Discourse and Dialogue, Tokyo, Japan. pp. 75–79. 2010 [PDF]

Conference Presentations

  • Loretto, Adam, Godley, Amanda, Alok Baikadi. A think-aloud study of adolescent writers’ peer review and revision strategies. Paper presented at the 2015 Literacy Research Association Annual Conference, Carlsbad, CA.. 2015, December

Workshop Publications

  • Alok Baikadi, Lee Becker, Jill Budden, Peter Foltz, Andrew Gorman, Scott Hellman, William Murray, Mark Rosentstein. An apprenticeship model for human and AI collaborative essay grading. The First Workshop on User Interactions for Building Knowledge (UIBK), Los Angeles, CA. 2019

  • Alok Baikadi, Christian Schunn, Kevin Ashley. Impact of Revision Planning in Peer-Reviewed Writing. To Appear In Proceedings of the Second Workshop on Computer-Supported Peer Review in Education, Raleigh, NC. 2016

  • Alok Baikadi, Jonathan P. Rowe, Bradford W. Mott, James C. Lester. Improving Goal Recognition in Interactive Narratives with Models of Narrative Discovery Events. In Proceedings of the Sixth Workshop on Intelligent Narrative Technologies, Boston, MA. pp. 2-8. 2013 [PDF]

  • Alok Baikadi, Jonathan P. Rowe, Bradford W. Mott, James C. Lester. Toward Narrative Schema-Based Goal Recognition Models for Interactive Narrative Environments. In Proceedings of the Fifth Workshop on Intelligent Narrative Technologies, Stanford, CA. pp. 2-7. 2012 [PDF]

  • Alok Baikadi, Rogelio E. Cardona-Rivera. Towards finding the fundamental unit of narrative: A Proposal for the Narreme. In Proceedings of the 2012 Workshop on Computational Models of Narrative, Istanbul, Turkey. pp. 44-46. 2012 [PDF]

  • Alok Baikadi, Julius Goth, Christopher M. Mitchel, Eun Young Ha, Bradford W. Mott, James C. Lester. Towards a Computational Model of Narrative Visualization. In Proceedings of the Fourth Workshop on Intelligent Narrative Technologies, Stanford, California. pp. 2-9. 2011 [PDF]

  • Eun Young Ha, Alok Baikadi, Carlyle Licata, James C. Lester. NCSU: Modeling Temporal Relations with Markov Logic and Lexical Ontology. In Proceedings of the Fifth International Workshop on Semantic Evaluation, Uppsala, Sweden. pp. 341-344. 2010 [PDF]

  • Julius Goth, Alok Baikadi, Eun Young Ha, Jonathan Rowe, Bradford Mott, James Lester. Exploring Individual Differences in Student Writing with a Narrative Composition Support Environment. In Proceedings of the First NAACL HLT Workshop on Computational Linguistics & Writing (CL&W), Los Angeles, California. pp. 56-64. 2010 [PDF]

Issued Patents

  • US Patent 11741849, Systems and methods for interface-based machine learning model output customization, August 29 2023

    Inventors: Hellman; Scott (Dallas, TX), Murray; William (Fort Collins, CO) Habermehl; Kyle (Niwat, CO), Baikadi; Alok (Boulder, CO), Budden; Jill (Boulder, CO), Gorman; Andrew (Bloomington, MN), Rosenstein; Mark (Boulder, CO), Becker; Lee (Boulder, CO), Hopkins; Stephen (Huntington Station, NY), Foltz; Peter (Boulder, CO)

  • US Patent 11475245, Systems and methods for automated evaluation model customization, October 18 2022

    Inventors: Foltz; Peter (Boulder, CO), Rosenstein; Mark (Boulder, CO), Baikadi; Alok (Boulder, CO), Becker; Lee (Boulder, CO), Hopkins; Stephen (Huntington Station, NY), Budden; Jill (Boulder, CO), Oros; Louis M. (Denver, CO), Habermehl; Kyle (Niwat, CO), Hellman; Scott (Dallas, TX), Murray; William (Bloomington, MN), Gorman; Andrew (Bloomington, MN)

  • US Patent 11449762, Real time development of auto scoring essay models for custom created prompts, Semptember 20 2022

    Inventors: Rosenstein; Mark (Boulder, CO), Habermehl; Kyle (Niwat, CO), Hellman; Scott (Dallas, TX), Baikadi; Alok (Boulder, CO), Foltz; Peter (Boulder, CO), Becker; Lee (Boulder, CO), Oros; Louis M. (Denver, CO), Budden; Jill (Boulder, CO), Derr; Marcia (Boulder, CO)

  • US Patent 11443140, Systems and Methods for automated machine learning model training for a custom authored prompt, September 13 2022

    Inventors: Hellman; Scott (Dallas, TX), Becker; Lee (Boulder, CO), Dawson; Samuel (Bloomington, MN), Baikadi; Alok (Boulder, CO), Murray; William (Fort Collins, CO), Habermehl; Kyle (Niwat, CO), Foltz; Peter (Boulder, CO), Rosenstein; Mark (Boulder, CO)

  • US Patent 11416551, System and method for automated sequencing database generation, September 21 2020

    Inventors: Murray; William (Lone Tree, CO), Baikadi; Alok (Boulder, CO)

  • US Patent 10860940, System and method for automated sequencing database generation, December 8 2020

    Inventors: Murray; William (Lone Tree, CO), Baikadi; Alok (Boulder, CO)

  • US Patent 10783185, System and method for automated hybrid sequencing database generation, September 22 2020

    Inventors: Murray; William (Lone Tree, CO), Baikadi; Alok (Boulder, CO)

  • US Patent 10754899, System and method for sequencing database-based content recommendation, August 25 2020

    Inventors: Murray; William (Lone Tree, CO), Baikadi; Alok (Boulder, CO)

Work & Education History

Work History

  • ETS - AI Labs
    2021-2024
    Managing Sr. Data Scientist
    • Manage a team of 4 data scientists and 5 data engineers tasked with collecting data from ML-enabled product prototypes, manage data sets for ML model development and efficacy research, and develop data-driven insights to aid in high level decision making. Hired and onboarded 3 new employees across both lines. Mentored 2 conversions to an AI Engineering role, and 3 promotions.
    • Develop a workflow using git CI/CD, python and Docker integrated into an AWS data platform that team members used to deliver KPI dashboards in PowerBI and Tableau for 10 prototypes accessed by 35,000 customers. Dashboards serviced internal and executive stakeholders.
    • Lead design and development of a centralized AWS data platform to gather and process over 4 million events from prototypes and products and provide data exports of derivative data sets for research and calculate KPIs to service the dashboards.
    • Design and implement workflows for cross-functional collaboration to leverage data to improve outcomes for other functional roles, including educational psychology researchers, learning designers, UX researchers, and ML model developers.
  • Pearson - AI Products & Solutions
    2017-2021
    Principal Data Scientist
    2019-2021
    Senior Data Scientist
    2017-2019
    • Manage a team of 2 software engineers, 3 data scientists, and a UX researcher to deliver novel NLP product prototypes and capabilities, evaluate the capabilities deployed in products, and deliver over 50 assessment models to over 1000 early customers for evaluation.
    • Design, analyze and publish the results of a multi-condition study to convey the usability, accuracy, and acceptability of ML-based prototypes and capabilities.
    • Manage data collection efforts, extracting data from internally developed prototypes, sql and nosql product databases, and external collaborators. Manage requirements and delivery timelines for a data annotation tool, design of annotation schema, and annotation process and evaluation to produce data for ML model design.
    • Communicate results and protect IP through 2 publications and 8 issued patents.
    • Manage requirements definition, development process, and timelines to ensure delivery of python flask product prototypes, market evaluation, and integrated classical and large-language NLP models for products and services.
    • Lead cross-functional design & development to integrate ML capabilities into internal and external products, collaborating with internal marketing, software development, and product teams.
  • University of Pittsburgh - Learning Research and Development Center
    20014-2017
    Post-doctoral Research Associate
    2016-2017
    Post-doctoral Research Assisstant
    2014-2016
    • Design, implement, and deliver a new software feature using Java/Swing into a edtech product to support the writing process in 3 studies with over 300 early customers.
    • Design and analyze 3 multi-condition studies to test the efficacy of the designed software.
    • Collaborate with researchers and professors in Education and Cognitive Psychology, Computer Science, and Law to design research studies and collect data to evaluate the impact of software solutions and develop NLP models.
    • Design and execute an analysis plan, using multiple regression approaches and factor analysis, to interpret student behavior in 4 university MOOCs containing over 10,000 students.
    • Communicate findings through 3 publications and presentations, and 2 journal publications.
    • Mentor 4 undergraduate students for research experience, and 1 graduate researcher to complete a novel research study and publication. Lead 6 local instructors through a data-annotation task to produce high-quality data labels for further analysis and ML model development.
  • North Carolina State University - Center for Educational Informatics
    2007-2013
    Research Assistant
    2009-2013
    • Design and implement software components using Unity/C#, java and lisp, and NLP & behavior-based graphical machine learning models using research toolkits for novel ML-enabled product prototypes, and deliver to classrooms with over 100 early users each.
    • Design, implement, and analyze multi-condition studies with over 100 users each for the purpose of evaluating the efficacy of ML interventions.
    • Communicate results through 9 publications and presentations.
    • Participate in cross-functional collaboration with education researchers, software developers, and AI researchers to design research studies and collect, organize and process data from studies for research and ML model development.
    • Mentor 2 undergraduate students for research experience and delivery of software components for prototypes.
    Teaching Assistant
    2007-2009
    • Graded exams and assignments and held office hours for 3 graduate classes: Intro to Artificial Intelligence, Machine Learning, and Graph Theory
    • Developed curriculum, classroom materials and assignments, Graded exams and assignments, and delivered all lectures for 2 semesters of an Intro to AI course at the undergraduate level.
  • Grammatech
    Summer 2007
    Summer Intern
    • Developed software to analyze gdb output in C and Lisp.
  • Intel
    Summer 2005
    Summer Intern
    • Developed utility software in VisualBasic, VisualC++, and VBA for the Automation Group.

Education

  • Ph.D. Computer Science
    2007-2013

    North Carolina State University

  • B.S. Computer Science
    2003-2007

    University of Illinois at Urbana-Champagin

© 2013-2023 Alok Baikadi. All rights reserved.