Publications

Dissertation

1. Ph.D.

   

   Investigating Conversational Agents to Support Secondary School Computer Science Exploratory Search

   Matthew Frazier

   University of Delaware, 2024

   Bib PDF Slides

   @phdthesis{frazier2024investigating,
     title = {Investigating Conversational Agents to Support Secondary School Computer Science Exploratory Search},
     author = {Frazier, Matthew},
     year = {2024},
     school = {University of Delaware},
     publisher = {University of Delaware},
     url = {https://udspace.udel.edu/items/1d3815f9-1e86-4665-9ac1-18e81419da45},
   }

Conference Articles

1. ITiCSE 2024

   

   Customizing ChatGPT to Help Computer Science Principles Students Learn Through Conversation

   Matthew Frazier, Kostadin Damevski, and Lori Pollock

   In Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1, 2024

   Abs Bib PDF Code Slides

   This paper explores leveraging conversational agents, specifically ChatGPT, to enhance the introduction of computing, focused on the Advanced Placement Computer Science Principles (CSP) course in secondary schools. Despite the potential benefits for diverse student audiences, little research has investigated their effectiveness and engagement in this context. We examine the customization of ChatGPT for secondary school CSP students, assessing its impact on exploratory searches for learning CSP concepts. Results from 20 high school students in grades 10-12 (ages 15-18) in a CSP course indicate that students preferred a customized ChatGPT, with its terminology more suitable to secondary school level, examples more understandable, and better connections to personal experiences compared to standard ChatGPT.

   @inproceedings{10.1145/3649217.3653570,
     author = {Frazier, Matthew and Damevski, Kostadin and Pollock, Lori},
     title = {Customizing ChatGPT to Help Computer Science Principles Students Learn Through Conversation},
     year = {2024},
     isbn = {9798400706004},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3649217.3653570},
     doi = {10.1145/3649217.3653570},
     booktitle = {Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1},
     pages = {633-639},
     numpages = {7},
     keywords = {chatgpt, computer science principles, conversational agent, exploratory search},
     location = {Milan, Italy},
     series = {ITiCSE 2024},
   }

2. ICSE 2022

   

   Investigating User Perceptions of Conversational Agents for Software-Related Exploratory Web Search

   Matthew Frazier, Shaayal Kumar, Kostadin Damevski, and Lori Pollock

   In Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results, 2022

   Abs Bib PDF Code Poster Slides

   Conversational agents that respond to user information requests through a natural conversation have the potential to revolutionize how we acquire new information on the Web (i.e., perform exploratory Web searches). Recent advances to conversational search agents use popular Web search engines as a back-end and sophisticated AI algorithms to maintain context, automatically generate search queries, and summarize results into utterances. While showing impressive results on general topics, the potential of this technology for software engineering is unclear.In this paper, we study the potential of conversational search agents to aid software developers as they acquire new knowledge. We also obtain user perceptions of how far the most recent generation of such systems (e.g., Facebook’s BlenderBot2) has come in its ability to serve software developers. Our study indicates that users find conversational agents helpful in gaining useful information for software-related exploratory search; however, their perceptions also indicate a large gap between expectations and current state of the art tools, especially in providing high-quality information. Participant responses provide directions for future work.

   @inproceedings{10.1145/3510455.3512778,
     author = {Frazier, Matthew and Kumar, Shaayal and Damevski, Kostadin and Pollock, Lori},
     title = {Investigating User Perceptions of Conversational Agents for Software-Related Exploratory Web Search},
     year = {2022},
     isbn = {9781450392242},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3510455.3512778},
     doi = {10.1145/3510455.3512778},
     booktitle = {Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results},
     pages = {51-55},
     numpages = {5},
     keywords = {wizard of Oz study, exploratory search, conversational search agent},
     location = {Pittsburgh, Pennsylvania},
     series = {ICSE-NIER '22},
   }