Abstract:

Multiple initiatives aim to apply technology for learning, specifically artificial intelligence. At least, two main reasons justify these efforts. First, education is a fundamental process in every society, and it should benefit from technological innovations. Second, students will live surrounded by technology, so they need to become familiar with it to be autonomous and responsible citizens. This is true a fortiori for STEM subjects, particularly programming and computational thinking, which are key to training professionals that will not only address the demands of the future industry but also understand and help to navigate a world where almost everything will be mediated by technology. In addition, programming subjects are known to be hard, and especially challenging for recent generations. Thus, any possible aid for these students deserves attention.

The irruption of generative AI (GenAI) has shaken the education sector, partly because it has questioned traditional evaluation systems, but also because of its huge possibilities to augment students’ learning capabilities, through content generation, virtual assistants, personalized question-solving, and many other applications. In any case, many questions remain unanswered about the best approaches to integrating these technologies into learning.

GenAI showed impressive results in generating programming code, which makes it an obvious candidate to augment programming learning environments, such as college-level courses. However, there is no consensus on the approach that will help students to learn rather than bypassing the evaluation instances. In addition, different attempts have been made to automate several tasks in the context of programming learning and teaching, with variable results. Large language models cannot yet reproduce human skills when asked to evaluate programming exercises, for example.

This research aims to advance the performance of GenAI-based systems for these tasks, by improving its alignment with human-produced examples. The chosen approach is prompt engineering and automatic prompt optimization. One key contribution of this approach is that it involves natural language prompts, which should lead to more interpretable results. In addition, this interpretability will facilitate modularity, which should lead to further improvements in efficiency and sustainability. Finally, this work is also expected to clarify the advantages and pitfalls of generative AI applied to college-level programming learning.

Keywords:

Learning, programming, GenAI, artificial intelligence, prompt optimization.