Automating Microeconomics II Tutorials

Description

Problem

A typical tutorial/exercise class in most microeconomic lectures looks as follows: the lecturer stands before the students and solves one exercise after another. Students copy the solutions and rarely ask any questions. This type of teaching is ineffective and provides little added value over simply handing out typed exercise solutions to the students. Motivating students to work on the exercises before the class and continuously engage with the material throughout the semester could bring multiple benefits. The students could use the tutorial time to ask questions and to discuss those parts of the exercises that were difficult or unclear with the lecturer. The students could notice if they are falling behind and need to catch up on the material before it is too late. Even better, the students might be motivated to think beyond the material strictly necessary for the lecture. One way to provide this motivation is to use graded problem sets. However, this would be difficult to implement with traditional teaching methods for several reasons. First, manually grading problem sets would drastically increase the workload of the already overextended assistants. Second, it would be too easy and too tempting for students to copy the solutions from their colleagues (or from previous years), thereby undermining the entire purpose of grading the problem sets.

Solution

This project implemented a solution to these two problems in the context of Microeconomics II (a large, mandatory lecture in the Master of Science in Economics program). This project's main goal was to develop an automated framework in Python that can generate individualized exercises for the problem sets by randomly varying the parameters of the exercises. Moreover, the framework also calculates the correct answers for each exercise version. Once the exercises and solutions are generated in this way, it is relatively easy to upload them to ILIAS as tests that students take throughout the semester. ILIAS has the functionality to randomly assign a problem version to a student and then to automatically grade the student's solutions. 

The evaluation of this FIL-project shows that the students were very satisfied with the automated problem sets and recommend that we continue using and developing them. The feedback from the teaching assistant has been very positive as well. Overall, this approach has succeeded in motivating a significant proportion of students to work on assigned problem sets continuously throughout the semester. The main problem with this approach is that some types of exercises (mainly quantitative exercises) are better suited than others (for example, more conceptual tasks like proving propositions). In the current version of the automated problem sets we have also included those conceptual exercises. Student feedback and TA experience have shown that these exercises are not suitable for the purpose, and they will be removed in future versions of the automated problem sets. The benefits of this FIL-project are that there is now a proven concept of how to successfully create individualized quantitative exercises and integrate them into ILIAS. This can be further used in Microeconomics II, but it can also be adapted to other classes using problem sets with quantitative exercises. It can also be implemented as a tool to help students study and prepare for the exam. They could solve a variety of exercises on their own and then receive immediate feedback.

Outlook

ChatGPT and similar AI tools pose a novel challenge to this approach. The issue is that the students could copy the text of the exercises and potentially receive correct answers from the AI. This would negate the motivating aspect of this approach. Whether this is a real problem or not is something that needs to be further studied.