After 5,000 Assignments: AI Grades Student Work at ITMO University
The experiment revealed something important: smart algorithms can do the grading, but they cannot do the thinking for students.
A neural network graded more than 5,000 test submissions in a machine learning course at ITMO University. Students were allowed to use AI, but instructors designed the questions to keep the algorithm from simply supplying ready-made answers. The results were revealing: even with AI assistance, students answered only 41% of the most challenging questions correctly.
Everyone Is Responsible for Their Own Work
ITMO conducted the experiment to address a practical challenge. Instructors assigned routine test grading to a neural network to reduce teaching assistants’ workload and get results back to students faster. At the same time, they deliberately made the assignments more challenging, wording questions so that students could not simply copy an answer from a language model and expect to succeed.
The experiment showed that students still need to understand the subject matter for themselves, even when AI tools are available. A neural network can help instructors grade assignments and students prepare answers, but each student must ultimately demonstrate the ability to think independently.
Why Context Matters
Russia has made rapid progress in developing automated grading systems. ITMO was not the first institution to explore the technology, but it was among those helping move the field forward. In December 2024, Yandex and the Moscow Institute of Physics and Technology (MIPT) used YandexGPT to grade 35,000 submissions to the Lomonosov Tournament, a physics, chemistry and astronomy competition. The process took just one month instead of three, and grading accuracy reached nearly 95%. Human experts, however, still had the final say.
Other competitions have revealed some of the nuances of AI-based assessment. At the International Economics Olympiad (IEO), for example, two models graded participants’ submissions, with a third brought in whenever their scores differed. Danil Fyodorovykh, deputy vice rector at HSE University who took part in the project, said AI graded more consistently than human evaluators but repeatedly made mistakes on one particular problem. Without strict constraints, he explained, a model tends to side with the participant in disputed cases, potentially awarding an inflated score. Anna Korchak, a researcher at HSE University, offered a different perspective: neural networks resemble vast digital libraries, but without context, they cannot tell what a person has already learned and what they still need to master.
As early as 2025, well before the current experiment, ITMO provided a neural network to process Russia’s nationwide assessment tests, known as Vserossiyskie Proverochnye Raboty (All-Russian Assessment Tests). The system accelerated the process 20-fold. This year, meanwhile, master’s students developed Avito AI Reviewer, a tool for the initial review of homework assignments, as well as TuneAI, a platform that lets instructors tailor grading criteria to a specific course.
A Nod to the Teacher
Students who use AI save time. For instructors, however, the bigger benefit is not simply fewer hours spent grading but the opportunity to redirect their attention. Egor Spirin, head of ITMO’s Applied Agents Laboratory, sees this as a chance to bring back oral discussions and project defenses, where graduates must demonstrate that they genuinely understand their work rather than merely know how to generate text. Ilya Dementyev, rector of Gazprom Neft’s Corporate University, puts it even more directly: digital avatars cannot yet replace a human teacher. An educator’s primary role, he says, is to help students organize information and learn to analyze it, something AI still cannot do.
Viktor Sadovnichy, rector of Lomonosov Moscow State University (MSU), also reflected on the role of teachers in a recent interview. He recalled: “Back when I was still a schoolboy, every housewife would bow in greeting when a teacher walked down the street. It was teachers’ high level of professionalism and authority that enabled our country to recover so quickly after the war. Today, too, professionalism and deep knowledge of one’s subject remain essential requirements for teachers.”
AI Grades the Work, but Who Grades the AI?
For now, widespread use of AI to assess student work faces several challenges. Grading must be not only accurate but also understandable; otherwise, students will not know why they received a particular score. Complex, unconventional and creative responses still do not fit easily into predefined assessment frameworks. Other vulnerabilities include protecting personal data, integrating AI tools with university platforms and covering the cost of computing resources. There is also a risk that students will be judged not on what they know, but on how effectively they communicate with a neural network.
Over the next few years, AI assistants are likely to become part of universities’ digital environments, while responsibilities are redistributed between people and algorithms. AI will take over repetitive tasks, leaving educators free to focus on complex cases and individualized support. The groundwork for this shift is already in place: the government has approved a strategic framework for the digital transformation of science and higher education through 2030.
