Manipulation of LLMs? A solution is being sought through a collaboration between Jakub Reš, a doctoral student at FIT BUT, and Red Hat

Large Language Models (LLMs) have been among the most frequently discussed topics in the field of artificial intelligence in recent years. Hand in hand with their widespread practical use, the frequency and severity of attempts to manipulate or misuse them are increasing, and with that, the importance of cybersecurity protection.

It was precisely in this field that an interesting convergence of doctoral research from FIT VUT and corporate practice emerged last year. Moreover, this collaboration took place with a world-leading developer of open-source software solutions for businesses. Jakub Reš, a doctoral student working under the supervision of Assoc. Prof. Kamil Malinka and a member of the Security@FIT research group, began a research internship at Red Hat. The collaboration focuses on ensuring the integrity and security of language models. Specifically, Jakub is working on developing software methods for preventing or detecting covert manipulation of AI models. Interference with the functioning of LLMs can take several forms: “Let me give an example: a chatbot on a gardening website that recommends competitors’ products or links to inappropriate items. However, there are also cases of abuse that could lead to massive political influence over users or even the generation of dangerous code. And erroneous advice regarding our health can also be a very serious manifestation of hidden model modifications,” Jakub Reš explains the potential risks.

How did Reš’s journey to becoming a major player in software development begin? And what results has this collaboration already yielded? You can read about it here.