In December, we concluded this year’s Alumni Hungary webinar series with an outstanding talk by Dr. Zijian Győző Yang, a research fellow at the HUN-REN Hungarian Research Centre for Linguistics. He discussed the creation of large language models, such as ChatGPT, and shared insights into the development of PULI, the largest Hungarian language model. We also interviewed him about his research and perspectives on these groundbreaking technologies.
How did you become interested in the field of human language technology?
At first, I was interested in robotics, so I enrolled in the Faculty of Information Technology and Bionics at Pázmány Péter Catholic University. However, I quickly realized that I wasn't strong in the subject of physics, so I started looking for another specialization. By chance, I met Professor Gábor Prószéky, who became my mentor and introduced me to this field. Being multilingual and still interested in artificial intelligence, I quickly fell in love with language technology.
How does PULI differ from ChatGPT? For which tasks is PULI recommended?
The PULI models are developed specifically for Hungarian, so their strength lies in their knowledge of Hungarian. The PULI models are continuously evolving; keeping up with ChatGPT is very challenging, but we aim to provide a competitive alternative, especially for partners who are not permitted to use ChatGPT.
In your own work and life, which language models do you use? And for what type of tasks?
Since my work involves researching large language models, I always work with the most popular ones. Currently, I spend most of my time with the Llama models. For personal use, I only use ChatGPT to proofread and translate English texts.
How do you see the development of AI-driven large language models in the next few years? How will they change our lives?
I believe that artificial intelligence will become a part of our everyday lives, so we need to learn how to coexist with it, both in our daily tasks and in our work.
What are you currently working on in your research?
My main research focus is on creating a high-quality foundation models for Hungarian, so I continuously follow the latest released models and adapt them for Hungarian. Additionally, I would like to start working with multimodal models, aiming to train models capable of processing not only text but also audio, images, and video.