KiLM is a Large Language Model (LLM) developed by the Zalo engineering team, officially launched to the public at the end of 2023, based on a from-scratch training technique. The Aspiration to Own the Technology In the context of the global AI wave, most current models are trained in two ways:

• Fine-tuned models: Optimizing an existing LLM for specific purposes.
• From-scratch models: Building everything from the initial parameters to the model architecture and training algorithms.

If fine-tuning is considered the easier and faster approach, Zalo chose the more challenging path: training from scratch. With this approach, every part of the process is controlled by Vietnamese engineers, making Vietnam one of the few countries in Southeast Asia to possess its own locally developed LLM. 3 unique big challenges The journey was far from easy. The team faced three major obstacles simultaneously: computational infrastructure, training data, and human resources. While major global companies operate thousands of high-performance GPUs, Zalo engineers in Vietnam initially had only consumer-grade GPUs to learn and experiment with. Even after ordering eight DGX H100 servers from Nvidia, the scarcity of AI chips meant they could not be delivered immediately but had to arrive in small, staggered batches. On the data side, Vietnamese is a low-resource language, with datasets far smaller and less diverse than those available for English or Chinese. The challenge for Zalo was not only to train a model but also to build and curate high-quality Vietnamese datasets to overcome this limitation. Flexibility Under Difficult Conditions Instead of waiting, the Zalo team utilized every small GPU available to hone their knowledge and processes. When the larger infrastructure gradually became ready, the team had sufficient capacity to accelerate. Optimizing training on incomplete infrastructure, combined with the investment in Vietnamese data, became the "secret" that helped Zalo overcome the difficult period. The results were truly surprising: after only 6 months of training, Zalo's 7-billion-parameter model focused on Vietnamese achieved 150% the performance of OpenAI's GPT-3.5 on the VMLU benchmark while the initial plan was 18 months. Surprising Results In 2024, Zalo released its 13-billion-parameter model. On VMLU, this model outperformed GPT-4 (OpenAI), gemma-2-9b-it (Google), and Phi-3 (Microsoft), ranking just below Llama-3-70B (Meta). This is evidence that the technical capability of Vietnamese engineers is fully comparable to global standards. From Research to Real-Life Application The training achievement is only the beginning. Zalo quickly integrated the LLM into practical products like AI Greeting Cards (Thiệp AI) and Q&A Assistants. Applications based on KiLM continue to expand. Starting as a project with many limitations, Zalo has demonstrated that Vietnamese people can fully master AI technology, creating real value for the community.