To Master AI, We Must First Master How To Evaluate AI.

Interview with Prof. Duong Nguyen Vu – Vice Provost, Chief Scientific Officer, Center for AI Research, VinUniversity.

Professor, when talking about AI, people often talk about increasingly powerful models. Why do you think benchmarks are equally important?

In any scientific field, to develop, you must first have reliable measurement methods.
The same applies to AI. We often focus on which model is more powerful, faster, or has more parameters. But without a suitable evaluation system, it’s difficult to know what a model actually does well, where its limitations lie, and whether it’s suitable for our needs.

In my opinion, benchmarks are not just tools for ranking AI models. More importantly, benchmarks help define the capabilities that AI should aim for. What is included in benchmarks will become the goals for the research community and developers to continue improving their models.
That’s why I believe that to master AI, we must first master how to measure AI.

What does that mean for Vietnam?

Most of the popular benchmarks currently in use are built in an international context and primarily assess universal capabilities.

Meanwhile, each country has its own unique characteristics in terms of language, culture, and application context.

An AI model might use Vietnamese quite well, but it may not fully understand the cultural nuances, forms of address, dialects, or specific requirements in fields like education or public health.

In addition, there are issues related to information security and digital sovereignty that need to be considered during the AI ​​evaluation process.

Therefore, the question is not just how powerful the AI ​​is, but how well the AI ​​understands Vietnam.

Is that why VinUniversity developed V-Bench?

Yes.

From the perspective of a research university, we aim to contribute a reference system to serve the AI ​​research and development community in Vietnam.

V-Bench was initiated by Center for AI Research, VinUniversity, with the entirely non-profit collaboration of a scientific council of 18 leading Vietnamese AI experts from both within and outside the country.
The benchmark is currently built on over 40,000 questions and tasks, aiming to more comprehensively assess the Vietnamese language proficiency of large language models.
The difference is that V-Bench not only assesses knowledge, but also includes five competency groups: implicit culture, regional diversity, digital security and sovereignty, practical application, and Agentic AI competency, including planning ability, tool usage, RAG, and autonomous action in the Vietnamese environment.

We hope that AI assessment will be closer to the real-world usage scenarios of Vietnamese people.

How is V-Bench different from many current international benchmarks?

We don’t aim to replace international benchmarks.

On the contrary, V-Bench is built using techniques similar and common in international LLM research, allowing developers to evaluate models on a reference system that reflects the Vietnamese context while still being able to connect with the global benchmark ecosystem.
What we want to add are aspects specific to Vietnam – factors that are difficult to fully reflect in global benchmarks.

That’s also why V-Bench aims not only to measure the ability to answer questions, but also to assess the level of understanding of language, culture, and the ability of AI to act in the Vietnamese context.

According to you, what role will benchmarks play as AI becomes increasingly widely applied?

AI is gradually becoming the infrastructure of many fields.

In this context, it’s not just about which model has the highest capabilities, but which model is best suited to each specific application.

A reliable reference system will help researchers, businesses, and organizations applying AI to better understand the strengths and limitations of each model in specific contexts.
This is also a condition for AI to be applied more effectively and responsibly.

In what direction will V-Bench continue to develop?

AI is developing very rapidly, so benchmarks must also be constantly updated.

In the future, V-Bench will continue to expand its evaluation criteria to include images and audio, including recognizing Nôm characters, regional signs, charts, cultural heritage, understanding accents from all three regions of Vietnam, understanding cultural context through short videos, as well as the ability to process very long Vietnamese documents such as laws, contracts, textbooks, or legal texts.

Our goal is to continue refining a valuable reference system for the Vietnamese AI research and development community.

If you were to convey a message about the future of AI in Vietnam, what would you say?
In AI, data is crucial. But data is only part of the story.
An AI system only truly creates value when we know how to objectively assess its capabilities in a way that is appropriate to the context of use.
If data is the foundation for building AI, then benchmarks are the measures that help us understand how AI is performing.