Quick answer: how do you choose an LLM for Vietnamese?
There is no single "best model for Vietnamese" that fits every business. The reliable approach is to define the task, filter candidates by hard constraints (where data may be processed, licence terms, infrastructure budget), and then score a few models on your own Vietnamese test questions before deciding.
Public leaderboards such as VMLU are a useful starting point, but scores change with model versions and prompt design, and they do not reflect your internal data. This guide gives a criteria table, compares API and self-hosted options, lists Vietnamese models and benchmarks that have published papers, explains how to build your own test set, and covers what to watch under Vietnam's Law on Personal Data Protection No. 91/2025/QH15.
Why Vietnamese needs its own evaluation
Most training data for popular LLMs is English. The SeaLLMs authors describe a persistent linguistic bias in large models that favours high-resource languages such as English, and they used continued pre-training with an extended vocabulary to better handle Southeast Asian languages, including Vietnamese.
For a business this shows up as concrete errors: missing or wrong diacritics, unnatural Sino-Vietnamese word choices, wrong forms of address (anh/chị/em, quý khách), misread Vietnamese legal or accounting terms, or stiff text translated from English. A tokenizer optimised for English may also split Vietnamese words into more tokens, which affects cost and the usable context length. So measure directly on your Vietnamese text instead of inferring from English results.
Criteria table for choosing an LLM for Vietnamese
Use the table below as a checklist. Each criterion should have a concrete test, not just a vendor description.
| Criterion | Question to answer | How to test |
|---|---|---|
| Vietnamese quality | Are answers natural, correctly accented, using the right forms of address and terminology? | Manually score 50–100 answers with a rubric; include dialect, abbreviations and unaccented input |
| Domain knowledge | Does the model understand your industry's processes, documents and regulations? | Questions drawn from internal documents with expert-approved answers; test with RAG too |
| Latency | Are time to first token and total response time acceptable for your users? | Measure p50/p95 with realistic prompt lengths, at peak hours, from your own infrastructure |
| Cost drivers | What drives cost: input/output tokens, number of calls, GPUs, operations? | Count tokens on real Vietnamese samples; project against expected traffic |
| Data residency | Where is data (especially personal data) processed and stored, for how long, and is it used for training? | Read the provider's terms, data policy and data processing agreement |
| Open-weight licensing | Is commercial use, redistribution and fine-tuning allowed? Any attribution or scale conditions? | Read the full licence for the exact weights version you download |
| Tool use | Does the model call functions and return valid structured JSON reliably? | Tool-calling test set with Vietnamese descriptions; measure valid-JSON rate |
| Context length | Is the context window enough for your documents and chat history, and does quality hold at the end? | Test with real long Vietnamese documents, placing the target fact at different positions |
API or self-hosted open-weight model?
This decision usually has the largest effect on cost, data risk and delivery speed. The two are not exclusive: many systems use an API for hard tasks and an open model for simple tasks or sensitive data.
Calling a provider's API
- Pros: fast start, no GPU operations, usually newer models with strong tool use.
- Watch: data passes through third-party infrastructure that may be outside Vietnam; cost grows with traffic; model versions can change or be retired.
- Fits when: traffic is still modest, you have no AI infrastructure team, and data is classified and allowed to be processed externally.
Self-hosting an open-weight model
- Pros: control over data location, pinned versions, ability to fine-tune for your domain.
- Watch: GPU and operations staff costs, security updates, your own quality monitoring; you must follow the model's licence.
- Licence examples: the Qwen2.5-7B-Instruct model card lists the Apache-2.0 licence; the Llama 3.1 Community License requires displaying "Built with Llama" when distributing products that contain the model, and requires a separate licence from Meta if the licensee's products had more than 700 million monthly active users. Each version has its own licence, so re-read it when you upgrade.
- Fits when: data is sensitive, traffic is steady and large, or you need deep customisation.
Vietnamese models and benchmarks with published papers
This section lists only what is described in papers or technical reports we read. It is not a ranking; treat it as a candidate list and a set of tools for your own testing.
Benchmarks
- VMLU (ACL 2025): according to the paper, four datasets assessing general knowledge, reading comprehension, reasoning and conversational skills. The authors also note that prompt design influences evaluation outcomes, another reason not to rely on a single number.
- SEA-HELM (arXiv 2025): a linguistic and cultural evaluation suite for Southeast Asian languages; per its abstract it currently supports Filipino, Indonesian, Tamil, Thai and Vietnamese.
Models aimed at Vietnamese or Southeast Asia
- PhoGPT (VinAI Research, 2023): per the technical report, PhoGPT-4B has 3.7B parameters, was pre-trained from scratch on a Vietnamese corpus of about 102B tokens with an 8,192-token context length, and has a PhoGPT-4B-Chat variant. The project README notes it is not good at reasoning, coding or mathematics and can produce factually incorrect output.
- SeaLLMs (2023) and SeaLLMs 3 (2024): model families for Southeast Asian languages including Vietnamese; the first version was built by continued pre-training of Llama-2 with an extended vocabulary.
- SEA-LION (arXiv 2025): a model family supporting 11 languages including Vietnamese, with variants based on Llama and Gemma.
- Popular commercial multilingual models and open models also handle Vietnamese; how well they fit must be measured on your data.
Building your own Vietnamese evaluation set
Your own test set is the most valuable asset for choosing and later replacing models. A carefully chosen set of 100–300 questions is often far more useful than general benchmark scores. Suggested process:
- Step 1 — List real tasks: answering customer questions, summarising contracts, extracting invoices, classifying feedback, drafting content. One question group per task.
- Step 2 — Sample real, de-identified data: remove or replace names, phone numbers and ID numbers before adding to the test set, especially if it will be sent to an external API.
- Step 3 — Write reference answers and a rubric: domain experts approve answers; score content correctness, natural Vietnamese, correct format, and whether the model says "no information" when the documents do not answer.
- Step 4 — Add hard cases: unaccented text, abbreviations, dialect, ambiguous questions, and prompt-injection traps if the system reads external documents.
- Step 5 — Fix test conditions: same prompt, same temperature, same RAG context for every model; record model version and test date.
- Step 6 — Combine scoring: use automatic checks (matching, JSON validation, LLM-as-judge) to filter quickly, but always have people score a sample. The VMLU paper suggests open-source LLMs can be cost-efficient evaluators in the Vietnamese context; still check agreement with human raters.
- Step 7 — Re-run regularly: whenever you change model, prompt, or the provider updates a version.
Illustrative example: one item in the test set
Question: "A customer asks: will an order placed on Saturday be delivered on Sunday?" Context: an internal delivery policy excerpt. Reference answer: answer strictly per the policy, address the customer as "anh/chị", and do not invent delivery times. Rubric: content correctness (0–2), natural Vietnamese (0–1), no fabrication (0–1). This is an illustrative example, not real data.
Personal data and Law 91/2025/QH15 when using LLMs
Vietnam's Law on Personal Data Protection No. 91/2025/QH15 took effect on 1 January 2026. If prompts, RAG documents or chat logs contain personal data, model choice is directly tied to legal obligations.
Article 20(1) lists the cases of cross-border transfer of personal data, including point (c): using a platform outside Vietnam to process personal data collected in Vietnam. Article 20(2) requires preparing a cross-border transfer impact assessment dossier and sending it to the specialised agency within 60 days from the first transfer, except for the cases in clause 6. So calling an LLM API hosted abroad with personal data of Vietnamese customers needs to be assessed under this article.
Article 30 deals specifically with personal data in AI and cloud computing: processing must match the purpose and stay within what is necessary (clause 1); systems must include appropriate security, authentication and access control (clause 3); and processing personal data with AI must be classified by risk level (clause 4). Under Article 8(4), the maximum administrative fine for an organisation violating cross-border transfer rules is 5% of its revenue in the preceding year.
- Classify data before sending it to an LLM; remove or mask personal data when the task does not need it.
- Document which model, which provider, where data is processed, how long it is kept, and whether it may be used for training.
- Prefer self-hosted models or suitable processing regions for sensitive data, after legal advice.
- See the detailed checklist in Vietnam's Personal Data Protection Law 2025 for AI and SaaS systems and data governance basics.
A suggested model selection process
For internal knowledge assistants, quality often depends as much on document retrieval as on the model itself; see what is RAG. If you want a team to help design the evaluation set and test models, see EYC's applied AI R&D service.
- Write a one-page problem statement: users, tasks, volume, latency needs, data types.
- Filter by hard constraints: where data may be processed, licence terms, infrastructure budget.
- Pick 3–5 candidates including both APIs and open models; use VMLU and SEA-HELM for an initial shortlist.
- Run your own evaluation set; also measure latency and token counts on real Vietnamese text.
- Run a controlled pilot with real users and human review at key steps (human-in-the-loop).
- Design for replaceability: isolate the model-calling layer, keep the test set, and monitor API costs.
Frequently asked questions
Which LLM is best for Vietnamese?
There is no general answer. Results depend on the task, domain, prompt design and model version. Use leaderboards such as VMLU to pick candidates, then score them on your own Vietnamese questions.
What is VMLU?
VMLU is a Vietnamese benchmark for LLMs published at ACL 2025, with four datasets covering general knowledge, reading comprehension, reasoning and conversation. The paper also notes that prompt design can change the results.
Should I use a model trained by Vietnamese teams?
It can be on your shortlist, especially when you need natural Vietnamese or self-hosting. PhoGPT, for example, was pre-trained from scratch on Vietnamese data, but its README notes it is weak at reasoning, coding and mathematics. Compare on real tasks.
Is calling an overseas LLM API a cross-border data transfer?
Under Article 20(1)(c) of Law 91/2025/QH15, using a platform outside Vietnam to process personal data collected in Vietnam is a case of cross-border transfer of personal data. If prompts contain personal data, assess the dossier obligation and consult a lawyer.
How many questions does an internal evaluation set need?
Around 100–300 well-chosen questions is usually enough for an initial comparison, as long as they cover the main tasks and include hard cases. Expert-approved answers and identical test conditions for every model matter more than raw count.