Diagnose hidden degradation
Identify whether apparent multilingual failure comes from comprehension, instruction adherence, output-language control or structured generation.
VinUniversity Seed Fund Research Project
PolyDrift
Instruction-Language Drift in Adapted Multilingual Instruction-Tuned LLMs
Fine-tuning a multilingual model to improve one task can silently change how it understands, follows and produces language elsewhere. PolyDrift studies where that drift appears, why it happens, and how it can be reduced.
About PolyDrift
What changes when a multilingual LLM is adapted in one language?
PolyDrift treats multilingual capability as something that can change after post-adaptation. Instead of relying on a single performance score, the project diagnoses which part of instruction-following behaviour changes after fine-tuning.
Instruction-language drift
A post-adaptation change in the model's ability to understand target-language input, follow instructions in the intended language, control the language of its output, or preserve required structured responses such as labels and JSON.
Support multilingual reliability
Examine whether adaptation gains in English come with reduced reliability in Vietnamese or Arabic, especially in real-world multilingual use.
Build trustworthy adaptation
Test lightweight stabilisation strategies that reduce drift while retaining the benefits of task adaptation.
Research objectives
Four questions drive the project
The project moves from measuring drift, to locating the failure, explaining the internal change, and testing practical mitigation.
O1
Measure instruction-language drift after adaptation
Quantify pre- and post-adaptation behaviour across English, Vietnamese and Arabic, with controlled changes to input, instruction and expected output language.
O2
Identify which component of instruction-following breaks
Separate failures in input understanding, instruction-language following, target-language output and structured response control rather than collapsing them into one score.
O3
Analyse the internal changes behind observed failures
Connect behavioural drift with hidden-state movement, language-identity encoding and changes in cross-lingual alignment before and after adaptation.
O4
Test lightweight mitigation strategies
Evaluate multilingual rehearsal, balanced multilingual instructions and representation anchoring as ways to reduce drift without sacrificing target-task gains.
Diagnostic framework
Baseline → Adapt → Evaluate → Analyse Drift → Mitigate
A controlled workflow isolates behavioural and representation-level changes rather than treating multilingual performance as a static property.
BaselineMeasure multilingual instruction-following before adaptation
AdaptApply controlled parameter-efficient task adaptation
EvaluateVary input, instruction and output language systematically
AnalyseConnect behavioural failures with internal representation change
MitigateTest practical strategies for multilingual stability
Input Understanding
Can the adapted model still understand target-language input when the instruction and expected label remain controlled?
Instruction Following
Can it follow Vietnamese or Arabic instructions reliably after adaptation, independent of output-language demands?
Output Control
Can it answer in the requested language, or does it drift back towards English or code-switch unexpectedly?
Structured Control
Can it preserve the required label, schema or JSON-style response format across multilingual prompt conditions?
Languages and methods
A compact multilingual setting with deep diagnostic control
English, Vietnamese and Arabic provide a meaningful test bed for studying adaptation across resource levels, scripts, morphology and multilingual representation.
EN
English
The high-resource adaptation language and primary reference point for measuring gains and post-adaptation change.
VI
Vietnamese
Directly relevant to Vietnam and VinUniversity, and underrepresented in many multilingual LLM evaluation settings.
AR
Arabic
A typologically and orthographically distinct language with rich morphology, regional variation and broad multilingual NLP relevance.
Experimental approach
- Open-weight multilingual models: the core experiments use Qwen2.5-7B-Instruct, with Aya-23-8B for selected comparative analysis.
- Efficient adaptation: QLoRA is the primary fine-tuning method, with LoRA used for selected comparisons.
- Multilingual evaluation resources: established resources such as FLORES-200 and TyDi QA are combined with an audited diagnostic prompt suite.
- Prompt control: prompt length, instruction complexity, label wording and translation quality are controlled and manually checked where appropriate.
Evaluation and analysis
- Behavioural metrics: task performance, instruction adherence, output-language accuracy, code-switching and structured-response validity.
- Human verification: targeted manual assessment complements deterministic checks, with agreement measured when model-based judging is used.
- Representation analysis: RSA and CKA form the core hidden-state comparison methods, with deeper probing used selectively.
- Confound control: ablation studies separate genuine instruction-language drift from tokenisation, prompt verbosity and translation artefacts.
Project deliverables
Reproducible tools, diagnostics and practical guidance
PolyDrift is designed to leave behind reusable research infrastructure for analysing multilingual model adaptation beyond this project.
Audited multilingual diagnostic suite
A controlled English–Vietnamese–Arabic instruction-language suite for testing input understanding, instruction following, output control and structured responses.
Reproducible evaluation harness
A baseline and post-adaptation evaluation pipeline with consistent prompt configurations, metrics, validation checks and experiment tracking.
Adapted model artefacts and drift logbook
Controlled model adapters or checkpoints where licences permit, together with systematic records of how behaviour changes across adaptation conditions.
Instruction-language drift toolkit
Tools for task metrics, structured-output checks, language control analysis and representation-level comparison before and after adaptation.
Mitigation and stability framework
A comparative assessment of multilingual rehearsal, balanced instructions and representation anchoring for reducing drift while preserving adaptation gains.
Open documentation and reusable resources
Public code where licences allow, benchmark and governance documentation, prompt templates, evaluation scripts and practical guidance for reliable multilingual adaptation.
Research plan
Four phases from baseline construction to toolkit release
The work plan keeps the core experiment focused while allowing selected extensions where data and computational resources permit.
Months 1–4
Infrastructure, diagnostic benchmark and baseline
Project setup, data governance, multilingual prompt construction, baseline evaluation and hidden-state activation caching.
Months 5–15
Controlled post-adaptation and drift analysis
Parameter-efficient adaptation, diagnostic evaluation across languages, structured output analysis and representation comparison.
Months 16–19
Mitigation strategies and stability evaluation
Multilingual rehearsal, balanced instruction prompts and representation anchoring, followed by comparative drift-reduction analysis.
Months 20–24
Toolkit refinement, documentation and dissemination
Software packaging, reproducible pipeline documentation, resource release where licences allow, research communication and future programme development.
Research team
Multilingual NLP expertise across Vietnam, the UK and Australia
PolyDrift brings together researchers in multilingual NLP, large language models, language resources, model adaptation and trustworthy AI.
Mo El-Haj
Principal Investigator
VinUniversity
Vietnam
Vietnam
Lancaster University
United Kingdom
United Kingdom
Paul Rayson
Co-Investigator
Lancaster University
United Kingdom
United Kingdom
VinUniversity
Vietnam
Vietnam
Wray Buntine
Co-Investigator
Monash University
Australia
Australia
VinUniversity
Vietnam
Vietnam
Project collaboration
VinUniversityLancaster UniversityMonash University
Expected impact
Towards more reliable, transparent and trustworthy multilingual LLM adaptation
The scientific contribution is a shift from asking whether multilingual performance drops to identifying precisely what changes after adaptation. This supports safer deployment in settings where multilingual systems are used for education, public communication, moderation and institutional decision support.
ScientificA diagnostic account of post-adaptation multilingual behaviour and its representation-level causes.
Vietnam and multilingual AIStronger evaluation of Vietnamese alongside Arabic and English, with attention to underrepresented multilingual settings.
Reusable practiceOpen tools, documentation and mitigation guidance that can support future trustworthy multilingual AI research.
Contact
Interested in PolyDrift?
For research collaboration, technical discussion or information about the project, contact the PolyDrift Principal Investigator at VinUniversity.