NativQA

Framework + benchmark for multilingual LLM evaluation

Build culturally aligned natural QA datasets grounded in native speakers and local context.

NativQA is a scalable, language-independent framework for constructing question answering datasets in native languages. It supports both evaluation and fine-tuning of large language models, with MultiNativQA as a public benchmark built from regionally grounded, native-speaker queries.

View framework on GitLab Install from PyPI Explore resources

Quick install pip install nativqa-framework

Explore resources

Framework

NativQA Framework

Use the framework to create culturally and regionally aligned QA datasets for multilingual LLM evaluation and tuning.

Framework resources

Dataset

MultiNativQA Dataset

See dataset links, download metrics, language coverage, and topic distribution in a dedicated resources page.

Dataset resources

Why NativQA

Native-speaker grounded

Queries are sourced from native speakers, making evaluation data closer to real local information needs.

Culturally aligned

The benchmark emphasizes region-specific and culturally situated questions that generic QA sets often miss.

Evaluation + tuning ready

The same framework supports both benchmarking open- and closed-source LLMs and creating fine-tuning data.

news

Nov 13, 2025	Fostering Native and Cultural Inclusivity in LLMs
Jan 23, 2025	Multilingual and Multimodal Cultural Inclusivity in LLMs
Nov 13, 2024	Fostering Native and Cultural Inclusivity in LLMs

latest posts

Jul 16, 2024	Just a moment...

publications

NativQA Framework: Enabling llms with native, local, and everyday knowledge

Firoj Alam, Md Arid Hasan, Sahinur Rahman Laskar , and 3 more authors

arXiv preprint arXiv:2504.05995, 2025

Bib

@article{alam2025nativqa,
  title = {{NativQA Framework:} Enabling llms with native, local, and everyday knowledge},
  author = {Alam, Firoj and Hasan, Md Arid and Laskar, Sahinur Rahman and Kutlu, Mucahid and Darwish, Kareem and Chowdhury, Shammur Absar},
  journal = {arXiv preprint arXiv:2504.05995},
  year = {2025},
}

NativQA: Multilingual Culturally-Aligned Natural Query for LLMs

Md. Arid Hasan, Maram Hasanain, Fatema Ahmad , and 6 more authors

In Findings of the Association for Computational Linguistics ACL 2025 , Jul 2025

Bib

@inproceedings{hasan-etal-2025-nativqa,
  title = {{NativQA:} Multilingual Culturally-Aligned Natural Query for {LLM}s},
  author = {Hasan, Md. Arid and Hasanain, Maram and Ahmad, Fatema and Laskar, Sahinur Rahman and Upadhyay, Sunaya and Sukhadia, Vrunda N and Kutlu, Mucahid and Chowdhury, Shammur Absar and Alam, Firoj},
  booktitle = {Findings of the Association for Computational Linguistics ACL 2025},
  month = jul,
  year = {2025},
  address = {Vienna, Austria},
  publisher = {Association for Computational Linguistics},
  url = {https://aclanthology.org/2025.findings-acl.770/},
  doi = {10.18653/v1/2025.findings-acl.770},
  pages = {14886--14909},
  isbn = {979-8-89176-256-5},
}