NativMMQA

Framework + datasets + benchmarks

Build and evaluate culturally grounded QA resources across languages and modalities.

NativMMQA brings together the NativQA Framework and public benchmark resources for culturally aligned evaluation. Our work currently spans MultiNativQA for multilingual natural QA and EverydayMMQA/OASIS for culturally grounded spoken visual QA, giving one entry point for building, benchmarking, and extending datasets for LLM evaluation and fine-tuning.

Explore resources View framework on GitLab Install from PyPI

Explore the ecosystem

Framework

NativQA Framework

Use the framework to create culturally and regionally aligned QA datasets for multilingual and multimodal model evaluation and tuning.

Framework resources

Dataset

MultiNativQA

See links, live download metrics, language coverage, regional scope, and topic distribution for the multilingual natural QA benchmark.

Dataset resources

Spoken visual QA

EverydayMMQA / OASIS

Explore the multilingual and multimodal framework for culturally grounded spoken visual QA, with paper context, dataset scale, and benchmark takeaways.

Open resource page

Why this work matters

Native-speaker grounded

Queries are sourced from native speakers and local contexts, making the data closer to real information needs than generic prompt collections.

Multilingual and multimodal

The site now covers both natural QA and spoken visual QA, making the resource collection broader than a single benchmark page.

Reusable across workflows

The same framework and resource family support benchmarking, analysis, and the creation of fine-tuning data for culturally aligned systems.

news

Nov 13, 2025	Fostering Native and Cultural Inclusivity in LLMs
Jan 23, 2025	Multilingual and Multimodal Cultural Inclusivity in LLMs
Nov 13, 2024	Fostering Native and Cultural Inclusivity in LLMs

latest posts

Jul 16, 2024	Arabic Language Technologies – Medium

publications

EverydayMMQA: A Multilingual and Multimodal Framework for Culturally Grounded Spoken Visual QA

Firoj Alam, Ali Ezzat Shahroor, Md. Arid Hasan , and 8 more authors

arXiv preprint arXiv:2510.06371, 2025

Bib

@article{alam2025everydaymmqa,
  title = {{EverydayMMQA:} A Multilingual and Multimodal Framework for Culturally Grounded Spoken Visual QA},
  author = {Alam, Firoj and Shahroor, Ali Ezzat and Hasan, Md. Arid and Ali, Zien Sheikh and Bhatti, Hunzalah Hassan and Kmainasi, Mohamed Bayan and Chowdhury, Shammur Absar and Mousi, Basel and Dalvi, Fahim and Durrani, Nadir and Milic-Frayling, Natasa},
  journal = {arXiv preprint arXiv:2510.06371},
  year = {2025},
}

NativQA Framework: Enabling llms with native, local, and everyday knowledge

Firoj Alam, Md Arid Hasan, Sahinur Rahman Laskar , and 3 more authors

arXiv preprint arXiv:2504.05995, 2025

Bib

@article{alam2025nativqa,
  title = {{NativQA Framework:} Enabling llms with native, local, and everyday knowledge},
  author = {Alam, Firoj and Hasan, Md Arid and Laskar, Sahinur Rahman and Kutlu, Mucahid and Darwish, Kareem and Chowdhury, Shammur Absar},
  journal = {arXiv preprint arXiv:2504.05995},
  year = {2025},
}

NativQA: Multilingual Culturally-Aligned Natural Query for LLMs

Md. Arid Hasan, Maram Hasanain, Fatema Ahmad , and 6 more authors

In Findings of the Association for Computational Linguistics ACL 2025 , Jul 2025

Bib

@inproceedings{hasan-etal-2025-nativqa,
  title = {{NativQA:} Multilingual Culturally-Aligned Natural Query for {LLM}s},
  author = {Hasan, Md. Arid and Hasanain, Maram and Ahmad, Fatema and Laskar, Sahinur Rahman and Upadhyay, Sunaya and Sukhadia, Vrunda N and Kutlu, Mucahid and Chowdhury, Shammur Absar and Alam, Firoj},
  booktitle = {Findings of the Association for Computational Linguistics ACL 2025},
  month = jul,
  year = {2025},
  address = {Vienna, Austria},
  publisher = {Association for Computational Linguistics},
  url = {https://aclanthology.org/2025.findings-acl.770/},
  doi = {10.18653/v1/2025.findings-acl.770},
  pages = {14886--14909},
  isbn = {979-8-89176-256-5},
}

SpokenNativQA: Multilingual Everyday Spoken Queries for LLMs

Firoj Alam, Md Arid Hasan, and Shammur Absar Chowdhury

In Proceedings of the 26th Interspeech Conference (Interspeech 2025) , Aug 2025

Bib

@inproceedings{alam2025spokennativqa,
  title = {{SpokenNativQA:} Multilingual Everyday Spoken Queries for LLMs},
  author = {Alam, Firoj and Hasan, Md Arid and Chowdhury, Shammur Absar},
  booktitle = {Proceedings of the 26th Interspeech Conference (Interspeech 2025)},
  year = {2025},
  address = {Rotterdam, The Netherlands},
  month = aug,
  organization = {ISCA},
}