Balsam Index

The Balsam Index is an evaluation and measurement benchmark for Arabic models in the field of data and artificial intelligence (AI). It was launched by the Saudi Data and AI Authority (SDAIA) and the King Salman Global Academy for Arabic during the third edition of the Global AI Summit on September 12, 2024, at the King Abdulaziz International Conference Center in Riyadh City, the Kingdom of Saudi Arabia.

The Balsam Index was launched as part of SDAIA's efforts, in collaboration with its strategic partners, to advance Arabic language models, a rapidly evolving field. The Balsam Index contributes by evaluating new models that are continually developed to incorporate advanced features and capabilities.

The Balsam Index is part of the initiatives of the AI Center for Arabic Language Processing affiliated with the King Salman Global Academy for Arabic. This center offers a range of integrated, free services to empower researchers and developers in employing AI techniques for automated Arabic language processing, in addition to building tools and programs that ensure the preservation of the Arabic language.

The Balsam Index contributes to evaluating AI technologies for the Arabic language to support research collaboration and to establish global standards for assessing the maturity of AI models in Arabic language tasks. This aligns with the strategic objectives of the King Salman Global Academy for Arabic.

The Balsam Index aims to organize datasets by pooling expertise and resources to create high-quality datasets across various levels of Arabic and diverse fields, specifically designed for AI testing. This supports the robustness and diversity of large language models (LLMs). It also seeks to standardize evaluation metrics to assess the performance of LLMs developed by contributors, providing clear comparisons and supporting continuous improvements.

The Balsam Index also aims to present evaluation results for LLMs in task performance and Arabic natural language processing, working to unify the perspectives of research communities in Arabic natural language processing. This includes building shared datasets and unified evaluation standards, along with prioritizing ethical considerations and responsible AI practices during development to ensure fairness and transparency.

The Balsam Index includes approximately 1,400 datasets, comprising fifty thousand questions and covering sixty-seven diverse tasks, such as grammar and spelling correction, paraphrasing, cause-and-effect classification, and text comprehension. Companies, researchers, and developers of large language models can use it to measure the performance of their models, with the ability to compare their models against others. This aligns with the objectives of the National Strategy for Data and AI, supporting Saudi Vision 2030’s goal of positioning the Kingdom as a global hub for advanced technologies related to AI.