D
Data
Text
Native Arabic STEM Multiple-Choice Benchmark for Evaluating Language Models
AraSTEM is a benchmark of 11,637 native (non-translated) Arabic multiple-choice questions spanning math, science, physics, biology, chemistry, computer science and medicine from elementary to college level, each carrying a cited source for traceability. It is used to evaluate large language models' knowledge and reasoning in scientific Arabic under zero-shot prompting, and shows that current open-source models of varying sizes struggle, underscoring the need for more localized Arabic models.