-(CoderPPL)
54 models
CoderPPL — a code perplexity (PPL) benchmark measuring how well a language model predicts real-world, hand-written code across 5 languages (Python, JavaScript, HTML, Lua, Shell) and diverse programming domains. Unlike natural-language PPL, code PPL tests syntax understanding, structural reasoning, and domain-specific patterns.
Top 5 Models Performance
| Qwen: Qwen3.5 9B | ################################## | -1.6914 |
| Qwen: qwen3.5-9b-base | ################################### | -1.7468 |
| Qwen: Qwen3.5 4B | #################################### | -1.7894 |
| Yandex: YandexGPT Lite 5 | ######################################## | -2 |
| Qwen: Qwen3.5 2B | ######################################## | -2.0063 |