MultiBLiMP 2.0 - Subject-Auxiliary Gender Agreement Overview

Subject-Auxiliary Gender agreement prediction through decision trees. Click a language row (or the Results button) to open its create_pairs results below. The small chevron after N Keep (or the Diagnostics button) is separate: it opens just the bucket breakdown, inline, right where those items dropped out of the pipeline.
152 languages omitted (too few or uninformative labels) — click to expand

The following languages were omitted for having too few or uninformative agreement labels (label distribution shown):

1 language with too sparse a joint-tagging rate to be confident — click to expand

The following languages have enough raw "Yes" samples (≥ 10) but too sparse a joint-tagging rate relative to the full dataset to be statistically confident this isn't incidental tagging (Wilson lower bound below 1%; label distribution shown):

  • Latvian (unk: 3371; yes: 34; no: 1)
14 languages with no data at all for this condition — click to expand

The following languages have NO data at all for this condition (zero raw instances extracted -- no tree, pairs, or page was ever produced for them). raw instances existed (head upos in ['AUX']; child upos filter (['NOUN', 'PROPN', 'PRON'])), but every row's head_nsubj_Gender_agreement came back null after _prepare_full_df (or rm_columns dropped every row) -- the agreement column itself never got computed for any of them.

  • Bororo
  • Mbya Guarani

The following languages have NO data at all for this condition (zero raw instances extracted -- no tree, pairs, or page was ever produced for them). 0 raw instances of child_deprels=['nsubj'] (head upos in ['AUX']; child upos filter (['NOUN', 'PROPN', 'PRON'])) -- likely this treebank has no attested instance matching every one of these filters (e.g. a required feature like VerbForm might simply never be annotated here at all, rather than genuinely disagreeing).

  • Abaza
  • Akkadian
  • Cebuano
  • Gorontalo
  • Gothic
  • Guajajara
  • Gwichin
  • Middle Armenian
  • Southern Kurdish
  • Telugu
  • Tupinamba
  • Yupik
Keep threshold: leaf accuracy > 0.95
Coverage 0%100%
Item loss 0%100%
Language
Distribution
Root Acc
Train Acc
Δ Acc
Test Acc
N RAW
N KEEP
N PAIRS