MultiBLiMP 2.0 - Morphological Agreement

How many candidate samples were considered and how many minimal pairs were actually generated from them — broken down per language, per grammatical condition (subject agreement with the verb/participle/auxiliary, and noun-phrase agreement across its role pairs), and per language×condition. "Samples" are the raw UD-derived candidate instances a condition considered; "minimal pairs" are the rows that came out the other end as usable contrastive pairs — one sample can yield more than one pair when several inflected forms are valid contrasts, so pairs isn't a subset of samples and the ratio between them can exceed 1×.

Generated 2026-10-06 10:55 UTC

Subject
Object
Indirect Object
142
Languages covered
45
Conditions tracked
3,568,040
Candidate samples
4,420,945
Minimal pairs generated
1.24×
Pairs per sample, overall
46 languages tried, zero samples anywhere

Abaza, Afrikaans, Alemannic, Assamese, Bavarian, Bokota, Cantonese, Cebuano, Chinese, Chukchi, Classical Chinese, Coptic, Gorontalo, Gujarati, Gwichin, Haitian Creole, Ika, Japanese, Kadiweu, Kinyarwanda, Latgalian, Luxembourgish, Madi, Makurap, Malayalam, Mbya Guarani, Munduruku, Neapolitan, Nenets, Northwest Gbaya, Old English, Old Turkish, Paumari, Pesh, Runyankore, Shanghainese, Sinhala, South Levantine Arabic, Swahili, Tagalog, Telugu, Tswana, Vietnamese, Western Sierra Puebla Nahuatl, Yiddish, isiXhosa

Samples & pairs per language

Totals summed across every condition a language appears in. The chart shows the top 25 languages by minimal pairs generated; the table below covers all 142.

Language × condition matrix

Minimal pairs generated for every language/condition combination. Sort by total pairs or alphabetically; hover a cell for exact samples, pairs, ratio, and decision-tree accuracy, or click it to open that language/condition's own diagnostics page. Blank cells are combinations that were never run, not zero-pair combinations.

142 languages × 45 conditions.

View as table

Samples vs. minimal pairs, per language

Each dot is one language. The dashed line marks where pairs equal samples (1×) — languages well above it produced unusually many contrastive pairs per candidate; languages near the bottom found samples but little usable contrast. Both axes are log-scaled given the range from a handful of samples to hundreds of thousands; languages with zero pairs are nudged just above zero so they stay visible rather than dropping off a log axis. Total sums each language across every condition it appears in (dots aren't clickable here -- a language's condition breakdown lives behind its own name in the table below); open see conditions and pick a single one instead to see that condition alone, with dots linking to each language's own diagnostics page.

Where samples don't become pairs

Aggregated outcome buckets across every language/condition (from each run's own diagnostic buckets). Not a strict partition of the sample count above — one sample can land in more than one outcome across its accepted swap candidates — but it shows where the corpus loses candidates: no matching swap candidate, an inflector that couldn't produce a contrastive form, a reinflection that came out identical, or a genuinely ambiguous subject.

View as table
OutcomeInstances

Samples by treebank

Which treebank each language's raw candidate samples actually came from, for every minimal pair those samples could produce — not an estimate, real per-row counts straight from this repo's own decision-tree cache. Coverage depends on how far each condition's own fitting run has gotten (currently 1698 language × condition combinations across 149 languages, out of this page's full 142), so it can be a subset of the rest of this page rather than a full match. Samples, not pairs, by treebank: nothing here flags which individual rows became accepted pairs, so a treebank's share of samples is used as the closest honest proxy for its share of that combination's pairs.

Treebanks per language

Every UD treebank this pipeline is eligible to draw samples from for each attempted language (some languages restrict to a specific subset; most use every treebank the UD release has for them) — an inventory of possible sources, not a per-sample or per-pair count by treebank. Includes the 46 zero-yield languages above, since which treebank(s) a language drew from can be part of why it came back empty. For the finer, per-sample breakdown, see above.

Treebanks