MultiBLiMP v2 - Language Overview

NP HEAD–DET (Case) agreement prediction through decision trees. Click a language row (or the Results button) to open its create_pairs results below. The small chevron after N Keep (or the Diagnostics button) is separate: it opens just the bucket breakdown, inline, right where those items dropped out of the pipeline.

Overview Entropy Type:

The following languages were omitted for having too few or uninformative agreement labels (label distribution shown): Beja (unk: 347; yes: 20), Georgian (98), Old Irish (24), Phrygian (95).

Keep threshold: entropy < 0.1

Coverage 0%100%

Item loss 0%100%

Language

Distribution

Base Entropy

Reduced Entropy

Δ Entropy

DT Acc%

N RAW

N KEEP

N PAIRS

Table columns

Distribution: The language's full Yes / No / unk label mix, before the decision tree drops unk rows. Always ordered Yes → No → unk, coloured to match the tree pages. Hover a bar for exact percentages.
Base Entropy: Entropy of the target label distribution before any splitting. The uninformed baseline. Coloured on the item-loss scale (teal = low = good), relative to 1 bit or the highest base entropy in this table, whichever is larger.
Reduced Entropy: Entropy remaining after fitting the decision tree; lower means the tree explains more of the outcome. Coloured on the item-loss scale (teal = low = good), on the same 1-bit-or-highest-base-entropy scale as Base Entropy above, so the two columns are directly comparable -- teal here means a language landed on a meaningfully low absolute entropy, not just a low entropy relative to its own (possibly already-low) starting point.
Δ Entropy: Base Entropy minus Reduced Entropy. How much uncertainty the tree removes. Coloured on the coverage scale (teal = good), by what fraction of the base entropy that delta represents, not the raw number. 0.3 off a base of 0.4 is a much bigger win than 0.3 off a base of 2.0.
DT Acc%: The decision tree's accuracy scored on its own training data. Coloured on the coverage scale (teal = high = good).
N RAW: Candidate rows before the entropy/leaf-confidence filter is applied. Not coloured. There's no cheap per-language reference point (like treebank size) to scale it against, and not informative enough to be worth adding one for.
N KEEP: Of those, the rows kept after the leaf-confidence filter (entropy < 0.1). The ones actually attempted. Coloured on the coverage scale, as a fraction of N RAW — faded (with a hover tooltip) when N RAW is below 10, since that fraction is too noisy to mean much at that size (2/2 reads as a perfect 100% off pure small-sample luck). The colour still shows, just dimmed, rather than disappearing outright.
N PAIRS: Rows that ended up as a correctly re-inflected minimal pair. Coloured on the coverage scale, as a fraction of N KEEP, with the same under-10 fade as N KEEP above (checked against N KEEP this time, not N RAW). Clickable when example rows are available — opens the same examples modal as the bucket breakdown.

Results (below-row panel)

# UD candidates (raw): Same value as N RAW above. Candidate rows before the entropy/leaf-confidence filter.
# UD candidates (kept): Same value as N KEEP above. The rows kept after that filter.
# forms of interest: Distinct DET/HEAD forms eligible for this prediction target at all, whether or not they ended up labelled "Yes".
% covered by UM: Share of those forms found directly in UniMorph's inflection tables.
% covered by UM+UD: Share covered once the UD-derived fallback is added on top of UniMorph.
# UM lemmas / forms: Size of the UniMorph inflection table available for this language.
# valid from multi: Minimal pairs that came from a row with more than one candidate swap form, as a share of all pairs. Clickable when example rows are available.
Feature distribution: Tally of attempted feature-value swaps, as "FROM → TO: count", across every attempt that reached the comparison step.
P(value|value): Of the attempts that started at that value, the fraction where re-inflecting still landed back on the same value. I.e. how often the inflector failed to actually change it.

Diagnostics (bucket breakdown)

Any bucket with at least one row (here, and N PAIRS / "valid from multi" in the Results panel above) has an examples link showing real sample rows — reconstructed sentences, feature values, and a treebank link — for that language and outcome.

no match: No candidate swap form was found for this row at all.
no inflection: A swap form was targeted, but the inflector had nothing to offer for it.
same inflection: The re-inflected form came back identical to the original surface form.
same feature: The re-inflected form's feature value still overlapped the original. The swap didn't actually change it.
undefined feature: The target feature was undefined or missing for this form.
ambiguous HEAD: The row's HEAD couldn't be uniquely resolved.

Numerator / denominator reference

What each percentage above is actually a fraction of. "Row" scope means once per candidate row; "item" scope means once per attempted swap_form. A single row can produce more than one item, so row- and item-scoped percentages use different denominators. extra_pairs isn't shown as a percentage in the table, but shares the same item-scoped pool as the four buckets above.

Column	Numerator	Denominator	Denominator comes from	Scope	Why
% covered by UM	`num_covered_um`	# forms of interest	`num_forms_of_interest`	forms	Distinct DET/HEAD forms found in UniMorph alone.
% covered by UM+UD	`num_covered_um_ud`	# forms of interest	`num_forms_of_interest`	forms	Distinct DET/HEAD forms found via UniMorph or the UD-derived fallback.
% no match	# no match (`no_candidates`)	# UD candidates (kept)	`n_keep`	row	Fires once per candidate row, in place of the swap_form loop. The two are mutually exclusive.
% no inflection	# no inflection (`no_inflections`)	# UD candidates (kept)	`n_keep`	row	Same as "no match": fires instead of entering the swap_form loop.
% same inflection	# same inflection (`same_forms`)	swap_items	`items_seen − no_match − no_inflection`	item	Fires once per attempted swap_form; a row can yield more than one item.
% same feature	# same feature (`same_features`)	swap_items	`items_seen − no_match − no_inflection`	item	Fires once per attempted swap_form.
% undefined feature	# undefined feature (`undefined_features`)	swap_items	`items_seen − no_match − no_inflection`	item	Fires once per attempted swap_form.
% ambiguous HEAD	# ambiguous HEAD (`ambiguous_subjects`)	swap_items	`items_seen − no_match − no_inflection`	item	Fires once per attempted swap_form.
(not shown) extra_pairs	`extra_pairs`	swap_items	`items_seen − no_match − no_inflection`	item	Would collect otherwise-valid swaps (same eligibility as correct_swaps) that get capped once their FROM→TO combination hits the max_num_of_pairs limit. Always 0 today since create_pairs is never called with that limit set.

Colour scales

Coverage: Vermilion (red) at 0%, violet around the middle, teal at 100%. Used where a high percentage is good (UM/UM+UD coverage, DT Acc%, N KEEP as a fraction of N RAW, N PAIRS as a fraction of N KEEP, and Δ Entropy as a fraction of Base Entropy).
Item loss: Teal at 0%, violet around the middle, vermilion at 100%, same scale mirrored. Used in the bucket breakdown (a high percentage means more rows dropped out at that step) and for Base/Reduced Entropy, scaled against the highest base entropy shown in this table rather than a fixed 0–100 range.