mantra/mantra-kmp
1
0
Fork 0
Code Issues 2 Pull Requests Actions 1 Packages Projects Releases 1 Wiki Activity

test: measure M3 conformance instead of asserting it, with a budgeted audit and a contrast test

Browse Source 
Phase 0 of docs/material-design-conformance.md. Every count in that document was
produced by hand, which makes the eight phases after it opinions rather than work
with acceptance criteria. This is the harness that turns them back into numbers.

**`docs/scripts/m3-audit.sh` regenerates the whole audit, and can fail a build.**
Plain invocation reports; `--check` exits 1 when a budget at the top of the file is
exceeded. The budgets are the tree as it stands -- 11 hardcoded colours, 33 bare
`.clickable`, 18 null content descriptions, 12 unassigned colour roles -- and the
contract written into the header is that they ratchet **down**, in the same commit
that earns the reduction, and are never raised. Counts a phase has not reached yet
are `-1`, which reports but never fails. Phase 8 wires `--check` into CI, at which
point a raised budget is the diff a reviewer is looking for.

Verified both directions: `--check` exits 0 on the clean tree, and appending a
single `Color(0xFF00FF00)` to LoadingScreen.kt makes it exit 1 naming the budget.

**Two counts are reported apart from each other on purpose.** Thirteen ColorScheme
roles are never assigned in Theme.kt, and reporting that as one number would
overstate it. Twelve are the `*Fixed*` family, which default to
`ColorLightTokens.PrimaryFixed` -> `PaletteTokens.Primary90` -> `#EADDFF`, so a
monochrome app renders Material baseline lavender the moment anything reads one.
The thirteenth is `surfaceTint`, whose default is `primary` -- correct, and not a
defect. The script labels the first group "lavender" and the second "not a defect".

The `.dp` histogram splits three ways for the same reason. 527 literals: 419 on the
M3 spacing scale, 19 dimensions rather than spacing (a 1dp hairline, an avatar, an
image height), and 89 genuinely off-scale. The naive split reported 101 off-scale by
counting 1dp borders as bad spacing, which would have sent phase 2 chasing hairlines.
`DIMENSION_EXEMPT` is deliberately short and the header asks for a justification in
the commit that lengthens it.

**`ColorSchemeContrastTest` walks the real schemes, which cost a visibility keyword.**
Four assertions over all six declared schemes: every content role on its container at
4.5:1, `onSurface` on each of the seven tonal surfaces at 4.5:1, `outline` against
every surface it is drawn on at 3:1, and `primary`/`error` against `surface` at 3:1.
WCAG relative luminance from first principles -- the 0.03928 knee and the 2.4
exponent, not a gamma-2.2 approximation, because the approximation moves borderline
pairs by enough to change a verdict and the tightest pair in this tree is 4.56:1.

`Theme.kt`'s six schemes went from `private val` to `internal val` so the test can
see them. The alternative -- rebuilding the schemes inside the test from `Color.kt`'s
public values -- keeps production visibility untouched and was rejected: it would
assert the palette and miss the wiring, and the wiring is the half that fails
silently. `surfaceContainerHigh = surfaceContainerHighestLight` is a one-character
slip, compiles, and reads fine in review. A comment above the first scheme says this,
so the keyword is not quietly widened back.

**Verified that it bites.** Nudging `onSurfaceVariantLight` from `#4C4546` to
`#9C9496` -- a plausible "soften the secondary text" edit that nothing else in the
build would object to -- fails with `light: onSurfaceVariant on surfaceVariant is
2.29:1`, naming scheme, pair and ratio. Reverted; the committed value is unchanged.

**Monotonicity across the contrast ladder is deliberately not asserted.** The obvious
invariant -- high-contrast beats medium beats default for every pair -- looks right
and is false. Ten pairs move the other way, and correctly: in the light high-contrast
scheme `surfaceContainerHighest` goes darker to separate it from `surface`, which
drops its ratio against `onSurface` from 13.30 to 12.29 while raising the separation
that the change exists for. `onErrorContainer on errorContainer` drops 7.24 -> 5.19
from default to medium for the same kind of reason. Asserting the ladder would have
meant either a red test or nine exemptions; the floor is the real invariant and every
one of those values is comfortably above it. The test's doc comment records this so
the next reader does not add the assertion.

**Also not asserted: `outlineVariant`, and the call sites.** `outlineVariant` reads
1.61:1 against surface, which looks alarming and is not a defect -- M3's own baseline
sits in the same range and the role is a decorative divider, so `outline` is what
gets the 3:1 assertion. The seven call-site pairings that are genuinely below
threshold, including the 1.00:1 one in ProposalListScreen, belong to phase 3; adding
them now would mean checking in a red test.

**Doc reconciled to the script rather than the other way round.** Three hand counts
were wrong and are corrected in docs/material-design-conformance.md: 520 `.dp`
literals -> 527 (the earlier figure omitted the exempt dimensions), 90 `label*`
typography uses -> 92 (it missed `labelSmallEmphasized` and `labelLargeEmphasized`,
which are label roles too), and 101 off-scale -> 89. The phase 0 section is rewritten
from a plan into what was built, including what was decided against.

**Tests.** 914 pass, 580 jvm over 70 classes and 334 android over 42 classes, up from
906/576/69 and 330/41 -- the four new assertions, in one new class, counted once per
target because commonTest flows into both. `:composeApp:compileDebugKotlinAndroid`
builds. No app behaviour changes: the only production edit in this commit is
`private` -> `internal` on six vals.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kgothatso Ngako
2026-09-07 23:51:34 +02:00
parent 1c02b25a07
commit 2b0ce8d73b
4 changed files with 499 additions and 46 deletions
Download Patch File Download Diff File Expand all files Collapse all files

88
docs/material-design-conformance.md
View File

@@ -195,21 +195,20 @@ is the specified disabled opacity and is exempt.
### Spacing is a habit, not a system
520 `.dp` literals. Roughly 419 land on a defined spacing stop and about 101 do
not:
527 `.dp` literals. 419 land on a defined spacing stop, 19 are dimensions rather
than spacing (a hairline border, an avatar, an image height), and 89 are off-scale:
```
1dp × 12 5dp × 10 15dp × 14 18dp × 1 22dp × 1
30dp × 2 35dp × 3 50dp × 53 55dp × 2 70dp × 2 75dp × 1
5dp × 10 15dp × 14 18dp × 1 22dp × 1 30dp × 2
35dp × 3 50dp × 53 55dp × 2 70dp × 2 75dp × 1
```
Two things are true at once here, and the second matters more. The 1dp values
are hairline borders and dividers, which is fine. But `10.dp` (132 uses) and
The off-scale values are the smaller half of the problem. `10.dp` (132 uses) and
`20.dp` (115) — the two dominant values — *are* on the scale, as `space125` and
`space250`. So this is not mostly an off-grid problem. It is that **nothing
records which of padding, gap or margin any of these is**, so there is no way to
adapt them per breakpoint or density later, and no way to tell a deliberate 15dp
from a typo.
`space250`, so a sweep for off-grid numbers would change almost nothing. What is
wrong is that **nothing records which of padding, gap or margin any of these
is**, so there is no way to adapt them per breakpoint or density later, and no
way to tell a deliberate 15dp from a typo.
`Modifier.height(50.dp)` appears 49 times, almost always as a `Spacer` pushing
an empty or error message down the screen. It is the same three lines copied
@@ -226,7 +225,7 @@ labelLarge 13 titleSmall 12 titleMedium 6 titleLarge 4 headlineSmall 4
headlineMedium 2 headlineLarge 2 displayMedium 1
```
`label*` roles are 90 of 240 uses. Labels are for component text — buttons,
`label*` roles are 92 of 240 uses. Labels are for component text — buttons,
tabs, chips — not for body copy or list content, and they are the smallest and
tightest roles in the scale. Reading a screen where `labelMedium` carries the
prose is the visual equivalent of everything being at the same pitch. Meanwhile
@@ -338,7 +337,7 @@ state change in the app is a hard cut.
|---|---|---|
| Theme completeness | 12 roles unset, 4 schemes unreachable, no shapes/motion | 1 |
| Colour at call sites | 7 pairings under threshold, one at 1.00:1 | 1, 3 |
| Spacing | 520 literals, no role recorded | 2 |
| Spacing | 527 literals, no role recorded | 2 |
| Typography | 90/240 uses on `label*`, 2/30 roles emphasized | 1 |
| Targets & labels | 33 unguarded `.clickable`, 18 untriaged nulls | 3 |
| Content | 334 literals, title case throughout | 4 |
@@ -361,38 +360,47 @@ so they lock in real state rather than aspiration.
**Why first.** Every count in this document was produced by hand. If they cannot
be regenerated, the phases below have no acceptance criteria — only opinions.
**Work.**
**Built.**
1. `docs/scripts/m3-audit.sh`, checked in, emitting the tables above: dp
histogram split by on/off the spacing scale, typography role distribution,
hardcoded colour sites, `.clickable` sites, `contentDescription = null`
count, string literal count, snackbar count, adaptive API count.
2. `composeApp/src/commonTest/.../ui/theme/ColorSchemeContrastTest.kt` — a pure
computation over the six declared schemes, no Compose runtime needed:
1. **`docs/scripts/m3-audit.sh`** regenerates every count in "Where this app
stands": the dp histogram split three ways, the typography role
distribution, hardcoded colour sites, `.clickable` sites,
`contentDescription = null`, string literals, snackbars, adaptive APIs. Each
number that a phase is meant to move carries a **budget** at the top of the
file, and `--check` exits 1 when one is exceeded. Budgets ratchet down in the
same commit that earns the reduction; Phase 8 wires `--check` into CI, at
which point raising one is the thing a reviewer looks for.
```kotlin
private fun ratio(a: Color, b: Color): Double { … } // WCAG relative luminance
Two counts it reports separately, because conflating them would overstate the
problem: the twelve `*Fixed*` roles that fall through to Material baseline
lavender, and `surfaceTint`, which is also unassigned but defaults to
`primary` and is therefore correct. The `.dp` histogram likewise splits
dimensions (a hairline border, an avatar) out of the off-scale count.
@Test
fun everyOnRolePairsAtFourPointFive() {
schemes.forEach { (name, scheme) ->
pairs.forEach { (bg, fg) ->
val r = ratio(bg(scheme), fg(scheme))
assertTrue(r >= 4.5, "$name: ${fg.name} on ${bg.name} is $r")
}
}
}
```
2. **`ColorSchemeContrastTest`** in `commonTest` — WCAG relative luminance from
first principles, no Compose runtime, four assertions over all six schemes:
every content role on its container at 4.5:1, `onSurface` on each of the
seven tonal surfaces at 4.5:1, `outline` against every surface it is drawn on
at 3:1, and `primary`/`error` containers against `surface` at 3:1. 4 tests,
green.
It passes today. It exists so Phase 1 cannot regress it, and so Phase 3 has
somewhere to add the call-site pairings.
3. Baseline the numbers into `docs/` alongside this file, dated.
It walks the real `ColorScheme` objects, which is why `Theme.kt`'s six
schemes moved from `private` to `internal`: rebuilding them in the test from
`Color.kt` would assert the palette and miss the wiring, and a role pointed at
its neighbour's value is exactly the slip that reads fine in review.
**Done when** the script runs from a clean checkout and its output matches the
tables in "Where this app stands", and `:composeApp:jvmTest` runs the contrast
test green.
Verified to bite. Nudging `onSurfaceVariantLight` from `#4C4546` to `#9C9496`
— a plausible "soften the secondary text" edit — fails it with
`light: onSurfaceVariant on surfaceVariant is 2.29:1`.
**Risk:** none. Nothing in the app changes.
**Deliberately not asserted:** monotonicity across the contrast ladder. The
obvious invariant — high-contrast beats medium beats default, for every pair —
is false, and correctly so: ten pairs move the other way because a high-contrast
scheme darkens `surfaceContainerHighest` to separate it from `surface`, trading
ratio against `onSurface` for the separation that matters. The floor is the
invariant; the ladder is not.
**Risk:** none to the app. The only production change is a visibility keyword.
---
@@ -463,7 +471,7 @@ alone.
### Phase 2 — spacing becomes a token
**Why here.** Phase 6 has to adapt spacing per breakpoint. It cannot adapt 520
**Why here.** Phase 6 has to adapt spacing per breakpoint. It cannot adapt 527
literals.
**Work.**
@@ -495,7 +503,7 @@ literals.
the audit distinguish padding from gap from margin, which the raw scale
cannot.
3. **Migrate, in the order the audit reports.** The 101 off-scale values are the
3. **Migrate, in the order the audit reports.** The 89 off-scale values are the
interesting ones and go first: each is either a typo (round to the nearest
stop) or deliberate (say why, in a comment, and pick the nearest stop
anyway). Then `10.dp` and `20.dp` en masse.