# Methodology: AttractivenessTest calibration benchmark (2026-10-21, v1)

## What was measured

How the free AttractivenessTest geometry score (engine 0.2.0, reference 0.2.0-frll, weights fixed-0.1.0) relates to human attractiveness ratings, how far its absolute level sits from the human level, and how much it moves between two photos of the same person and under small landmark errors.

## Data

- Face Research Lab London Set (DeBruine & Jones, 2017), DOI 10.6084/m9.figshare.5047666.v5, licensed CC BY 4.0. 102 adults photographed in a studio; each neutral front face was rated 1–7 by the same 2,513 raters ("much less attractive than average" to "much more attractive than average").
- Files used: `neutral_front.zip`, `smiling_front.zip`, `london_faces_info.csv`, `london_faces_ratings.csv`. The dataset authors are not involved with and do not endorse AttractivenessTest.
- Images and per-face values were processed offline on one machine and are not published. Only aggregate numbers appear in the results files.

## Pipeline

1. Landmarks: MediaPipe Face Landmarker (`face_landmarker_noblend.task`, tasks-vision 1.0.1) in a browser, CPU delegate, image long edge scaled to 1,280 px, the same model and scaling the live site uses.
2. Quality gate: the live `assessQuality` check (one face, frontal pose, eyes open, lips closed, in frame, even light). Faces that fail are not scored, exactly as on the site.
3. Scores: the live `measureFace`, `dimensionScores` and `pslScore` functions. The free PSL is a range because Misc (skin, hair, grooming) is unknown; the midpoint of that range is used for level comparisons. For rank comparisons the geometry composite 0.35 × Harmony + 0.25 × Dimorphism + 0.2 × Angularity is used (Misc is constant and cancels).
4. Human score: each face's mean rating across raters; for agreement it is converted to a z-score within the face's sex, because the site scores men and women against separate references.

## Analyses

- **Agreement (in sample):** Spearman rank correlation between geometry and human z on all scored neutral faces; 95% interval from 2,000 bootstrap resamples. The reference was fitted on these same faces, so this number is optimistic.
- **Agreement (held out):** 500 random half splits. Reference bands are refitted on one half with the production rule (band = 5–95th percentile of the better-rated half widened to the population 10–90th percentile, ideal = median of the better-rated half, spread = population SD) and tested on the other half. We report the median and the 2.5–97.5th percentile range of the 500 test correlations.
- **Human ceiling:** raters are split at random into two halves 500 times; the two half-mean ratings are rank-correlated across the same faces. This shows how well one group of people predicts another.
- **Single rater versus everyone else:** for each of the 2,513 raters (all rated every face), their own ratings and the mean of all other raters are converted to z within sex and rank-correlated across the same scored faces. Raters who gave every face of one sex the same rating have no defined correlation and are excluded (counted in `results.json`). This puts the geometry score next to a typical individual person.
- **Absolute level:** the PSL midpoint distribution on its 1–8 scale and the human mean on its 1–7 scale, both expressed as a share of their own scale ((score − 1) / (max − 1)), plus the share of faces above each scale's midpoint. Scores are also grouped into geometry quintiles with the mean human rating of each group.
- **Expression:** for every scored person, the smiling front photo is run through the same pipeline. We report how many smiling photos the live quality gate refuses and, separately, the score change if the same measurements are forced on them ("ungated"; the site itself would not score these photos).
- **Landmark noise:** Gaussian noise with standard deviation 0.25%, 0.5%, 1% and 2% of face width (landmarks 234 to 454) is added independently to every landmark, 100 times per face. We report the spread of the PSL midpoint per face, the mean absolute change, the rank correlation with the unperturbed scores and the quality-gate pass rate.

All random steps use fixed seeds listed in `results.json`, so a rerun on the same inputs gives the same numbers.

## Reproducing

Download the four FRLL files from figshare, export landmarks with the same MediaPipe model and scaling, and score them with engine 0.2.0. The scoring rules, bands and weights are published on https://attractivenesstest.dev/methodology. The engine source is not open source, so a third party can reproduce the inputs, rules and statistics but not run our exact code; differences in MediaPipe version or image scaling change landmarks slightly.