Benchmark
Is AI Attractiveness Test Accurate? A 2,513-Rater Benchmark
We ran our own face rating on 101 faces that 2,513 people had already rated, then checked how often it agrees with them, whether its numbers run high, and how much a different photo moves them.
By the AttractivenessTest team · · Benchmark 2026-10-21-v1 · engine 0.2.0 · reference 0.2.0-frll
Short answer: only partly
On faces it had not been fitted to, our geometry score put people in roughly the same order as a crowd of human raters with a rank correlation of about 0.36. A typical single person agrees with the rest of the crowd at about 0.56. The free score also sits higher on its own scale than human ratings sit on theirs, and it can move by several tenths of a point between two photos of the same face.
Studies of AI rating sites ask exactly these questions, and almost no site answers them with data. So here are ours, including the parts that do not flatter the tool.
What we tested
- 101 adult faces from the Face Research Lab London Set (DeBruine & Jones, 2017, CC BY 4.0), each rated 1–7 by the same 2,513 people. One of the 102 neutral photos failed our quality check (head turned).
- The pipeline the site runs: MediaPipe Face Landmarker in a browser, the live quality check and the live scoring code from our methodology.
- No photo, landmark or per-face score from the set is shown or stored on this site; only the summary numbers below. The dataset authors are not involved with and do not endorse AttractivenessTest.
1. Does the score agree with people?
On 500 random half-splits of the Face Research Lab London Set, AttractivenessTest's geometry score (engine 0.2.0) reached a median held-out Spearman correlation of 0.36 with mean human ratings (95% of splits between 0.15 and 0.52); a typical single rater agreed with the other raters at a median of 0.56.
| Comparison | Rank correlation |
|---|---|
| Our geometry score vs crowd, held out (median of 500 splits) | 0.36 (0.15–0.52) |
| Our geometry score vs crowd, in sample (reference fitted on these faces) | 0.45 (bootstrap 95%: 0.27–0.60) |
| Single rater vs everyone else (median, middle half) | 0.56 (0.45–0.64) |
| Half the raters vs the other half | 0.998 |
The score predicts the crowd better than about 13% of individual raters do, so it is a weak signal, not a verdict. That is expected: it measures facial proportions only and ignores skin, hair, expression and style.
For context, a 2018 test by an attractiveness researcher on 79 faces from the same set reported Spearman correlations of 0.53 and 0.52 for two other rating sites. It used a different subset and those sites do not publish how they score, so it is not a head-to-head comparison.
2. Is the score inflated?
On the same 101 faces, AttractivenessTest's free score averaged 4.71 on its 1–8 scale (53% of the scale), while the human mean was 3.01 on a 1–7 scale (34%); 64% of faces scored above the site's scale midpoint, against 9% for human ratings.
| Our free score (1–8) | Human mean rating (1–7) | |
|---|---|---|
| Mean | 4.71 | 3.01 |
| 10th–90th percentile | 3.75–5.45 | 2.13–3.95 |
| Mean as share of scale | 53% | 34% |
| Faces above scale midpoint | 64% | 9% |
Yes, by design: the 1–8 scale was anchored so that an average face from this calibration set lands near 4.8, which is above where the human average sits on the human scale. Published studies found the same pattern elsewhere: five AI rating websites averaged 6.9 against 5.0 for a human focus group (Goshtasbi et al., 2024), and four AI chat models averaged 4.71 against 3.02 for humans on these same London faces (Grandas et al., 2026). Absolute numbers from different raters are not comparable; rank order is the fairer test.
| Geometry quintile | Mean free score | Mean human rating (1–7) |
|---|---|---|
| 1 (lowest) | 3.78 | 2.56 |
| 2 | 4.41 | 2.73 |
| 3 | 4.78 | 3.18 |
| 4 | 5.04 | 3.07 |
| 5 (highest) | 5.53 | 3.51 |
3. Does a smile change the result?
When each person's smiling front photo was scored instead of the neutral one, AttractivenessTest's quality gate refused 33 of 101 photos (32 for an open mouth); for the 68 it accepted, the free score changed by 0.38 points on average and kept a 0.74 rank correlation with the neutral-photo scores.
Retaking a photo with a different expression can move the score by a few tenths, and an open-mouth smile is refused rather than scored. Forcing the same measurements onto every smiling photo, including ones the site would refuse, changed the score by 0.45 points on average, and 43% moved by half a point or more. Use a neutral, closed-mouth photo and compare two or three shots.
4. How much do small landmark errors matter?
Adding random landmark error with a standard deviation of 0.5% of face width changed AttractivenessTest's free score by 0.37 points on average, mostly downward (mean −0.31), and lowered the rank correlation with the original scores to 0.81; at 1% of face width the average change was 0.84 points.
| Landmark error (SD, share of face width) | Mean absolute change | Mean signed change | Rank correlation with original |
|---|---|---|---|
| 0.25% | 0.18 | −0.09 | 0.93 |
| 0.5% | 0.37 | −0.31 | 0.81 |
| 1% | 0.84 | −0.82 | 0.58 |
| 2% | 1.48 | −1.47 | 0.29 |
Noise pushes measurements away from the ideal values, so scores drift down as detection gets worse. Good light, a straight-on head and a phone held at arm's length keep detection error small.
What this means for any AI attractiveness test
- Treat the number as a rough, proportion-based signal. Check whether a site says what it measures and how well it agrees with people; most do not.
- Compare ranks or changes between your own photos, not absolute scores across apps.
- Expect a few tenths of movement between photos; differences that small are noise.
Methods, limits and sources
Agreement uses a 2,000-sample bootstrap and 500 random half-splits that refit the reference bands on one half with the production rule and test on the other. Every random step uses a fixed seed listed in results.json. The engine code is not open source, so others can reproduce the inputs, rules and statistics but not run our exact code.
- All results, seeds and versionsresults.jsonJSON ↓
- Every number on this pagesummary-tables.csvCSV ↓
- Full methodmethodology.mdMarkdown ↓
- Scope and limitationslimitations.mdMarkdown ↓
- Sources and attributionsources.jsonJSON ↓
Limits. Small studio sample from London (about two-thirds white, ages 18–54), fitted and tested on the same set, geometry only, one alternative photo per person, a simplified noise model, not a test of any other site or model, and not medical or cosmetic advice. Read the full limitations before reusing the numbers.
- DeBruine, L. & Jones, B. (2017). Face Research Lab London Set. figshare. doi.org/10.6084/m9.figshare.5047666.v5 (CC BY 4.0)
- Goshtasbi, K., Hakimi, A. A. & Wong, B. J. F. (2024). Artificial Intelligence Versus Human Focus Group Rating of Facial Attractiveness. Facial Plastic Surgery & Aesthetic Medicine.
- Grandas, S. et al. (2026). Beauty is in the AI of the beholder: MLLMs systematically overrate facial attractiveness. arXiv:2609.02512 (preprint).
- Manandhar-Richardson, T. (2018). An attractiveness researcher puts the Internet's most popular “hotness algorithms” to the test. Medium.
- Google. MediaPipe Face Landmarker.
Cite as: AttractivenessTest (2026). AI attractiveness test accuracy benchmark v1. https://attractivenesstest.dev/research/ai-attractiveness-test-accuracy. Authored by the team operating AttractivenessTest; external researchers do not endorse it. Report a correction to support@attractivenesstest.dev. Frozen attachments stay unchanged; corrections receive a new dated version.
Try the measured test
The free test measures a photo in your browser, and your photo stays on your device. Now you know how far to trust the number.