NIST Face Recognition Benchmarks: How to Read the Numbers
NIST runs the most credible independent evaluation of face recognition algorithms in the world, testing on operational datasets rather than public benchmarks.
What it publishes are error rates: false match rate and false non-match rate, reported at fixed operating points and broken down by demographic group. What it does not publish is a pass mark.
NIST evaluates and reports, it does not certify, so a vendor’s accuracy figure only means something once you know the false match rate it was measured at.
What NIST Actually Tests: FRVT, FRTE, and FATE
NIST has evaluated face recognition independently since 1999, originally under the Face Recognition Vendor Test.
The programme has since been reorganised. Per NIST, what was formerly FRVT has been rebranded and split into FRTE, the Face Recognition Technology Evaluation, covering tracks that pertain to identity verification, and FATE, the Face Analysis Technology Evaluation, covering tracks that process and analyse images. Existing participation and submission procedures remained unchanged.
That split follows the same line drawn in our guide to 1:1 and 1:N matching: recognition establishes identity, analysis estimates attributes. The FRVT name still dominates search results and older reports, so expect to encounter all three labels.
Why the testing methodology matters
Two features separate NIST evaluation from vendor self-reporting and from academic benchmarks.
It uses sensitive operational datasets not available publicly, including mugshots, visa application photographs, and imagery from border kiosks. Public benchmark datasets can be trained against; these cannot.
And developers submit algorithms for third-party testing rather than reporting their own results. Participation involves a signed agreement, wrapping the software behind NIST’s published API, and submitting an encrypted package that NIST runs itself. Evaluation is free and ongoing, so developers may submit whenever they are ready.
The scale is considerable. NIST’s demographic study covered nearly 200 algorithms from nearly 100 developers, using four collections containing more than 18 million images of more than 8 million people, and 1:N tracks search galleries of up to 12 million identities.
FMR and FNMR: The Two Error Rates
Every accuracy claim reduces to two numbers, and they move in opposite directions.
| False Match Rate (FMR) | False Non-Match Rate (FNMR) | |
|---|---|---|
| What happens | Two different people are matched | The same person is not matched |
| Business consequence | Fraud gets through | A genuine customer is rejected |
| Also called | False positive, false accept | False negative, false reject |
| Effect of a stricter threshold | Falls | Rises |
| Who complains | Risk and compliance | Growth and customer support |
NIST reports FNMR at a fixed FMR, which per the literature is equivalent to the TAR at FAR metric used elsewhere. Common causes of false non-matches include long-run ageing, acute injury, and most commonly poor image quality.
Why There Is No Single “Accuracy” Number
A vendor claiming 99.7% accuracy has told you almost nothing, and the reason is structural rather than evasive.
Both error rates depend on the threshold. Move it stricter and false matches fall while false rejections rise. Move it looser and the reverse happens. A single percentage figure is a point on a curve with the axis labels removed.
What makes a figure interpretable is the operating point: FNMR quoted at a stated FMR. “FNMR of 0.4% at FMR of 1 in 100,000” is a claim you can compare against another vendor. “99.6% accurate” is not, because it may have been measured at an FMR three orders of magnitude looser.
Which dataset the figure came from matters equally. Error rates on high-quality portrait photographs bear little relation to error rates on phone selfies taken in poor light, and Philippine remote onboarding produces the second kind.
Demographic Differentials: What NIST Measures
NIST quantified demographic effects in NISTIR 8280 and extended the analysis in NISTIR 8429, and the findings should shape how any institution with a diverse customer base sets its thresholds.
The clearest illustration from NISTIR 8429: at a threshold set to achieve a false match rate of 1 in 25,000 for Polish men aged 35 to 50, Nigerian women aged 65 and over show a false match rate of roughly 1 in 35.
One threshold. Error rates differing by around three orders of magnitude depending on who is standing in front of the camera. NIST notes that many algorithms are assumed to be trained on imbalanced, majority white databases, which is the mechanism behind the spread.
NIST measures these differentials across sex, age group, and region of birth, and publishes them on the demographic tabs of the results pages.
The counterweight worth knowing
Reporting only the bias finding overstates it, and NIST is explicit about the corrective.
Per NISTIR 8429, the range of FNMR across algorithms is larger than the range across demographics. Put plainly, choosing a better algorithm does more for your error rates than demographic variation costs you. Bias is real and measurable, and algorithm selection remains the larger lever.
NIST also plots algorithms achieving both low FNMR and more even FMR across demographics, which demonstrates that the two properties are not in tension. Vendors do not have to trade accuracy for evenness.
Why this matters specifically in the Philippines
NIST’s demographic reporting groups by region of birth, and Southeast Asian populations are not the group most algorithms were optimised for. Any figure quoted from a Western-dominated evaluation set is a weak predictor of performance on Philippine customers.
This also connects to a live regulatory requirement. As covered in our guide to account takeover and the BSP authentication rules, replacement authentication must remain accessible to all users, including elderly customers. NIST’s data shows error rates rising with age, so an accessibility obligation and a demographic differential are the same problem viewed from two directions.
What “NIST Certified” Does and Does Not Mean
Worth being direct here, because the phrase is widely searched and widely echoed.
NIST describes its face programmes as independent government evaluations. Developers submit algorithms, NIST runs them against its datasets, and NIST publishes the resulting error rates in reports and on its results pages. There is no threshold a vendor passes and no mark NIST issues.
So a claim of NIST certification does not correspond to anything NIST awards. What a vendor can legitimately say is that their algorithm was submitted to and evaluated by NIST, and they can point to the specific report and the row containing their results.
That distinction gives you a clean procurement question. Ask which report, which submission date, and which rank on which track. A vendor genuinely evaluated by NIST can answer in one sentence, because the results are public. A vendor using the phrase decoratively cannot.
One further caveat applies even to a genuine result. Submissions are dated, and algorithms are updated frequently. A strong result from three years ago describes an algorithm that may not be the one in the product you are buying.
Threshold Setting: Where Accuracy Becomes a Business Decision
Vendors supply algorithms. Institutions choose thresholds, and that choice is where measured accuracy turns into experienced performance.
The question is not which error you prefer but what each one costs you. A lending platform rejecting genuine applicants loses revenue per rejection that is easy to quantify. A bank admitting a fraudulent account carries loss plus regulatory exposure that is harder to bound. Those two institutions should not run the same threshold, even on the same algorithm.
Segmentation helps where a single value serves a diverse book poorly, and the NIST demographic data is the argument for it. So does routing borderline scores to human review rather than auto-declining, which our guide to biometric deduplication covers as adjudication.
One category of failure sits outside all of this. A false match caused by an algorithm confusing two similar faces is a threshold problem. A false match caused by a presentation attack is not, and no threshold setting corrects for it. That is what liveness detection addresses, and NIST error rates say nothing about it.
Questions to Ask a Face Recognition Vendor
Has your algorithm been submitted to NIST, and which report and submission date should we look at? Public results make this verifiable in minutes.
What is your FNMR at a stated FMR, and on which dataset? An accuracy percentage without an operating point is not comparable.
What do your demographic differentials look like, particularly by age and region of birth? Relevant to any Philippine customer base.
Is the evaluated algorithm the one in the product we would deploy? Version drift between submission and shipping is common.
How is liveness handled, and is it tested separately? Matching accuracy and spoof resistance are different properties measured differently.
Which thresholds do you recommend, and can we configure them per segment? A vendor unwilling to expose thresholds is making a business decision on your behalf.
Frequently Asked Questions
Does NIST certify face recognition algorithms?
- No. NIST describes its face programmes as independent government evaluations. Developers submit algorithms, NIST tests them against its datasets, and NIST publishes error rates in reports. There is no pass mark and no certification NIST issues, so a vendor can accurately say their algorithm was evaluated by NIST, but not that it is NIST certified.
What is the difference between FRVT, FRTE, and FATE?
- FRVT was the original programme name. NIST rebranded and split it into FRTE, the Face Recognition Technology Evaluation, covering identity verification tracks, and FATE, the Face Analysis Technology Evaluation, covering image processing and analysis tracks. Participation and submission procedures were unchanged, and the FRVT label still appears widely in published reports.
What do FMR and FNMR mean?
- False Match Rate is the rate at which two different people are incorrectly matched, meaning fraud gets through. False Non-Match Rate is the rate at which the same person is not matched, meaning a genuine customer is rejected. They move in opposite directions as the threshold changes, which is why NIST reports FNMR at a fixed FMR rather than a single accuracy figure.
Why is a 99% accuracy claim not useful?
- Because both error rates depend on the threshold, so a single percentage is a point on a curve with the axis labels removed. A comparable figure states FNMR at a specified FMR and names the dataset it was measured on. Error rates on high-quality portraits differ substantially from error rates on phone selfies in poor light.
Do face recognition algorithms perform differently across demographic groups?
- Yes, and NIST quantifies it. NISTIR 8429 reports that at a threshold achieving a false match rate of 1 in 25,000 for Polish men aged 35 to 50, Nigerian women aged 65 and over show a false match rate of roughly 1 in 35. NIST also notes that the range of FNMR across algorithms is larger than the range across demographics, so algorithm choice remains the larger lever.
Do NIST results tell us anything about deepfake resistance?
- Not directly. NIST error rates measure matching accuracy, meaning how often the algorithm confuses two faces or fails to match the same face. Resistance to presentation attacks, screen replays, and synthetic media is a separate property addressed by liveness detection and tested separately.
Ask for the Operating Point, Not the Percentage
Almost every unproductive vendor conversation about face recognition accuracy comes from comparing numbers that were never comparable. One vendor quotes a figure measured at a loose false match rate on clean portraits, another quotes a stricter operating point on harder imagery, and the second looks worse while being better.
Two questions fix that. What is your FNMR at a stated FMR, on what data. And which NIST report shows it.
Then set the threshold against your own economics rather than the vendor’s default, and treat liveness as a separate requirement, because no matching accuracy figure describes what happens when the face presented was never real.
Verihubs eKYC API runs 1:1 face verification against 15+ Philippine government ID types with configurable thresholds, liveness, and deepfake detection in the same flow.
Talk to the Verihubs team about face matching thresholds for your customer base.
