Homomorphic Encryption for Biometric Template Protection
EAB Workshop on Biometric Template Protection
Vishnu Naresh Boddeti, Michigan State University
14th October, 2026
Biometric Systems are vulnerable to Template Attacks
Attacks on Face Recognition Systems
Attacks on Face Recognition Systems: Template Inversion
Template inversion attack on High resolution image
From Template inversion attack to Replay and Presentation attack
Template inversion attack enables Presentation attack
Presentation attack via digital replay and printed photograph
Biometric Template Protection
- Goal: Protecting templates in a biometric system.
- Conceptual idea of BTP:
- Template -> Transform -> Protected Template
Encrypted Template Protection
The blind spot of traditional encryption
Privacy of user data is not guaranteed.
Encryption Schemes
What we have.
What we want.
Is there an encryption scheme that satisfies our security desiderata?
FHE can help AI models achieve trustworthiness
- FHE is conjectured to be post-quantum secure.
FHE enables AI models to process encrypted data without decryption.
What is Fully Homomorphic Encryption?
Run programs on encrypted data without ever decrypting it.FHE can—in theory—handle universal computation.
Evolution of FHE schemes
Learning with Errors (LWE)
LWE problem prevents attackers from breaking FHE schemes' securityThe Hardness Foundation of LWE-Based FHE
$\mathbf{s} = \begin{bmatrix}10 \\ 82 \\ 50 \\ 51\end{bmatrix}$
\[ \begin{align} 77w + 7x + 28y + 23z &= 2859 \text{ } \nonumber \\ 21w
+ 19x + 30y + 48z &= 3508 \text{ } \nonumber \\ 4w + 24x + 33y + 38z &=
3848 \text{ } \nonumber \\ 8w + 20x + 84y + 61z &= 6225 \text{ }
\nonumber \\ \end{align} \]
\[ \begin{align} 77w + 7x + 28y + 23z &= 2859 \text{ } \nonumber \\ 21w
+ 19x + 30y + 48z &= 3508 \text{ } \nonumber \\ 4w + 24x + 33y + 38z &=
3848 \text{ } \nonumber \\ 8w + 20x + 84y + 61z &= 6225 \text{ }
\nonumber \\ \end{align} \]
\[ \begin{align} 77w + 7x + 28y + 23z &= 2859 \text{ } \nonumber \\ 21w
+ 19x + 30y + 48z &= 3508 \text{ } \nonumber \\ 4w + 24x + 33y + 38z &=
3848 \text{ } \nonumber \\ 8w + 20x + 84y + 61z &= 6225 \text{ }
\nonumber \\ \end{align} \]
\[ \begin{align} 77w + 7x + 28y + 23z &= 2859 \text{ } \color{#ff8080}{
+ \text{ } (-3)} \nonumber \\ 21w + 19x + 30y + 48z &= 3508 \text{ }
\color{#ff8080}{ + \text{ } (+2)} \nonumber \\ 4w + 24x + 33y + 38z &=
3848 \text{ } \color{#ff8080}{ + \text{ } (-1)} \nonumber \\ 8w + 20x +
84y + 61z &= 6225 \text{ } \color{#ff8080}{ \underbrace{+ \text{ }
(+0)}_{noise}}\nonumber \\ \end{align} \]
\[ \begin{align} 77w + 7x + 28y + 23z &= 2859 \text{ } \color{#ff8080}{
+ \text{ } (-3)} \nonumber \\ 21w + 19x + 30y + 48z &= 3508 \text{ }
\color{#ff8080}{ + \text{ } (+2)} \nonumber \\ 4w + 24x + 33y + 38z &=
3848 \text{ } \color{#ff8080}{ + \text{ } (-1)} \nonumber \\ 8w + 20x +
84y + 61z &= 6225 \text{ } \color{#ff8080}{ \underbrace{+ \text{ }
(+0)}_{noise}}\nonumber \\ \end{align} \]
\[ \begin{align} 77w + 7x + 28y + 23z &= 2859 \text{ } \color{#ff8080}{
+ \text{ } (-3)} \color{cyan}{\text{ (mod 89)}} \nonumber \\ 21w + 19x +
30y + 48z &= 3508 \text{ } \color{#ff8080}{ + \text{ } (+2)}
\color{cyan}{\text{ (mod 89)}} \nonumber \\ 4w + 24x + 33y + 38z &= 3848
\text{ } \color{#ff8080}{ + \text{ } (-1)} \color{cyan}{\text{ (mod
89)}} \nonumber \\ 8w + 20x + 84y + 61z &= 6225 \text{ }
\color{#ff8080}{ \underbrace{+ \text{ } (+0)}_{noise}}\text{
}\color{cyan}{\underbrace{\text{(mod 89)}}_{ring}} \nonumber \\
\end{align} \]
\[ \begin{align} 77w + 7x + 28y + 23z &= 2859 \text{ } \color{#ff8080}{
+ \text{ } (-3)} \color{cyan}{\text{ (mod 89)}} \nonumber \\ 21w + 19x +
30y + 48z &= 3508 \text{ } \color{#ff8080}{ + \text{ } (+2)}
\color{cyan}{\text{ (mod 89)}} \nonumber \\ 4w + 24x + 33y + 38z &= 3848
\text{ } \color{#ff8080}{ + \text{ } (-1)} \color{cyan}{\text{ (mod
89)}} \nonumber \\ 8w + 20x + 84y + 61z &= 6225 \text{ }
\color{#ff8080}{ \underbrace{+ \text{ } (+0)}_{noise}}\text{
}\color{cyan}{\underbrace{\text{(mod 89)}}_{ring}} \nonumber \\
\end{align} \]
Breaking FHE $\Leftrightarrow$ Solving LWE problem: recovering private key from public key.
| System | Problem | Complexity | Solution |
| $b=As$ in $\mathbb{R}$ | System of Linear Equations | P | Gaussian Elimination |
| $b=As+e$ in $\mathbb{R}$ | Least Squares Problem | P | Least Squares Estimator |
| $b=As+e \mod q$ in $\mathbb{Z}_q$ | Learning with Errors Problem | NP-hard | No known efficient algorithm (not even quantum) |
| $b(X)=A(X)s(X)+e(X) \mod q$ in $\mathbb{Z}_q[X]/(X^N+1)$ | Ring Learning with Errors Problem | NP-hard | No known efficient algorithm (not even quantum) |
Pipeline of Homomorphic Evaluation of Encrypted Data
Encoding/Decoding
Key generation
Encryption/Decryption
Evaluation
FHE Schemes support basic HE capabilities
Bootstrapping enables unlimited HE operations over encrypted data
Bootstrapping is very slow and need to be avoided
Data Packing and SIMD operations
Choosing the right packing scheme significantly reduces latency.
FHE Integration Challenges in face search
We need an efficient and scalable inner product under FHE.
Packing scheme selection
Minimize #Mult and #Rot
Avoid bootstrapping
SecureFace: Inner Product with one homomorphic multiplication
- Introduced in FHE-based face matching [B18].
- Encrypted inner product $$ct = \left[\!\left[ \mathbf{x} \right]\!\right] \times \left[\!\left[ \mathbf{y} \right]\!\right] = \left[\!\left[ (x_1 y_1, \cdots, x_d y_d) \right]\!\right]$$ $$\left[\!\left[ < \mathbf{x},\mathbf{y}>\right]\!\right] = \sum_{i=1}^{\log_2(d)} \text{rot}(ct, 2^i)$$
- Runs in $31.22$ ms for 128 bits security level.
Packing scheme
Complexity
| $\log_2(d)-1$ | $1$ | $\log_2(d)$ |
Bootstrapping: not needed
Reducing latency
$\Downarrow$
Reducing multiplications and rotations.
Idea to eliminate multiplications: Pre-computing IP into a Lookup table
MFIP lookup table applies this idea to reduce latency in face matching.
MFIP: Inner Product with zero homomorphic multiplication
- 2nd generation of practical FHE-based face matching [BHV+23].
- Preserves inner product accuracy.
Packing scheme
Complexity
| $\log_2(d)-1$ | $0$ | $\log_2(d)$ |
Bootstrapping: not needed
Effect of Eliminating homomorphic multiplications on Latency
Removing multiplications decreases latency by half.
Is this efficiency gain enough to scale up to search?
Packing scheme
Bootstrapping: not needed
SecureFace Complexity
| $\log_2(d)-1$ | $1$ | $\log_2(d)$ |
MFIP Complexity
| $\log_2(d)-1$ | $0$ | $\log_2(d)$ |
Two things to consider for search
- Scalability of the horizontal packing
- Complexity depends on $d$.
Packing Approaches for face Search
- Analysis of the special case of #References = Ciphertext capacity
Face Search using advanced Horizontal packing
- Runs $1:1000$ in $218.18s$
- Homomorphic multiplications are sequentially computed.
Packing scheme
Complexity
| $d \cdot \left\lceil \frac{d \cdot m}{c} \right\rceil$ | $\left\lceil \frac{d \cdot m}{c} \right\rceil$ | $(d-1) \cdot \left\lceil \frac{d \cdot m}{c} \right\rceil$ |
Bootstrapping: needed
HERS: Face Search using Vertical packing
Homomorphic multiplications are computed in parallel.
Packing scheme
Complexity
| $(d-1) \cdot \left\lceil \frac{m}{c} \right\rceil$ | $d \cdot \left\lceil \frac{m}{c} \right\rceil$ | $0$ |
Bootstrapping: not needed
MFIP Search: Face Search with zero homomorphic multiplication
Only homomorphic additions are used.
Packing scheme
Complexity
| $(d-1) \cdot \left\lceil \frac{m}{c} \right\rceil$ | $0$ | $0$ |
Bootstrapping: not needed
Can search efficiency be further improved?
Packing scheme
Bootstrapping: not needed
HERS Complexity
| $(d-1) \cdot \left\lceil \frac{m}{c} \right\rceil$ | $d \cdot \left\lceil \frac{m}{c} \right\rceil$ | $0$ |
MFIP Search Complexity
| $(d-1) \cdot \left\lceil \frac{m}{c} \right\rceil$ | $0$ | $0$ |
Their complexity still depends on $d$.
Dimensionality reduction can further reduce latency
Dimensionality reduction reduces $d$ $\implies$ less homomorphic operations
Effect of the dimensionality reduction on search Accuracy (Rank-1)
Effect of the dimensionality reduction on search CPU runtime
Lookup table-based approach is practical (1:1M in $1.28s$)
Today FHE runs on GPUs
Bringing practical speed to secure computationMFIP Search Speed at scale using a single GPU (A100)
FHE-based search takes $5min$ for a 1 Billion vector gallery with up to 512-dim
Summary of FHE-based face recognition Search solutions
| Solutions | Application | $K$ | $\#M_{HE}$ | $\#R_{HE}$ | $\#A_{HE}$ | Storage | Protected Reference | Protected Probe | Runtime (1:1Million) |
|---|---|---|---|---|---|---|---|---|---|
| SecureFace (linear) | Biometric matching | $m$ | 1 | \(\log_2 \left( \frac{\text{RingDim}}{2} \right)\) | \(\log_2 \left( \frac{\text{RingDim}}{2} \right)\) | \(d \cdot m\) |
Encrypted
|
Encrypted
|
8.67h
|
| MFIPv2 (linear) | Biometric matching | \(m\) | 0 | \(\log_2 \left( \frac{\text{RingDim}}{2} \right)\) | \(\log_2 \left( \frac{\text{RingDim}}{2} \right)\) | \(\left( \sum_{i=1}^{d} N_i \right) \cdot m\) |
Permuted & Encrypted
|
Permuted
|
4.7h
|
| BOK'+22 | Biometric search | \(\left\lceil \frac{d \cdot m}{c} \right\rceil\) | 1 | \(d - 1\) | \(d\) | \(\left\lceil \frac{d \cdot m}{c} \right\rceil\) |
Encrypted
|
Encrypted
|
12.86h
|
| HERS | Biometric search and RAG | \(\left\lceil \frac{m}{c} \right\rceil\) | \(d\) | 0 | \(d - 1\) | \(d \cdot \left\lceil \frac{m}{c} \right\rceil\) |
Encrypted
|
Encrypted
|
38.37s
|
| BHV+'25 | Biometric search and CLIP | \(\left\lceil \frac{m}{c} \right\rceil\) | 0 | 0 | \(d - 1\) | \(\left( \sum_{i=1}^{d} N_i \right) \cdot \left\lceil \frac{m}{c} \right\rceil\) |
Permuted & Encrypted
|
Permuted
|
1.28s
|
Limitations and Next Steps
Template Recovery Attack from Scores
- If the client is corrupted by an adversarial, templates on the database would be stolen!!!
- step 1: generate adversarial samples (features) and send them to the server
- step 2: obtain similarity scores from the server
- step 3: solve a linear system to recover templates
Assumption: Secret Key is compromised.
Protecting Match Scores and Decisions
CryptoFace: End-to-End Encrypted Face Recognition
- If the client is adversarial, it may attempt to infer encrypted features stored on the server.
- The client cannot access to features or networks.
- If the server is adversarial, it may attempt to collect biometric data from the client.
- The server cannot access to the secret key.
CryptoFace: Same Accuracy, 341× Faster on GPU
End-to-end encrypted face recognition
Accuracy
89.42%
Average verification accuracy across five datasets
Preserved across all CPU and GPU versions
LFW · AgeDB · CALFW
CPLFW · CFP-FP
Latency per match pair
Original CPU
1,364s
22.7 minutes
Original paper
Baseline
Updated CPU
36s
Current
implementation
38× faster
Updated GPU
4s
Current
implementation
341× faster
Speedups relative to the original CPU result.
Desired BTP Criterion
- Classical BTP criterion: recognition accuracy, irreversibility, renewability/unlinkability
- Modern BTP methods require new criterion to be considered.
- Provable Guarantee: many BTP schemes provide no proof of privacy/security
- Privacy/Security Level: numerically quantified and guaranteed level of protection
- Computational Efficiency: BTP schemes have different computational costs
- Memory/Storage Efficiency: BTP schemes have different memory costs
- Key Management: cryptographic BTP schemes require key management
HE for Biometric Template Protection
Key Takeaways
- Lattice-based HE provides a post-quantum foundation for template protection.
- Efficient encrypted matching and large-scale search are now practical.
- The next frontier is end-to-end encrypted recognition.
Next Steps
- Explore our open-source code and FHE4CV tutorial.
- Collaborate on secure, efficient biometric recognition.
Thank you — Questions?
Human Analysis Lab · hal.cse.msu.edu
vishnu@msu.edu