“My card was charged twice for the same order.”
Likely intent duplicate chargeEMBEDDINGS · RANKING · CLUSTERING
When everything
looks similar.
A practical guide to a real problem in vector search: some candidates look relevant to almost every query. Learn what “debiasing” can mean, which correction fits which failure, and when a frozen assignment model earns your trust.
No universal debias switch. Several precise tools, each for a different cause.
THE HUB PROBLEM cosine neighbours
01UNDERSTAND THE FAILURE02CHOOSE A CORRECTION03TEST BEFORE FREEZING
A SHORT STORY
A generic answer can win
for the wrong reason.
Imagine an assistant sorting short customer messages. “I can’t pay”, “my card won’t work”, and “where is my refund?” all mention payments. A broad “payment help” prototype can score well against every message, even when a more specific intent is a better match.
Shared words raise scores for card, payment, charge, and refund groups.
Similarity is evidence, not a verdict.These lines are illustrative, not sampled benchmark results. The public BANKING77 dataset below contains online banking queries and intent labels; it is support intent data, not product-review feedback.
FIRST, NAME THE FAILURE
“Bias” can mean
three different things.
Pick the symptom you can measure. A direction shared by vectors, a candidate that wins too often, and a mistaken training negative are not the same failure.
Directions bunch together
Many embeddings occupy a narrow cone or share dominant components. This is often called anisotropy. Mean subtraction or removing top components may help, if the shared directions are nuisance variation.
Representation geometry · inspect firstOne candidate wins everywhere
A point retrieved by many unrelated queries is a hub. This is a neighbourhood / scoring problem. CSLS and local scaling compare a match against its local similarity background.
Ranking geometry · count top-k hitsTraining calls a true match “negative”
Contrastive training can push apart two texts that belong to the same intent. A false negative is a sampling/objective problem. Debiased contrastive methods adjust learning; they do not bolt onto frozen scores.
Training objective · audit pair labelsWords such as “finance” or “manager” pointing in a similar direction do not by themselves prove bias. First decide what the category means: function, industry, seniority, issue type, or customer intent. Then inspect actual errors and neighbourhoods.
FOUR FAMILIES OF TOOLS
Correct the cause.
Keep the useful signal.
Methods that share an intuition can act at very different points in a system. This is the map to the right tool.
WHEREMETHODWHAT CHANGESWHEN TO TRY IT
InferenceLocal score correction
CSLS / local scalingCorrects a candidate that looks similar to many queries.Retrieval or prototype assignment has frequent hubs.
CSLS / local scalingCorrects a candidate that looks similar to many queries.Retrieval or prototype assignment has frequent hubs.
InferenceCluster baseline offset
calibrated prototypesSubtracts a per-cluster background score estimated from reference data.A cluster gets a systematic score boost across unrelated examples.
calibrated prototypesSubtracts a per-cluster background score estimated from reference data.A cluster gets a systematic score boost across unrelated examples.
EmbeddingsCenter / remove directions
All-but-the-Top familyTransforms embeddings, then recomputes similarities consistently.Validation confirms dominant directions obscure task distinctions.
All-but-the-Top familyTransforms embeddings, then recomputes similarities consistently.Validation confirms dominant directions obscure task distinctions.
TrainingDebiased contrastive learning
false-negative correctionChanges which examples the encoder pulls together or pushes apart.You can train and have positives that are mislabeled as negatives.
false-negative correctionChanges which examples the encoder pulls together or pushes apart.You can train and have positives that are mislabeled as negatives.
Before modelTaxonomy + data review
human intent definitionChanges target labels, data balance, or ambiguous examples.“Product domain” blends multiple axes or examples lack enough context.
human intent definitionChanges target labels, data balance, or ambiguous examples.“Product domain” blends multiple axes or examples lack enough context.
THE IMPORTANT GEOMETRY CHECK
For ordinary Euclidean distance, centering both a point and every centroid by the same mean changes nothing:
‖(x − μ) − (ck − μ)‖₂ = ‖x − ck‖₂
For cosine, re-normalising after centering can change angles. Removing directions can change either geometry. The transform must match the distance rule used at inference.
A SMALL, HONEST LAB
What if “generic”
is the whole problem?
For one invented message, raw cosine slightly prefers “payment help”. But that cluster also scores highly for many unrelated messages. A simple background correction asks how much of this score might be generic.
A CANDIDATE BASELINE CORRECTION
adjusted(q,k) = cosine(q,ck) − λ · bk
q new messageck cluster prototypebk average background similarity
This is a hypothesis, not a universal algorithm. The correction strength λ must be selected on held-out validation data. A wider margin does not mean the new ranking is correct.
CSLS: the established local-scaling idea ↗ILLUSTRATIVE NUMBERSTOY · NOT MODEL OUTPUT
NEW SUPPORT MESSAGE
“My employee card payment never arrived.”
INTENT CANDIDATERAWBACKGROUNDADJUSTED
RELATIVE SCORE
illustrative units
illustrative units
The score values and background baselines are invented solely to show the arithmetic. A changed winner is not evidence of improved accuracy. In a real run, estimate backgrounds on reference data and compare raw vs corrected top‑1 accuracy, macro‑F1, calibration, and reject/abstain behaviour.
DISCOVER → MEASURE → FREEZE
Freezing is the
release step.
A frozen model makes category assignment stable. It also preserves mistakes. Check the pipeline before calling the boundary “better”.
Choose one axis
Intent, issue type, job function, industry or seniority. Write examples that should stay together and examples that should separate.
Cluster the corpus
Embed the training corpus. Explore clusters, duplicates, frequent neighbours and the documents nearest each centre. Human-review the taxonomy.
Test corrections
Keep train, validation and test separate. Compare a modern embedding baseline against calibrated prototypes, local scaling, and any justified embedding transform.
Freeze the full recipe
Version the encoder, instructions, preprocessing, centres, corrections, thresholds, taxonomy and reference snapshot together.
TRAIN-TIME ARTEFACTSencoder + text rules + transform + centres + correction + rejection thresholds
→INFERENCEnew message → same recipe → ranked intents, or “uncertain”
Do not silently force every message into a cluster. Choose a rejection rule from validation examples and monitor drift. A new issue that does not fit the frozen taxonomy needs a path to review and a versioned taxonomy update.
THE CONCEPT MAP
Follow the signal
to its failure point.
Choose a node to open a short explanation. Each path ends with a different measurement and intervention.
WHERE THE IDEAS STILL MATTER
The same geometry.
Different stakes.
These ideas remain useful in today’s embedding pipelines. The right proof changes with the task.
Find the customer’s actual problem
Discover emerging themes in short support messages; assign future messages to a reviewed, versioned intent set. Keep “unclear” available.
Measure · macro-F1, NMI/ARI for discovery, coverage vs errorRetrieve evidence, not just generic language
A broadly relevant document can crowd out a specific answer. Test hub frequency and local score calibration in a retrieval pipeline.
Measure · Recall@k, MRR / nDCG, answer supportSeparate shared words from shared work
“Senior analyst” appears in many functions. Define whether the taxonomy describes role function, industry, or seniority before clustering.
Measure · held-out assignment + human reviewMatch across different neighbourhood densities
CSLS was developed for cross-domain word-vector matching. A 2026 preprint revisits hubness in multilingual retrieval; its results motivate testing, not a universal default.
Measure · mutual retrieval / recall on target languagesModern use does not make every historical method universally effective. Results depend on encoder, corpus, labels, normalisation, candidate set, and metric. Model cards and broad embedding leaderboards do not replace a domain-specific test.
A RESEARCH TRAIL, NOT A VICTORY LAP
Old ideas.
Current questions.
A short chronology shows how the topic branches. Each paper answers a narrower question than “does debiasing work?”
- 2010FOUNDATION
Hubs in high-dimensional spaces
Formalises the tendency of some points to become neighbours of many others. The general geometry predates modern transformer embeddings.
Radovanović, Nanopoulos & Ivanović · JMLR ↗ - 2017REPRESENTATION
All-but-the-Top
Subtracts a global mean and removes dominant principal components from word vectors. Evidence on word/sentence representation tasks; not automatic proof for modern job-title encoders.
Mu, Bhat & Viswanath · ICLR 2018 ↗ - 2017–18RANKING
CSLS for cross-lingual retrieval
Corrects similarity using local neighbourhood density. It changes retrieval ranking; it does not alter an encoder’s training or discover clusters.
Conneau et al. · ICLR 2018 ↗ - 2020TRAINING
Debiased contrastive learning
Corrects for same-class examples sampled as negatives during self-supervised training. This “debias” has a distinct meaning from subtracting a cluster baseline at inference.
Chuang et al. · NeurIPS 2020 ↗ - 2022NLP
DCLR for sentence representations
Uses instance weighting and noise-based negatives; evaluates seven semantic-textual-similarity tasks. A useful bridge to NLP, though still not a job-taxonomy result.
Zhou et al. · ACL 2022 ↗ - 2023–24TEXT EMBEDDINGS
Hubness reduction in Sentence-BERT spaces
Study reports reduced hubness and neighbourhood-classifier error when hubness is high. The paper explicitly measures its evaluated pretrained models; it does not guarantee gains on every dataset.
Nielsen & Hansen · NLDL 2024 ↗ - 2024–25PRACTICE
False negatives stay a live training issue
Modern retrieval recipes still mine hard negatives and filter examples that may actually be relevant. Newer is not “solved”: label quality and the target domain matter.
NV-Retriever · 2024 · current Sentence Transformers guide ↗ - 2026CURRENT PREPRINT
Hubness vs anisotropy in multilingual retrieval
A recent arXiv preprint reports hubness, rather than anisotropy, as the main driver of a particular multilingual retrieval asymmetry. Its scope is five encoders and four languages; independent replication and your task’s evaluation still matter.
Sakhawat et al. · arXiv preprint ↗
Why does a 2017 method still appear in a modern guide?
Because the mechanism is still testable, not because old numbers transfer. Use a contemporary encoder as the starting point, diagnose your task’s failure mode, then compare simple post-processing and local ranking methods on held-out examples. Retire a correction if it fails your benchmark.
FIRST HELD-OUT MEASUREMENT
What did the held-out test say?
On five closely related BANKING77 intents, a validation-tuned mean-background offset raised test accuracy by 0.5 percentage points and macro-F1 by 0.52 points. CSLS lowered both by about 0.5 points. That is a small, task-specific result, not proof of a general “debias” improvement.
MEASURED · official test split held outMEASURED · λ selected on validation onlyMEASURED · cosine, mean correction, and CSLSLIMIT · five known intents, not discoveryLIMIT · 200 filtered test messagesNEXT · error review, abstention, and calibration
Reference task: BANKING77 — 77 online-banking query intents, distributed by PolyAI under CC BY 4.0. It benchmarks customer-support intent matching, not consumer product reviews. The selected intent names, split counts, and model details are included with the result. The small product-feedback examples elsewhere are hand-authored teaching examples.
EXPERIMENT STATUS
No held-out measurement is published here yet. The interactive score lab above uses invented arithmetic.
LATER · FINAL STEPOnce the explanation holds up,
Once the explanation holds up,
then make the film.
The eventual video can follow one message through the embedding, the crowded neighbourhood, the correction, the held-out test, and a versioned frozen model. First, make sure this lesson and its experiment are right.
Return to the beginning ↑