KNN Prompting
Not all few-shot examples are created equal. KNN Prompting dynamically selects the most relevant demonstrations for each query using nearest-neighbor retrieval — replacing static example lists with context-aware, similarity-driven selection that adapts to every input.
Introduced: KNN Prompting was published in 2022 by Xu et al. The technique addresses a fundamental limitation of standard few-shot prompting: the examples you choose dramatically affect performance, yet most practitioners select them arbitrarily or use the same fixed set for every query. KNN Prompting formalizes example selection as a retrieval problem — for each new input, it encodes the query into a vector embedding, searches a pool of labeled examples for the k nearest neighbors by cosine similarity, and uses those semantically closest examples as few-shot demonstrations. The result is a prompt that is tailored to the specific characteristics of each individual query.
Modern LLM Status: The core principle of selecting relevant examples based on semantic similarity has become foundational to retrieval-augmented systems and remains highly practical. Modern embedding models like text-embedding-3 and open-source sentence transformers make KNN-style retrieval faster and more accurate than ever. While today’s large models are more robust to example choice than their 2022 predecessors, dynamically selected examples still consistently outperform random ones — especially in specialized domains, low-resource languages, and tasks where the input distribution is highly varied. The principle lives on in every RAG pipeline that retrieves contextually relevant content.
Similar Examples Teach Better
When you show a model how to handle a task, the examples you provide act as implicit instructions. If those examples closely resemble the current query — in structure, topic, vocabulary, or complexity — the model extracts more relevant patterns. A sentiment analysis prompt for a product review about headphones benefits far more from seeing other electronics reviews than from seeing restaurant reviews.
KNN Prompting automates this intuition. Instead of hand-picking examples or cycling through a fixed set, it treats example selection as a search problem. Every candidate example and the incoming query are converted into vector embeddings — dense numerical representations that capture semantic meaning. The system then retrieves the k examples whose embeddings are closest to the query’s embedding, ensuring maximum relevance on every single prompt.
Think of it like a librarian who, instead of handing you the same three reference books regardless of your question, goes into the stacks and pulls the most topically relevant volumes for each specific inquiry you bring.
Vector embeddings compress text into high-dimensional numerical space where semantically similar texts land near each other. “I love this phone” and “This device is amazing” end up close together despite sharing no words, while “The bank was steep” and “The bank approved my loan” land far apart despite sharing a keyword. KNN Prompting leverages this geometric property to find examples that are meaningfully similar — not just superficially similar.
The KNN Prompting Process
Four stages from example pool to dynamically assembled prompt
Build the Example Pool
Curate a diverse collection of labeled input-output pairs that cover the range of your task. Each example should include the input text and its corresponding correct output. The pool should be large enough to contain relevant neighbors for any likely query — hundreds to thousands of examples is typical. Quality matters: noisy or mislabeled examples will propagate errors into every prompt that retrieves them.
For a sentiment classifier, assemble 500+ product reviews each labeled as positive, negative, or neutral — spanning categories like electronics, clothing, food, and software to ensure broad coverage.
Encode into Vector Embeddings
Pass every example in the pool through an embedding model to generate a vector representation for each one. These embeddings are computed once and stored in a vector index for fast retrieval. When a new query arrives, it is also encoded using the same embedding model. The choice of embedding model determines what “similarity” means for your retrieval — domain-specific embeddings often outperform general-purpose ones.
Using a sentence transformer, convert each review into a 768-dimensional vector. Store all vectors in a FAISS index for sub-millisecond retrieval. When a new review arrives, embed it with the same model to produce a query vector.
Retrieve k Nearest Neighbors
Compute the cosine similarity between the query vector and every example vector in the pool, then select the top k most similar examples. The value of k is a tunable parameter — typically 3 to 8, balancing between providing enough context and staying within token limits. Some implementations also apply diversity filters to avoid retrieving near-duplicate examples that waste prompt space.
For a query about wireless earbuds, the retriever returns 5 nearest examples: three headphone reviews, one Bluetooth speaker review, and one smartwatch review — all semantically close to the audio-device domain.
Construct the Prompt
Assemble the final prompt by placing the retrieved examples as few-shot demonstrations followed by the actual query. The examples serve as in-context demonstrations that prime the model with patterns most relevant to the current input. Order matters — placing the most similar example closest to the query often yields the best results, as models tend to attend more strongly to nearby context.
Prompt: Classify the sentiment of each review.
Review: “These headphones have incredible noise cancellation.” Sentiment: Positive
Review: “The Bluetooth keeps disconnecting from my phone.” Sentiment: Negative
Review: “Battery life is about average for this price range.” Sentiment: Neutral
Review: “The sound quality on these wireless earbuds blew me away.” Sentiment:
See the Difference
Why dynamically selected examples outperform fixed ones
Fixed Few-Shot
Example 1: “The pizza was cold and stale.” → Negative
Example 2: “Great customer service!” → Positive
Example 3: “The movie was okay, nothing special.” → Neutral
“The firmware update bricked my router and support was unhelpful.” — The model must generalize from food, service, and movie examples to a technical hardware complaint.
KNN Prompting
Example 1: “After the update, my device won’t even turn on.” → Negative
Example 2: “Tech support resolved my connectivity issue quickly.” → Positive
Example 3: “The new firmware fixed some bugs but introduced others.” → Neutral
“The firmware update bricked my router and support was unhelpful.” — The model sees closely related tech-support examples and classifies with high confidence.
Practice Responsible AI
Always verify AI-generated content before use. AI systems can produce confident but incorrect responses. When using AI professionally, transparent disclosure is both best practice and increasingly a legal requirement.
48 US states now require AI transparency in key areas. Critical thinking remains your strongest tool against misinformation.
KNN Prompting in Action
See how dynamic example retrieval improves performance across domains
“The noise cancellation on these earbuds is phenomenal, but the charging case feels cheap and flimsy.”
Retrieved Example 1 (similarity: 0.94):
“Sound quality is top-tier for the price, but the build quality leaves a lot to be desired.” → Mixed
Retrieved Example 2 (similarity: 0.91):
“ANC works great on the train, though the ear tips don’t stay in well during workouts.” → Mixed
Retrieved Example 3 (similarity: 0.89):
“Battery life is excellent but the Bluetooth connection drops every few minutes.” → Mixed
Model Output: Mixed — The model correctly identifies the positive-negative structure because all retrieved examples demonstrate the same “good feature, bad feature” pattern in the audio-device domain.
“Write a REST endpoint that accepts a user ID and returns their order history with pagination.”
Retrieved Example 1:
“Write a REST endpoint that takes a customer ID and returns their purchase records.” → GET /customers/:id/purchases with query params for limit and offset, including error handling for invalid IDs.
Retrieved Example 2:
“Create an API route that fetches paginated transaction data for a given account.” → GET /accounts/:id/transactions with cursor-based pagination, returning next_cursor and has_more fields.
Retrieved Example 3:
“Build an endpoint to retrieve a user’s activity log with filtering by date range.” → GET /users/:id/activity with start_date and end_date params, paginated response with total_count.
Result: The model generates a well-structured paginated endpoint because it saw three closely related parameterized data-retrieval examples — far more useful than generic CRUD examples would have been.
“Patient reports sudden onset of severe headache, stiff neck, and sensitivity to light. No history of migraines.”
Retrieved Example 1 (similarity: 0.93):
“Patient presents with acute headache, neck rigidity, and photophobia following a two-day fever.” → Urgent: Possible meningitis. Immediate evaluation required.
Retrieved Example 2 (similarity: 0.88):
“Sudden severe headache described as the worst of their life, with neck stiffness.” → Emergency: Rule out subarachnoid hemorrhage. CT and lumbar puncture recommended.
Retrieved Example 3 (similarity: 0.85):
“Patient with fever, headache, and photophobia. Recent travel to endemic region.” → Urgent: Screen for infectious causes including meningitis and encephalitis.
Result: The model correctly triages as urgent/emergency because all retrieved examples share the same symptom cluster and demonstrate appropriate escalation. Random examples about ankle sprains or skin rashes would have provided no useful signal for this critical assessment.
When to Use KNN Prompting
Best when your task has diverse inputs and a rich example pool
Perfect For
When queries span many topics, styles, or domains — no single fixed example set can cover the full range, but KNN retrieval adapts to each input automatically.
Sentiment analysis, intent detection, topic categorization — any classification task where performance depends heavily on seeing relevant labeled examples in context.
Legal, medical, scientific, or technical tasks where domain-specific examples dramatically outperform generic demonstrations from unrelated fields.
When you have hundreds or thousands of labeled examples but can only fit a handful into any single prompt — KNN retrieval makes the entire pool accessible on demand.
Skip It When
If all your queries are structurally identical — same format, same domain, same complexity — fixed examples work just as well without the retrieval overhead.
KNN Prompting requires a pre-built pool of labeled input-output pairs. If you have no labeled data or only a handful of examples, the retrieval step adds complexity with no benefit.
The embedding and retrieval step adds latency before the LLM call even begins. For real-time applications where every millisecond counts, the overhead may be prohibitive.
Use Cases
Where KNN Prompting delivers the most value
Customer Support Routing
Classify incoming support tickets by retrieving examples from similar past tickets, ensuring the model sees domain-relevant intent patterns for accurate routing to the right team.
Document Classification
Categorize legal documents, research papers, or compliance reports by matching each incoming document against a pool of pre-labeled examples from the same regulatory or academic domain.
Multilingual Tasks
Retrieve examples in the same language or language family as the query, ensuring the model sees structurally relevant demonstrations rather than unrelated language patterns.
Code Translation
When converting code between languages or frameworks, retrieve translation examples that use similar APIs, patterns, or data structures to the specific code being translated.
Clinical Note Processing
Extract structured data from clinical notes by retrieving similar notes from the same specialty, ensuring the model recognizes domain-specific terminology and documentation conventions.
Threat Intelligence
Classify security alerts by retrieving examples from similar attack vectors, network patterns, or vulnerability types — ensuring the model applies the correct threat categorization framework.
Where KNN Prompting Fits
Bridging static few-shot learning and full retrieval-augmented generation
KNN Prompting occupies a practical sweet spot on the retrieval spectrum. It is simpler to implement than full RAG pipelines — you need only an embedding model and a vector index, not a complete document-chunking infrastructure. Yet it captures most of the performance gains that come from context relevance. For classification, structured output, and domain-specific tasks, KNN Prompting often delivers RAG-level improvements with significantly less engineering overhead.
Related Techniques
Explore complementary example selection and retrieval techniques
Select Smarter Examples
Start building retrieval-enhanced prompts with dynamically selected examples or explore our full framework library.