Introduction
Retrieval-Augmented Generation (RAG) systems combine the power of large language models (LLMs) with external knowledge bases to produce grounded, up-to-date answers. Since the landmark paper by Lewis et al. (2020), RAG has become the backbone of enterprise AI applications. However, crafting effective prompts for each stage—indexing, retrieval, and generation—remains a subtle art. This article presents 15 carefully designed prompts, organized by complexity (basic, advanced, expert), with concrete code examples and real-world use cases. Each prompt is battle-tested in production environments and references key research such as Dense Passage Retrieval (Karpukhin et al., 2020) and Sentence-BERT (Reimers & Gurevych, 2019).
Basic Prompts (Starter Level)
1. Task: Split long documents into semantic chunks
Prompt: "Split the following text into coherent paragraphs. Each paragraph should be a self-contained unit of meaning, roughly 200–300 words. Use the delimiter [CHUNK] between chunks."
Example Result:
Input: [long article on climate change]
Output:
The greenhouse effect is a natural process... [CHUNK]
Human activities have accelerated warming... [CHUNK]
Mitigation strategies include carbon pricing...
Why it works: Semantic chunking preserves context, improving retrieval accuracy. Source: LangChain's RecursiveCharacterTextSplitter.
2. Task: Generate embeddings for each chunk
Prompt: "Convert each of the following text chunks into a 768-dimensional vector using the sentence-transformers model 'all-MiniLM-L6-v2'. Return the embeddings as a JSON list."
Example Result: [[0.024, -0.101, ..., 0.087], ...] — ready for insertion into FAISS or Pinecone.
3. Task: Create a hybrid search query (dense + sparse)
Prompt: "Generate a dense vector embedding and a sparse BM25 query for the user question: 'What are the side effects of aspirin?' Return both in a single JSON object."
Example Result: {"dense": [0.12, -0.45, ...], "sparse": "side effects aspirin dosage"}.
Advanced Prompts (Intermediate Level)
4. Task: Re-rank retrieved passages using cross-encoder
Prompt: "Given the query and the top-5 retrieved passages below, assign a relevance score from 0 to 1 to each passage using a cross-encoder model. Output scores in descending order."
Example Result: Passage 3: 0.92, Passage 1: 0.85, Passage 5: 0.71, ...
5. Task: Generate context-aware answer with citations
Prompt: "Answer the user question using only the provided context. For each sentence in your answer, cite the source passage number in brackets [1], [2], etc. If no context supports a claim, say 'The documents do not contain this information.'"
Example Result: "Aspirin can cause gastrointestinal bleeding [1]. It is not recommended for children under 12 [2]."
6. Task: Adapt chunk size dynamically based on content type
Prompt: "Classify the following document as 'legal', 'scientific', or 'conversational'. Then split it into chunks of size 500 tokens for legal, 200 for scientific, and 300 for conversational. Overlap 10%."
Example Result: Legal document → chunk size 500, overlap 50 tokens.
7. Task: Build a self-querying retriever for time-filtered results
Prompt: "From the user query 'What regulations were updated in 2024?', extract the time filter 'after 2024-01-01' and the free-text search terms. Return a JSON with 'date_filter' and 'text_query'."
Example Result: {"date_filter": {"gte": "2024-01-01"}, "text_query": "regulations updated"}.
8. Task: Generate synthetic queries for document chunk augmentation
Prompt: "For each of the provided document chunks, generate 3 diverse questions that this chunk could answer. Use the format: 'Chunk ID: [id] | Question: [question]'."
Example Result: "Chunk ID: 4 | Question: What is the dosage of ibuprofen for adults?" — useful for building training data for retriever models.
Expert Prompts (Production Level)
9. Task: Multi-hop retrieval across heterogeneous data sources
Prompt: "The user asks: 'How did COVID-19 affect supply chains in Southeast Asia?'. Retrieve info from 1) a database of economic reports, 2) a vector store of news articles, and 3) a graph of company relationships. Merge results into a single context with source labels."
Example Result: A fused context paragraph with tags like [economic_report] and [news_article].
10. Task: Implement query rewriting for iterative retrieval
Prompt: "Based on the original query and the initial retrieved passages, rewrite the query to be more specific. Output the new query only."
Example Result: Original: "electric car batteries" → Rewritten: "lithium-ion battery degradation in electric cars: lifecycle and recycling methods".
11. Task: Generate a structured summary of retrieved passages
Prompt: "Given the top-10 retrieved passages about machine learning, produce a markdown table with columns: 'Topic', 'Key Findings', 'Source Passage ID'. Group by topic."
Example Result:
| Topic | Key Findings | Source ID |
|---|---|---|
| Transformers | Self-attention enables parallel processing | 5 |
| Training | Batch normalization stabilizes deep nets | 8 |
12. Task: Anomaly detection in retrieved relevance scores
Prompt: "You are a quality monitor. Given a list of queries and their top-5 retrieval scores, flag any query where the score drops sharply (more than 0.3) between the first and second result. Output the flagged queries."
Example Result: ["What is the capital of Australia?"] — indicates poor retrieval diversity.
13. Task: Prompt-based index maintenance (auto-chunking trigger)
Prompt: "Monitor the average cosine similarity between consecutive chunks in the vector store. If the similarity drops below 0.5 for more than 10% of adjacent chunks, output 'REINDEX' with the affected range."
Example Result: REINDEX: chunks 34–47 (low similarity suggests boundary misalignment).
14. Task: Generate adversarial test cases for retrieval robustness
Prompt: "Create 5 adversarial queries that a naive RAG system would answer incorrectly (e.g., ambiguous pronouns, false premises, out-of-domain terms). For each, provide the correct answer and a short explanation."
Example Result: Query: "It caused the accident" (no antecedent) → Correct: "Ask for clarification; the context is missing."
15. Task: Optimize prompt template for temperature and top-p sampling
Prompt: "Given the following 10 generated answers for the same question, compute the average token-level entropy and the proportion of repeated n-grams. Suggest optimal temperature (0.1–1.0) and top-p (0.8–0.95) to reduce hallucination."
Example Result: Recommended: temperature=0.3, top-p=0.9 (current entropy 1.2 is high; target <0.8).
Conclusion
RAG systems are only as good as the prompts that guide them. From basic chunking to expert-level adversarial testing, the 15 prompts above cover the full lifecycle of a robust RAG pipeline. Start with the basic prompts to grasp the fundamentals, then advance to production-ready techniques as your system scales. Remember to benchmark each stage—use tools like RAGAS or TruLens to measure retrieval precision and generation faithfulness. Now go ahead and experiment with these prompts in your own vector stores and LLM chains. The best RAG system is the one you continuously refine.
References:
- Lewis, P., et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS.
- Karpukhin, V., et al. (2020). Dense Passage Retrieval for Open-Domain Question Answering. EMNLP.
- Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. EMNLP.
Comments