18 Prompts for Working with LLMs: Fine-Tuning, RAG, and Prompt Engineering

18 Prompts for Working with LLMs: Fine-Tuning, RAG, and Prompt Engineering

Large Language Models are extremely powerful, but their output quality depends on how you talk to them. In the rush to build AI-powered features, many teams start with a single, naive prompt and then blame the model when the results are inconsistent. After years of shipping production LLM systems, I've learned that a structured prompt library is the single most important asset you can create. This article contains 18 field-tested prompts that cover the entire spectrum of LLM interaction: prompt engineering, Retrieval-Augmented Generation (RAG), and fine-tuning.

You don't need to be a machine learning researcher to benefit from this list. If you are a backend developer, you'll learn how to enforce structured outputs and avoid hallucinations. If you are a data scientist, you'll discover prompts for building and evaluating fine-tuning datasets. If you're a technical writer or product manager, you'll understand how to design system prompts and user flows that keep model behavior predictable.

I've organized the prompts by difficulty: basic, advanced, and expert. The basic prompts are universal and will work with GPT, Claude, Llama, Mistral, and most other modern LLMs. The advanced prompts assume you have a retrieval pipeline in place. The expert prompts are designed for anyone fine-tuning open-source models. For each prompt, I include the task, the exact prompt, an example result, and a pro tip for adapting it to your own needs.

Why Prompts Are the Hidden Interface

Think of an LLM as a probability distribution over tokens. A prompt is not just a query — it's a probability distribution over distributions. Every word you choose shifts the output distribution. This is why the same model can appear brilliant or mediocre depending on the prompt.

Prompt engineering is the practice of deliberately shaping that distribution. But prompts are also the primary interface for two more advanced techniques: RAG and fine-tuning. RAG uses a system prompt to inject external documents into the context, grounding the model in facts it did not learn during training. Fine-tuning uses curated prompt-response pairs to teach the model new abilities directly in its weights.

The key insight is that these techniques are not competitors. They operate on different layers. Prompt engineering is the cheap, reversible layer. RAG is the dynamic context layer. Fine-tuning is the permanent behavior layer. You should use them in that order of investment: first improve your prompts, then add retrieval, and only then consider fine-tuning.

The Three Pillars of LLM Optimization

Let's compare these techniques to understand where each prompt type fits.

The prompts below are organized around these pillars, but they cross-pollinate. An expert prompt from the fine-tuning section is also useful for evaluating a RAG pipeline.

Basic Prompts: Prompt Engineering Fundamentals

These are the first prompts I test when experimenting with a new model. They focus on role adoption, format control, and reasoning. Master these before moving to the advanced sections.

1. Task: Role prompting for domain expertise

Role prompting forces the model to adopt a persona with the right vocabulary, tone, and reasoning framework. It prevents the model from responding with generic platitudes.

When to use: You need an answer from a specific professional perspective, e.g., a legal review, a medical opinion, or a senior developer critique.

Prompt:

Act as a senior data scientist with 10 years of experience in experimental design.
Critique the following research plan for a causal inference study:
[insert research plan]

Example result:
"Your plan has a strong setup, but the lack of a sharp regression discontinuity around the treatment cutoff will bias the Local Average Treatment Effect. I recommend splitting the sample by intent-to-treat and running a fuzzy RD design instead."

Pro tip: Add a sentence like "Ask me clarification questions before critiquing" to force the model to gather context first.

2. Task: Few-shot prompting for consistent formatting

When to use: You need to enforce a strict classification or extraction format with minimal instructions.

Prompt:

Classify each customer complaint as BUG or FEATURE. Examples:
- "App crashes when I tap upload" → BUG
- "Can you add a dark mode?" → FEATURE
- "Photos are blurry after the latest update" →

Example result:
BUG — the model correctly recognizes that a regression is a bug, even though phrased as a complaint.

Pro tip: Use 2-3 examples that cover edge cases, not just the obvious ones. This calibrates the model for ambiguity.

3. Task: Chain-of-thought for complex reasoning

When to use: Any task that requires multiple arithmetic or logical operations. The model is forced to externalize its reasoning, making errors easier to catch.

Prompt:

Solve this problem step by step before giving the final answer.
A client buys 120 necklaces at $8 each, sells 70% of them at $15, and the rest at $5.
What is the profit? Show your reasoning.

Example result:
Step 1: Cost = 120 × 8 = $960
Step 2: Sold at $15 = 84 items, revenue = $1260
Step 3: Remaining 36 items at $5, revenue = $180
Total revenue = $1440, profit = $480.

Pro tip: For older or smaller models, add "Start with 'Let's think step by step'." For models that over-reason, set a step limit.

4. Task: Strict JSON output

When to use: You are integrating the LLM into a programmatic pipeline where a single parser needs reliable structured data.

Prompt:

Return a valid JSON object with the following schema, no commentary:
{
"tone": "positive"

| "neutral" | "negative",
  "confidence": 0-1,
  "summary": "string"
}
Analyze the text: "The update broke everything, but support was helpful."

Example result:
{"tone": "neutral", "confidence": 0.8, "summary": "The update caused issues, but the support team provided good service."}

Pro tip: Add a fallback: "If the text is invalid, return a JSON object with confidence 0 and tone neutral." This prevents schema-breaking outputs.

5. Task: Succinct summarization with constraints

When to use: Meeting notes, customer call summaries, and news digests where brevity is critical.

Prompt:

Summarize the following meeting transcript in exactly 3 bullet points.
Use one line per bullet. Focus on decisions, not discussion.
[transcript]

Example result:
- Decision: launch beta on June 1st
- Action: team to finalize the auth flow by May 20
- Risk: migration from v1 API needs extra QA

Pro tip: Add a constraint such as "Do not use more than 12 words per bullet" to control length.

6. Task: Style transfer

When to use: You need to adapt onboarding messages, error messages, or documentation copy to different audiences.

Prompt:

Rewrite this API error message for a non-technical user.
Keep it friendly and under 30 words.
Error: "401 Unauthorized: token expired"

Example result:
"Your session has timed out for security reasons. Please log in again to continue."

Pro tip: Provide two versions in the prompt — one for a mobile app and one for email — to get localized copy.

Advanced Prompts: RAG and Grounded Generation

RAG is about grounding. The following prompts are designed for production systems where the model must answer from documents, not from its latent memory.

7. Task: Context-grounded answering with constraints

When to use: Every RAG pipeline needs this baseline prompt. It prevents hallucination by setting an explicit "I don't know" condition.

Prompt:

Answer the question using ONLY the <context> tags.
If the answer is not contained in the context, reply "I don't know."
<context>
...chunk of retrieved documents...
</context>
Question: [question]

Example result:
If the context mentions that the refund policy is 30 days, the model answers: "Refunds are available within 30 days of purchase." Otherwise, it responds: "I don't know."

Pro tip: Use XML tags for context, not Markdown, because the tags create an unambiguous boundary in the token stream.

8. Task: System prompt for customer support RAG

When to use: A production support bot with a fixed policy chain of command.

Prompt:

You are the support bot for Acme Corp.
Follow these rules:
1. Use only the provided knowledge base.
2. Quoting the knowledge base is encouraged.
3. Never mention "knowledge base" to the customer.
4. If you don't know, offer a human handoff.

Knowledge base:
[relevant document chunks]

Customer question: [question]

Example result:
A courteous response with inline references to return policies, steps to troubleshoot, and a handoff flow when the answer is not in the knowledge base.

Pro tip: Add a rule that the bot must start with "I understand you're dealing with..." to soften the interaction.

9. Task: Answer with citations

When to use: Legal, financial, or medical use cases where you need traceable claims and automatic fact-checking.

Prompt:

Answer the question with a short explanation. Include inline citations [1] [2]
matching the reference list below. If no reference supports a claim, say "unsupported."
References:
1. ...
2. ...
Question: ...

Example result:
"According to the 2025 security report, the vulnerability had a 9.1 CVSS score [1]. The claim that it was exploited in April is unsupported."

Pro tip: For source-level refutation, ask the evaluation prompt to compare the citations to the source list after generation. This catches fabricated references.

10. Task: Query expansion for retrieval

When to use: When your RAG retriever misses relevant documents due to paraphrase gaps or vocabulary mismatches.

Prompt:

Given the user question, generate 3 alternative queries that preserve the intent but use different terms to improve retrieval recall.
Question: "How does the update affect battery drain on mobile?"

Example result:
- "Battery performance changes in the mobile update"
- "New update power consumption issues"
- "Mobile app update and battery life regression"

Pro tip: Embed these queries separately, then merge the retrieved documents and deduplicate before passing them to the context prompt.

11. Task: RAG reranking

When to use: Your initial retriever (e.g., cosine similarity) produces a coarse top-k; you have one LLM call to burn on finer ranking.

Prompt:

The following are retrieved passages for the question. Rerank them from most to least relevant. Output only the indices, separated by commas.
Question: ...
Passages:
1. ...
2. ...
3. ...

Example result:
2, 1, 3 — the model assessed semantic relevance and placed the most specific passage first.

Pro tip: Use a cross-encoder for reranking when you need CPU efficiency; the LLM-based reranker is more accurate but slower.

12. Task: Hallucination self-check

When to use: As an online guardrail after generation, especially for high-risk verticals like healthcare and finance.

Prompt:

Answer the question based on the context. Next, review your answer and check whether every claim is supported by the context. List any unsupported claims.
Context: ...
Question: ...

Example result:
"Answer: The policy is applied retroactively. Unsupported claims: The policy is applied retroactively — the context only says 'since 2024', not retroactive."

Pro tip: Run this self-check on a small test set first to calibrate whether the model is overconfident about its own errors.

Expert Prompts: Fine-Tuning and Dataset Curation

Fine-tuning requires not just training data, but systematic prompts to create, clean, and evaluate that data. These prompts are the output of my consulting practice.

13. Task: Generating training examples for a fine-tune

When to use: You need a high-quality dataset for supervised fine-tuning, but your logs contain only a few hundred real examples.

Prompt:

Generate 20 diverse training examples for a support intent classifier.
Each example must include: "text" and "intent" where intent is one of [refund, shipping, account, complaint].
Use varied phrasings, emotions, and typos to reflect real users.
Return examples as a JSON array.

Example result:
[{"text": "can I get my money back its been 2 weeks", "intent": "refund"}, ...]

Pro tip: Use your production logs to seed a few examples. The LLM will generate more variations around your real user distribution.

14. Task: Synthetic data generation with constraints

When to use: You need a stress-test dataset for a conversational AI that must handle edge cases and avoid harmful outputs.

Prompt:

Create a synthetic dataset of 50 customer-chatbot interactions for a banking assistant.
Include edge cases: missing account number, angry users, ambiguous utterances.
For each turn, label whether the bot's response was correct or harmful.
Format as CSV with columns: user, bot, label.

Example result:
"I want to close my account","I can help you close your account. What's the reason?",correct — plus rows with missing values, causing the model to suggest invalid actions.

Pro tip: Validate synthetic data by running a "difficulty score" prompt that rates each example on ambiguity, toxicity, and PII risk. Filter low-quality examples before training.

15. Task: Evaluating a fine-tuned model against a gold answer

When to use: After an epoch or two of fine-tuning, you need an automated LLM judge to score the model on a held-out set.

Prompt:

You are an evaluator. Compare the reference and model output.
Assign a score from 1 to 5 for factual accuracy, completeness, and format compliance.
Explain any score below 4.
Reference: ...
Model output: ...

Example result:
"Accuracy: 5, Completeness: 2, Format: 4. The model omitted the second step of the refund process."

Pro tip: Use a different LLM as the evaluator than the one you fine-tuned. This reduces self-evaluation bias.

16. Task: Prompt distillation into a system prompt

When to use: You have a long, verbose instruction that works, but you want a shorter, cheaper system prompt for production.

Prompt:

My detailed instruction is below. Distill it into a concise system prompt of no more than 100 words.
Keep all constraints, but remove redundancy.
Detailed instruction: [long text]

Example result:
"You are a product manager assistant. Reply in bullet points, maximum 5 items, no preamble. Prioritize decisions over discussion. Ask one clarifying question if ambiguous."

Pro tip: After distillation, compare a few outputs of the long prompt vs the distilled system prompt to measure information loss.

17. Task: Error taxonomy analysis for fine-tuning data

When to use: You have a batch of model errors and want to prioritize which failure mode to fix first by retraining or prompt changes.

Prompt:

Use this error taxonomy to classify the following model errors: HALLUCINATION, MISSING_FACT, WRONG_FORMAT, OVERREFUSAL, OFF_TOPIC.
Model errors:
1. ...
2. ...
Output a JSON array with error id and category.

Example result:
[{"model_error": "am I allowed to use VPN?", "category": "OVERREFUSAL"}, ...] — often catches refusal errors that simple metrics miss.

Pro tip: Feed the aggregated error counts back into your prompt version history to know which iteration introduced which regression.

18. Task: Choosing between RAG and fine-tuning

When to use: At the start of a new project, when you need an evidence-based decision on where to invest engineering effort.

Prompt:

Given the following project requirements, recommend a strategy: RAG, fine-tuning, or a hybrid.
List the risks and the one experiment you would run first.
Requirements: ...

Example result:
"If your data changes quarterly, RAG is better because fine-tuning would require retraining. Hybrid is recommended if you need a specialized response style. First experiment: evaluate a naive RAG baseline on 100 queries."

Pro tip: Use the same prompt on multiple LLMs to test how their biases affect your strategic decision. Sometimes the model recommends the technique it "sees" more often in its training data.

How to Choose the Right Prompt Strategy

Use the following decision flow to decide which section to invest in.

Common Pitfalls to Avoid

• Using a huge system prompt that causes the model to forget instructions. Keep system prompts under 500 tokens if possible.
• Putting all retrieved chunks in a single context. Use prompting to select the top 3-5 chunks.
• Fine-tuning on 100 examples and expecting miracles. Fine-tuning works best with 1,000-10,000 high-quality examples.
• Not versioning your prompts. Treat them like code.

Conclusion

Mastering LLMs is not about memorizing a few tricks — it's about building a toolkit of prompts that spans the entire model lifecycle. The 18 prompts above cover prompt engineering, RAG, and fine-tuning, giving you a practical starting point for almost any LLM task.

I recommend you pick one prompt from each category today and apply it to a real problem. Track the outputs, iterate, and watch your model go from generic to production-grade.

If you'd like a custom prompt library for your project, or help setting up a fine-tuning pipeline, leave a comment below or contact us at asibiont.com. We build practical AI solutions that actually work in production.