Discovery Loop: The Secret Engine Behind Effective Vibe Coding

Vibe coding — a term coined by Andrej Karpathy in February 2025 — has transformed the way a new generation of developers build software. Instead of writing every line manually, you describe the desired behavior in plain English, and an AI assistant translates it into code. Karpathy famously said, "It's not really about the code, it's about seeing the code." But the honeymoon phase wore off quickly. As the first wave of projects matured, developers realized that a single, unstructured prompt rarely yields a production-ready result.

The difference between a chaotic AI coding session and a smooth one is not the model you use. It's the process you wrap around it. At the heart of that process is a technique called the Discovery Loop. In this article, we'll break down what it is, why it works, and how you can implement it in your daily workflow to dramatically improve the quality of your AI-generated code.

What Is a Discovery Loop?

A Discovery Loop is a systematic, iterative cycle of gathering context, generating a candidate solution, testing it, and feeding the results back into the model. It turns a one-shot prompt into a multi-turn conversation, where each turn is informed by the output (and failures) of the previous one.

Think of it as the scientific method for software engineering:

1. Observe — Read the codebase, run existing tests, inspect the environment.
2. Hypothesize — Formulate a design or plan based on evidence.
3. Experiment — Generate code that implements the hypothesis.
4. Test — Run the code against real tests, linters, and user-facing scenarios.
5. Learn — Analyze failures, adjust the hypothesis, and repeat.

The key is that the loop is not blind trial and error. Each iteration increases the model's "awareness" of your project. This mirrors the way human developers work: you don't immediately write a solution; you first explore the code, understand the requirements, and then implement, test, and fix.

Why Naive Prompting Fails

If you've ever asked ChatGPT to "write a login page" and received a generic, insecure mess, you've experienced the context gap. LLMs are trained on public data, not on your private codebase. Without access to your architecture, naming conventions, dependencies, or business logic, any answer is essentially a guess based on statistical similarity.

Naive prompting assumes the model knows what you know. It doesn't. For instance, if you ask for "a function to calculate the total price of an order," the model might forget about discounts, taxes, shipping, and currency conversion — all of which are critical in real business logic. A discovery loop would first ask: "What currency? Do you support multiple tax rates? How are discounts applied?" These questions are not annoying; they are the foundation of a working implementation.

Research supports this view. The Self-Refine paper showed that an LLM can iteratively improve its own output by generating feedback and applying it. Similarly, the ReAct paper demonstrated that interleaving reasoning and acting (e.g., searching for relevant information) dramatically improves performance on complex tasks. And the Reflexion paper took this further by having an agent retry after reading its own error messages. These three papers are the theoretical foundation of the Discovery Loop.

The Anatomy of a Discovery Loop

Let's dive into each phase with concrete examples and practical tips.

1. Observation: Build Context

The goal of observation is to give the AI the same context a new developer on your team would have. This includes:

• The project's directory structure (tree . -L 2).
• A list of key dependencies (cat package.json or pip freeze).
• The relevant source files (grep -R "class User" src/).
• Existing tests and how to run them.

You can prompt the AI to do the exploration itself. Many coding agents can execute terminal commands. If not, you can manually paste the output. For example:

"Here's the project structure and the current auth module. Read the code, then explain how the login flow works."

This phase also involves running the existing test suite to establish a baseline. If tests are already red before you start, you need to fix that first — otherwise, you'll be chasing failures that exist independently of your change.

2. Hypothesis: Plan First

Before generating code, ask for a plan. A good hypothesis includes:

• What files need to change?
• What new functions or classes need to be created?
• What edge cases need to be handled?
• What will the tests look like?

This is the "Chain-of-Thought" idea in practice. Wei et al. (2022) showed that prompting the model to "think step by step" improves reasoning. When you force the hypothesis stage, you are effectively turning on the model's internal monologue.

Example prompt:

"Don't write any code yet. Analyze the auth module and outline a plan for adding password reset. Consider security best practices. Finally, list three possible pitfalls."

The resulting plan is your contract. You can critique it, ask for changes, and only then proceed to generation.

3. Generation: Small and Testable

Once the plan is approved, start generating code in small increments. Instead of asking for an entire module, ask for one function or one test. Each generated snippet should be immediately verifiable.

For example:

"Implement the sendPasswordResetEmail function. Add unit tests for both the success and invalid-token cases. Do not change any other files."

By limiting the scope, you minimize the blast radius of errors. If the AI writes a broken function, you can isolate and fix it without tearing down the whole module. This is also how human engineers work — they commit small, reviewable changes, not giant monoliths.

4. Testing: The Feedback Engine

This is the most critical phase. After generation, run the tests. The results (pass, fail, error messages) are the fuel for the next iteration. Modern AI coding tools can do this automatically, but if you're using a plain chatbot, you need to copy the error output and paste it back.

Example dialogue:

User: "The test suite shows 2 failures. Here's the log: ..."
AI: "The failures are caused by undefined being returned when the user ID is missing. I'll add a null check and regenerate the function."

Notice how the loop is now grounded in observable reality. The model isn't guessing anymore; it's debugging against real data. This is what separates an experienced vibe coder from a beginner.

5. Refinement: Review and Consolidate

When tests pass, the loop isn't over. Ask the AI to review its own code for edge cases, potential security issues, and performance bottlenecks. You can also request a diff and check it manually. Some developers call this the "code review" phase.

For example:

"Review the code you just wrote. Are there any time-of-check to time-of-use (TOCTOU) races? Is the input validation sufficient? Suggest improvements."

This meta-cognitive step helps catch problems that tests might miss. It also builds trust: you're not blindly accepting AI output; you're treating it as a draft that needs editorial review.

A Real-World Case Study: Refactoring a Payment Module

Let me share a real scenario from my own practice. I maintain a small e-commerce platform written in Django. We had a fragile payment module with a lot of duplicated code. I decided to use a discovery loop to refactor it.

Step 1: Observation. I copied the module's file tree and the main PaymentGateway class into the chat. I also ran the existing unit tests, and there were three failing tests that I noted.

Step 2: Hypothesis. I asked the AI to propose a design that consolidates the duplicated logic into a single ChargeProcessor class. It suggested three options: a simple refactor, a strategy pattern, or an adapter pattern. We chose the strategy pattern because the platform supports multiple providers.

Step 3: Generation. I asked the AI to implement ChargeProcessor first, then migrate each provider one by one. Since I limited each generation to a single provider, every iteration was testable.

Step 4: Testing. We ran the tests after each migration. The first migration broke the handling of the card_network field. The error log showed a KeyError. The AI immediately proposed a fix: use .get() with a default value. The next iteration passed.

Step 5: Refinement. After all tests passed, I asked the AI to review the code for PEP 8 compliance and to add a docstring. It also suggested adding a retry mechanism for transient network errors — something I hadn't thought of.

The refactoring took about two hours with the loop. The original estimates from my team were two days. This is a direct, realistic outcome of integrating a discovery loop into an existing codebase.

Discovery Loop vs. Naive Prompting: A Side-by-Side Comparison

The table above summarizes why the loop is so effective. It's not about being smarter than the AI; it's about giving the AI a way to correct its own mistakes.

How to Build Your Own Discovery Loop

Even if your AI tool doesn't have agentic capabilities, you can simulate a discovery loop manually. Here's a simple checklist and some prompt templates to use.

Checklist for each iteration:

• [ ] Define a single, atomic task.
• [ ] Provide relevant context (files, error logs, test output).
• [ ] Ask for a plan before generating code.
• [ ] Generate a small chunk.
• [ ] Run the tests yourself (or ask the AI to).
• [ ] Paste the error output back into the chat.
• [ ] Repeat until tests pass, then ask for a review.

Prompt templates:

• Plan-first: "Before writing code, give me a numbered plan for [task]. Include edge cases and potential failures."
• Test-driven: "Write a unit test for [function] that covers [business rule]. Then implement the function to pass that test."
• Error-driven: "Here is the stack trace from my test run: [...] Explain what's happening and propose a fix."
• Review: "Review the code you just wrote. Look for security flaws, performance issues, and readability problems. Suggest improvements."

Tools That Amplify Discovery Loops

As of 2026, several coding assistants and frameworks support loop-like workflows. GitHub Copilot's "agent mode" and OpenAI's Codex are leading examples. But you can also build your own loop by connecting any LLM to a terminal emulator via an API. For instance, you can write a script that invokes the model, runs the resulting code, and feeds exceptions back into the conversation.

If you're building such a pipeline, you'll likely want to integrate a robust language model. OpenAI's API is still the most widely used, and many orchestration platforms support it. One example is ASI Biont, which helps developers build and manage complex AI workflows. ASI Biont supports connection to OpenAI via API — learn more at asibiont.com/courses. This is especially useful when you want to automate the discovery loop across multiple files and runs.

But remember: the tool is less important than the methodology. You can create a discovery loop in any chat interface by manually following the steps.

Common Pitfalls and How to Avoid Them

Even with a discovery loop, things can go sideways. Here are the most common failure modes I've seen:

1. Infinite loop. The AI keeps producing the same incorrect output. Break the cycle by explicitly asking it to re-read the error and explain what it thinks the root cause is. This re-grounding often dislodges a false assumption.
2. Context overload. Do not paste an entire 10,000-line file into the prompt. The model will drown in irrelevant details. Use targeted grep searches and include only the parts that matter.
3. Blind trust in green tests. Passing tests don't guarantee correctness. The AI can "hack" the test by hardcoding an expected value. Always ask for an explanation of how the code works.
4. Skipping the plan. When you're in a hurry, it's tempting to jump straight to "fix this." Resist the urge. A 30-second plan can save 30 minutes of debugging.
5. Forgetting the baseline. Always run the tests before making any changes. If the suite was already failing, the AI might "fix" unrelated issues, leading to confusion.

The Future of Discovery Loops

The discovery loop is not just a personal technique — it's becoming the default interaction model for AI coding. In 2025, Anthropic and OpenAI both introduced agentic coding features that plan and execute multi-step tasks. By 2026, these capabilities are maturing rapidly. The next evolution will likely include:

• Persistent project memory. AI assistants will remember past discoveries (decisions, mistakes, conventions) and apply them to future tasks. This is the "long-term memory" feature that many tools are beta-testing.
• Test-first generation as a standard. Instead of writing code and then tests, the AI will generate tests from your requirements, see them fail, and then write code to make them pass. This is the ultimate discovery loop: tests are the specification.
• Collaborative multi-agent loops. A "planner" agent, a "worker" agent, and a "reviewer" agent will run their own discovery loops, communicating results to each other. Early frameworks like LangGraph and AutoGen are already exploring this territory.

The underlying principle will remain unchanged: iterate, learn from feedback, and get closer to the target with each pass. The more complex the task, the more valuable the loop becomes.

Conclusion

Discovery Loop is the hidden engine that separates productive vibe coding from random code generation. By systematically observing, hypothesizing, generating, testing, and refining, you can transform a helpful but ignorant AI into a reliable pair programmer. The method is grounded in solid research, proven by thousands of developers, and accessible to anyone — even without fancy agents.

Start using it today. Pick a small task, gather context, ask for a plan, generate a small piece, run the test, and paste the error back. Do this a few times, and you'll see why the phrase "vibe coding" has evolved from a joke to a serious engineering methodology. The vibe is not about trusting the AI; it's about teaching it to see what you see.