How I Taught a Neural Network to Draw in Excalidraw: An Agentic Cycle in 100 Lines

In July 2026, a developer published a fascinating experiment on Habr: they built an AI agent that can generate drawings in Excalidraw using a neural network, all within just 100 lines of code. This isn't about a simple image generator pasting a picture into a canvas—it's a true agentic cycle: the AI interprets a prompt, creates a drawing, evaluates its result, and iteratively improves it. The approach leverages language models (like GPT-4 or similar) to produce structured code that Excalidraw can render, opening new possibilities for prototyping, diagramming, and creative collaboration. Let's break down how this works and what it means for developers and designers.

The Core Idea: An Agentic Drawing Cycle

The key innovation is replacing static image generation with an iterative, self-correcting loop. Instead of asking a model to output a final image, the agent: 1) receives a user description (e.g., "draw a cat sitting on a table"), 2) generates a set of drawing commands in a format Excalidraw understands (typically JSON describing shapes, lines, and text), 3) renders it in the canvas, 4) uses the same or another model to critique the result (e.g., "the cat has no tail, the table is missing a leg"), and 5) repeats the cycle until the quality meets a threshold.

This is implemented in roughly 100 lines of Python (or Node.js) code, calling an LLM API and the Excalidraw API. The agent doesn't need training—it uses pretrained models and simple prompt engineering. The result is a dynamic drawing that can improve over seconds.

Practical Implementation Steps

To replicate this, you need three components: an LLM (like GPT-4o, Claude, or Gemini), the Excalidraw API (available via their open-source editor or integration), and a simple loop. Here's a simplified pseudocode outline:

import openai, json, excalidraw_api

def draw_agent(prompt):
    history = []
    for i in range(5):  # max 5 iterations
        # Step 1: Generate drawing commands
        response = openai.ChatCompletion.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": f"Generate Excalidraw JSON for: {prompt}. Improve based on previous feedback: {history}"}]
        )
        drawing = json.loads(response.choices[0].message.content)

        # Step 2: Render in Excalidraw
        excalidraw_api.render(drawing)

        # Step 3: Evaluate result
        critique = openai.ChatCompletion.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": f"Critique this drawing JSON: {json.dumps(drawing)}. What's missing?"}]
        )
        history.append(critique.choices[0].message.content)

        # Step 4: Check quality (simple heuristic)
        if "perfect" in critique.choices[0].message.content.lower():
            break
    return drawing

This loop can be extended with more sophisticated checks, such as counting objects or validating geometric consistency.

Real-World Use Cases

This approach is not just a tech demo. It has immediate practical applications:
- Rapid Prototyping: Product managers can describe a UI layout, and the agent iteratively draws it in Excalidraw, refining based on feedback.
- Educational Diagrams: Teachers can generate science diagrams (e.g., cell structure) by describing components, and the agent ensures all parts are present.
- Brainstorming Aid: Teams can verbally describe a process flow, and the agent creates a visual representation, which they can edit further.

For example, a developer at a startup used a similar agent to generate wireframes for a mobile app. They described screens like "a login page with email and password fields, a forgot password link, and a sign-up button." The agent produced a rough but functional diagram in under 10 seconds, saving hours of manual drawing.

ASI Biont supports connecting to Excalidraw via API for automated workflow integrations—learn more on asibiont.com/courses. This allows enterprises to embed similar agentic loops into their own tools.

Why 100 Lines Matters

The 100-line constraint forces efficiency. It means the logic is lightweight, easy to understand, and deployable on serverless functions or edge devices. This contrasts with heavy, monolithic AI systems. The code relies on existing APIs and minimal boilerplate, making it accessible to intermediate developers.

Limitations and Future Directions

Current limitations include: the agent can get stuck in loops (e.g., over-correcting), the quality of drawings is limited by the LLM's ability to generate precise coordinates, and complex scenes (e.g., 3D perspectives) are challenging. However, with multimodal models (which can "see" the drawing), the critique step becomes more accurate. In 2026, models like Gemini 2.0 already support image understanding, so the agent can visually inspect its output.

Conclusion

Teaching a neural network to draw in Excalidraw with an agentic cycle is a practical example of how small, focused AI systems can automate creative tasks. The 100-line implementation demystifies agentic loops and shows that you don't need massive infrastructure to build useful AI tools. Whether you're a developer automating diagram generation or a designer exploring AI collaboration, this approach is worth experimenting with. The source code and detailed walkthrough are available on Habr, and the community is already extending it with new features like style transfer and real-time collaboration.

Source