Why I Built an Open-Source AST Code Scanner to Insure My AI-Generated Code

Why I Built an Open-Source AST Code Scanner to Insure My AI-Generated Code

As AI coding assistants become the default for many developers, we’ve entered the era of "vibe coding" — letting the model write the majority of code while we review the big picture. That’s fun and fast, but it comes with a hidden tax: AI-generated code often looks plausible and passes unit tests, while hiding subtle logic errors, security vulnerabilities, and style violations that can bite you in production. I learned this the hard way when a seemingly harmless AI-written function caused a data leak in a client project.

Instead of giving up on AI assistants, I decided to build an open-source code scanner that parses source code into an Abstract Syntax Tree (AST) and applies a set of heuristic rules to detect the most common issues in AI-generated code. The scanner is not another linter — it’s a focused tool that understands the structural patterns that LLMs tend to repeat. In this article, I’ll explain why I built it, how it works, and how you can use it to make your AI-assisted workflow safer.

Why AST matters

An Abstract Syntax Tree (AST) is a tree representation of your code’s structure. Unlike regex, which sees code as plain text, an AST reveals the true hierarchy of statements, expressions, and assignments. This makes it possible to detect issues that are invisible to text-based tools. For example, a simple function that uses eval on user input can be detected not just by matching the string eval(, but by analyzing the context and following the data flow to confirm the argument is untrusted.

Additionally, ASTs enable cross-function analysis. When you see a variable that is user-supplied and then passed to a dangerous sink, you can trace the path through multiple nodes. Regex just can't do that without a full dataflow engine, which is exactly what makes AST-based tools like this one different.

Most modern linters and static analysis tools rely on ASTs, but their parsing backends are often tightly coupled to their generic rule language. For my scanner, I chose tree-sitter because it is fast, incremental, and supports a wide range of languages through maintainable grammars. It is also easy to embed in your own tooling, which is exactly what I needed.

The problem with AI-generated code

I have spent the last year reviewing code produced by several large language models. The issues that kept appearing were surprisingly consistent. While each model has its own quirks, they all share common failure modes:

• Unnecessary complexity — functions that do far more than their name suggests, with deeply nested conditionals.
• Poor error handling — code that assumes an operation always succeeds, leaving the underlying exception unhandled.
• Security anti-patterns — calls to eval(), child_process.exec(), or raw SQL with unescaped values.
• Resource leaks — unclosed file handles, database connections, or HTTP clients.
• Hallucinated imports — importing modules that don’t exist in the project’s environment.

These are not new problems, but the frequency is different. Developers who write code by themselves usually stop to think about the error path. AI models, in contrast, tend to generate the optimistic path and leave error handling to the developer. The result is code that passes the happy-path tests but fails under real-world input.

Consider this function that the assistant happily produced:

def load_config(path):
    with open(path) as f:
        return yaml.safe_load(f.read())

This looks fine until path is None or points to a non-existent file. A human would likely wrap the file read in a try/except. The AI sometimes does, but often doesn't.

Another common pattern in JavaScript:

app.get('/user', (req, res) => {
    const query = "SELECT * FROM users WHERE id = " + req.query.id;
    db.query(query, (err, rows) => {
        res.json(rows);
    });
});

This is a textbook SQL injection, yet I saw this exact pattern generated by multiple models.

My scanner: an AST-based check engine

The scanner is a Python package that I named ast-code-scanner. It currently supports Python, JavaScript, and TypeScript — the languages that make up the majority of my work. The architecture is straightforward:

1. Parse the source file into an AST using tree-sitter.
2. Walk the tree, starting at the root.
3. For each node, call every registered rule to check if it matches the node.
4. Collect diagnostic messages and report them in JSON or human-readable format.

If the parser fails to produce an AST (which can happen with mixed-language files), the scanner falls back to a lightweight regex pattern to at least flag potential issues, but it emits a warning that the AST mode is unavailable.

Example rule: detect unsafe eval()

Here is a simplified version of one of the first rules I wrote:

class EvalRule:
    def check(self, node):
        if
python
        if node.type != 'call':
            return None
        if node.function_name != 'eval':
            return None
        if self._is_in_safe_context(node):
            return None
        return Diagnostic(
            severity='error',
            message='Unsafe eval() detected',
            line=node.line,
            column=node.column,
            suggested_fix='Avoid eval() or sanitize the input thoroughly.'
        )

The _is_in_safe_context method is a quick heuristic: it checks whether the argument is a hardcoded string literal or a constant expression. Anything else gets flagged. This is intentionally conservative — I'd rather have a false positive than miss a real vulnerability.

What the scanner found in AI-generated code

I ran the scanner on a corpus of about 200 code snippets generated by several popular AI assistants — some from OpenAI's models, some from Anthropic's, and a few from open-source models. The snippets were responses to prompts like