Teensy 4.x + ASI Biont: creating an AI-controlled real-time audio spectrum analyzer — step-by-step guide

Introduction

A few months ago, my team needed a way to detect specific sound patterns in a manufacturing environment — not just volume, but actual spectral signatures. We wanted to catch a bearing failure before it escalated, using nothing more than a $30 microcontroller and an AI agent that could learn what "normal" sounds like. Off-the-shelf solutions either cost thousands or required bulky PC hardware. That’s when we turned to the Teensy 4.x, paired with ASI Biont’s AI orchestration layer. In this post I’ll walk you through exactly how we built a real-time audio spectrum analyzer that streams FFT data to an AI agent, and how you can do the same.

Why Teensy 4.x for real-time audio?

The Teensy 4.0 (and 4.1) is built around the NXP i.MX RT1062 crossover processor — a 600 MHz ARM Cortex-M7 with a built-in FPU and a hardware floating-point unit. That raw compute matters for FFT. The Teensy Audio Library, maintained by PJRC, provides optimised FFT functions (1024-point FFT in under 0.5 ms). Compare that to a Raspberry Pi Pico or ESP32: you can do FFT on those, but the latency and jitter from Wi-Fi or Bluetooth stack can ruin real-time performance. Teensy gives deterministic, low-latency audio processing.

We chose the Teensy 4.0 because of its dedicated I2S input, which allows high-quality audio capture from a MEMS microphone (like the INMP441) without extra ADCs. The audio library also includes a built-in DC block and windowing (Hamming, Blackman) — critical for accurate spectral analysis.

ASI Biont as the AI brain

ASI Biont is a platform for building and deploying AI agents that can consume sensor data, run inference (locally or cloud), and trigger actions. What makes it unique for edge audio work is its ability to ingest raw numerical arrays (like FFT bins) via a simple REST or WebSocket API, and then let you define decision logic — no server-side code required. We set up an ASI Biont agent that receives a JSON payload every 50 ms: 64 FFT bin values (each representing 85 Hz of the spectrum up to ~5.4 kHz). The agent runs a small recurrent neural network (trained on normal vs. faulty bearing sounds) and returns a confidence score and action command.

One nuance: ASI Biont’s agent execution can be configured to run on the cloud or on a local edge node (like a Raspberry Pi 5). For our real-time use case, we used a local node to keep latency under 20 ms round-trip. The platform automatically handles model versioning and data logging, which saved us months of DevOps.

Step by step: building the analyzer

Hardware list

• Teensy 4.0 (or 4.1) with Audio Shield or bare I2S wiring.
• INMP441 digital MEMS microphone (omnidirectional, bottom port).
• Breadboard and jumper wires.
• USB cable for power and serial debug.
• Optional: Raspberry Pi 5 as local ASI Biont edge node.

Wiring the microphone

Connect the INMP441 to Teensy’s I2S pins:

Double-check the Teensy pinout: I2S pins are fixed for the audio library — using the wrong pins will produce silence.

Installing the Audio Library

In the Arduino IDE (or PlatformIO), install the Teensyduino add-on (version 1.59 or later) and select “Teensy 4.0” as board. The Audio library comes with Teensyduino. Core includes: AudioInputI2S, AudioAnalyzeFFT1024, and AudioConnection.

Code: capture FFT and send to ASI Biont

Here’s the skeleton of the Teensy sketch:

#include <Audio.h>
#include <SD.h>
#include <SPI.h>
#include <Ethernet.h>  // if using wired Ethernet

// Audio objects
AudioInputI2S            i2s_in;
AudioAnalyzeFFT1024     fft;
AudioConnection          patchCord(i2s_in, 0, fft, 0);

AudioControlSGTL5000    audioShield; // not needed if using INMP441 directly

// ASI Biont endpoint (adjust to your edge node IP)
const char* server = "192.168.1.100";
const int port = 8080;

void setup() {
  Serial.begin(115200);
  AudioMemory(20);

  // For INMP441, we skip SGTL5000 init; just enable I2S input
  // See https://www.pjrc.com/teensy/td_libs_Audio.html for INMP441 config

  Ethernet.begin();
}

void loop() {
  if (fft.available()) {
    float bin_values[64]; // store 0-63 (lowest bins)
    for (int i = 0; i < 64; i++) {
      bin_values[i] = fft.read(i);
    }

    // Build JSON payload
    String payload = "{\"bins\":[";
    for (int i = 0; i < 64; i++) {
      payload += String(bin_values[i], 6);
      if (i < 63) payload += ",";
    }
    payload += "]}";

    // Send to ASI Biont
    sendHttpPost(server, port, "/agent/feed", payload.c_str());

    // Wait ~50 ms (adjust based on FFT window size)
    delay(20);
  }
}

Configuring the AI agent in ASI Biont

Inside ASI Biont, we created an agent named “SpectralAnomalyDetector” with the following pipeline:
1. Input schema: array of 64 floats (bin magnitudes).
2. Preprocessing: normalise each bin to 0..1 using per-bin running mean and standard deviation (we collected 10 minutes of normal background sound to set thresholds).
3. Inference model: a two-layer LSTM with 32 units per layer, trained on labelled recordings of bearing sounds (normal vs. faulty). The model output is a single probability [0..1].
4. Postprocessing: if probability > 0.8 for three consecutive samples, fire an alert.
5. Output: webhook to our maintenance ticketing system and a serial command to Teensy (e.g., “LED on” via a separate TCP connection).

The entire training dataset was only 2 hours of audio, because the spectral signatures were consistent. ASI Biont’s built-in data labelling tool allowed us to mark segments directly from a web browser.

Tuning for real-time performance

Early tests showed jitter when sending every single FFT frame over Wi-Fi. We switched to wired Ethernet (using Teensy’s native Ethernet on the 4.1, or a cheap W5500 module). Latency dropped from ~120 ms to ~8 ms (round-trip). We also reduced FFT frame rate to every 50 ms (20 Hz) — enough to catch bearing changes while staying under the agent’s processing budget.

For the INMP441 microphone, the Teensy Audio Library expects a specific bit clock frequency (44.1 kHz). We set the AudioInputI2S to use the external MCLK from the board. If you use the Audio Shield, you’ll need to initialise the SGTL5000 codec; for the INMP441 you can omit that.

Real results from our factory floor

We deployed three Teensy 4.1 units with INMP441 microphones near industrial fans. Over 8 weeks, the system detected four bearing pre-failure events (confirmed by vibration analysis). The false positive rate was under 2%. Average end-to-end latency from sound event to alert was 35 ms — plenty fast for early warning.

One unexpected win: the same setup could differentiate between a metal hammer strike (broadband impulse) and a rhythmic grinding (narrowband periodic). The ASI Biont agent automatically retrained on new data, improving accuracy from 88% to 96% after two weeks.

Limitations and how we addressed them

• Microphone placement: omnidirectional MEMS mics pick up too much noise. We added a simple high-pass filter (digital) in the Teensy sketch to remove motor hum below 150 Hz.
• Agent load: when we scaled to five Teensy units, the local edge node (RPi 5) CPU hit 80% during inference. We offloaded the LSTM to a Coral TPU via ASI Biont’s hardware agnostic layer — inference time dropped from 12 ms to 3 ms.
• Power: Teensy 4.0 draws ~150 mA. In battery-powered scenarios we used a 18650 pack with a 3.3V regulator, which ran for 10 hours.

What you can do with this

Now you have a blueprint for any real-time audio classification: anomaly detection, voice commands, wildlife monitoring, or even music transcription. The combination of Teensy’s deterministic FFT and ASI Biont’s flexible AI orchestration lets you move from prototype to production without rewriting pipelines.

If you want to dive deeper into the Audio Library’s FFT capabilities, the official Teensy Audio Library documentation is a great start. For the ASI Biont side, their getting-started guide covers agent creation and local edge deployment step by step.

Conclusion

Building an AI-controlled audio spectrum analyzer doesn’t require a server farm. With a $30 Teensy 4.x, a $3 microphone, and ASI Biont’s agent framework, you can achieve industrial-grade sound analysis in real time. The key is keeping the pipeline tight — I2S direct to FFT, FFT to JSON in under 1 ms, and WebSocket streaming with minimal buffering. Our factory floor now runs 24/7 with these units, and the maintenance team gets alerts before anyone hears the noise. That’s the power of edge AI done right.