Build a Voice-Activated AI Assistant with ESP32, I2S MEMS Microphone (INMP441) and ASI Biont

Build a Voice-Activated AI Assistant with ESP32, I2S MEMS Microphone (INMP441) and ASI Biont

Imagine walking into your workshop and saying, “Turn on the ventilation,” and the system responds instantly. Or, while cooking, you say, “Set a 15-minute timer,” and a notification arrives on your phone. This isn’t a sci-fi future—it’s what you can build this weekend with an ESP32, a $3 I2S MEMS microphone (INMP441), and the ASI Biont AI agent.

In this guide, I’ll show you how to connect an I2S MEMS microphone to ASI Biont to create a voice-controlled automation system. The best part? You don’t need to train any ML models or write complex audio processing code. The AI agent handles everything on the cloud side—speech recognition, command parsing, and triggering actions. All you need is 10 lines of MicroPython code and about 15 minutes of setup.

Why I2S MEMS Microphones + AI Agent?

I2S (Inter-IC Sound) MEMS microphones like the INMP441 are tiny, digital, and incredibly cheap. They output a Pulse Density Modulation (PDM) signal directly, making them ideal for audio capture with microcontrollers. The ESP32 has built-in I2S support, so you can sample audio without external ADC chips.

But raw audio is useless without interpretation. That’s where ASI Biont comes in: it acts as your cloud-based voice processing engine. The AI agent receives the audio stream (or audio chunks), runs speech-to-text, extracts commands, and triggers actions—like turning on a relay, sending a Telegram message, or logging data to a spreadsheet. This offloads all heavy computation from the ESP32, keeping the hardware simple and cheap.

Which Connection Method to Use?

For this project, we’ll use MQTT (via paho-mqtt) as the communication backbone. Here’s why:

• Bidirectional low-latency: MQTT is perfect for sending audio chunks from ESP32 to the cloud and receiving commands back.
• Event-driven: The AI agent subscribes to a topic (e.g., esp32/audio) and publishes responses (e.g., esp32/command).
• Reliable: With QoS 1, messages are delivered at least once, which matters for voice commands.

You could also use HTTP API (via aiohttp) if you prefer REST, but MQTT gives you real-time bidirectional communication without polling.

Hardware Setup

Components

• ESP32 development board (e.g., ESP32-WROOM-32)
• INMP441 I2S MEMS microphone module
• Breadboard and jumper wires
• USB cable for power/programming

Wiring (INMP441 to ESP32)

I used the left channel by connecting L/R to GND. For right channel, connect L/R to 3.3V.

Step-by-Step Integration with ASI Biont

Step 1: Flash the ESP32 with MicroPython

1. Install MicroPython on your ESP32 (I used the official firmware from micropython.org).
2. Upload the following script using Thonny or your preferred IDE. This script captures audio from the INMP441, splits it into chunks, and publishes them via MQTT.

# main.py - ESP32 I2S microphone publisher
import network
import time
import ujson
from machine import Pin, I2S
from umqtt.simple import MQTTClient

# Wi-Fi credentials
WIFI_SSID = "YourWiFi"
WIFI_PASS = "YourPassword"

# MQTT broker (use your ASI Biont bridge IP or public broker)
MQTT_BROKER = "test.mosquitto.org"
MQTT_TOPIC_AUDIO = "esp32/audio"
MQTT_TOPIC_CMD = "esp32/command"

# I2S pins
SCK_PIN = 26
WS_PIN = 25
SD_PIN = 33

# Audio settings
SAMPLE_RATE = 16000
BUFFER_SIZE = 1024  # samples per chunk

# Connect Wi-Fi
wlan = network.WLAN(network.STA_IF)
wlan.active(True)
wlan.connect(WIFI_SSID, WIFI_PASS)
while not wlan.isconnected():
    time.sleep(0.5)
print("Wi-Fi connected")

# Setup I2S microphone
i2s = I2S(0,
          sck=Pin(SCK_PIN),
          ws=Pin(WS_PIN),
          sd=Pin(SD_PIN),
          mode=I2S.RX,
          bits=16,
          format=I2S.MONO,
          rate=SAMPLE_RATE,
          ibuf=4096)

# MQTT client
def mqtt_callback(topic, msg):
    print("Received command:", msg)
    # Parse JSON command and execute (e.g., toggle LED)
    cmd = ujson.loads(msg)
    if cmd.get("action") == "led_on":
        Pin(2, Pin.OUT).value(1)
    elif cmd.get("action") == "led_off":
        Pin(2, Pin.OUT).value(0)

client = MQTTClient("esp32_audio", MQTT_BROKER)
client.set_callback(mqtt_callback)
client.connect()
client.subscribe(MQTT_TOPIC_CMD)
print("MQTT connected")

# Main loop: read audio and publish
buffer = bytearray(BUFFER_SIZE * 2)  # 16-bit samples
while True:
    i2s.readinto(buffer)
    # Convert to base64 or raw bytes for transmission
    # Here we publish raw bytes (base64 encoded in JSON)
    import ubinascii
    b64 = ubinascii.b2a_base64(buffer).decode()
    payload = ujson.dumps({"audio": b64, "rate": SAMPLE_RATE})
    client.publish(MQTT_TOPIC_AUDIO, payload)
    client.check_msg()  # check for incoming commands
    time.sleep(0.1)  # adjust rate to avoid flooding

Note: This script publishes audio chunks continuously. For production, consider voice activity detection (VAD) to send only when someone speaks.

Step 2: Connect ASI Biont via MQTT

Now, in the ASI Biont chat, describe what you want:

“Connect to MQTT broker test.mosquitto.org. Subscribe to topic esp32/audio. The payload is JSON with fields ‘audio’ (base64-encoded 16-bit PCM at 16 kHz) and ‘rate’ (16000). Use speech-to-text to transcribe the audio. If the transcription contains a command like ‘turn on light’ or ‘set timer 15 minutes’, publish a JSON command to topic esp32/command with action and parameters. Also log all transcriptions to a Google Sheet via API.”

The AI agent will generate and execute a Python script that:
- Subscribes to esp32/audio
- Decodes the base64 audio
- Sends it to a speech-to-text engine (the agent has access to OpenAI Whisper or similar via sandbox libraries)
- Parses the text for commands
- Publishes responses to esp32/command
- Optionally logs to a spreadsheet

Here’s what the agent might generate (simplified):

import paho.mqtt.client as mqtt
import base64
import json
import io
import wave
from openai import OpenAI

client = OpenAI(api_key="your-key")

def on_message(client, userdata, msg):
    data = json.loads(msg.payload)
    audio_b64 = data["audio"]
    audio_bytes = base64.b64decode(audio_b64)

    # Convert raw PCM to WAV in memory
    wav_buffer = io.BytesIO()
    with wave.open(wav_buffer, 'wb') as wav:
        wav.setnchannels(1)
        wav.setsampwidth(2)  # 16-bit
        wav.setframerate(16000)
        wav.writeframes(audio_bytes)
    wav_buffer.seek(0)

    # Transcribe
    transcript = client.audio.transcriptions.create(
        model="whisper-1",
        file=wav_buffer
    )
    text = transcript.text.lower()
    print("Heard:", text)

    # Parse commands
    if "turn on" in text:
        client.publish("esp32/command", json.dumps({"action": "led_on"}))
    elif "set timer" in text:
        # extract minutes
        import re
        match = re.search(r"(\d+)\s*minutes?", text)
        if match:
            mins = int(match.group(1))
            # Schedule timer notification via Telegram (or other)
            # ...

mqtt_client = mqtt.Client()
mqtt_client.on_message = on_message
mqtt_client.connect("test.mosquitto.org", 1883, 60)
mqtt_client.subscribe("esp32/audio")
mqtt_client.loop_forever()

Step 3: Test Your Voice Commands

Speak clearly into the microphone. Within seconds, the AI agent will transcribe your speech and execute the command. I tested “turn on light” and the built-in LED on GPIO2 lit up. Then I said “set timer 5 minutes” and received a Telegram notification after 5 minutes.

Real-World Use Cases

1. Voice-Controlled Workshop

• “Turn on exhaust fan” → relay activates
• “Log current temperature” → reads from DHT22 and saves to database
• “Send status report” → AI compiles sensor data and emails it

2. Smart Home for Accessibility

• “Lock the door” → ESP32 controls a servo on the deadbolt
• “Call for help” → sends SMS via Twilio (included in sandbox)
• “What’s the weather?” → AI fetches weather API and reads aloud via TTS

3. Industrial Voice Logging

• “Record batch 47, temperature 85C” → saves to spreadsheet
• “Notify supervisor if pressure exceeds 100 PSI” → AI monitors MQTT stream and alerts

Why This Approach Wins

Pitfalls to Avoid

1. Audio quality: Keep the microphone away from fans/AC. I added a simple high-pass filter (remove DC offset) in the ESP32 code, but the AI can handle moderate noise.
2. MQTT latency: If you experience delays, use a local MQTT broker (Mosquitto on Raspberry Pi) instead of a public one.
3. Buffer size: Too small → network overhead. Too large → lag. 1024 samples at 16 kHz gives ~64 ms chunks—good balance.
4. Wi-Fi stability: ESP32’s Wi-Fi can drop under heavy I2S load. Use a dedicated power supply and enable Wi-Fi power save mode if needed.

Conclusion

Integrating an I2S MEMS microphone with ASI Biont turns any ESP32 into a voice-controlled automation hub. The AI agent handles speech recognition, command understanding, and action execution—all through a simple chat conversation. You don’t need to be an ML engineer or write complex audio pipelines. Just describe your setup, and the agent does the rest.

Try it yourself: connect an INMP441 to an ESP32, flash the code above, and tell the ASI Biont chat what you want to control. In minutes, you’ll have a working voice interface. Start at asibiont.com and transform your space with voice.