From Edge to Agent: Connecting Raspberry Pi + TensorFlow Lite / ONNX Runtime with ASI Biont

From Edge to Agent: Connecting Raspberry Pi + TensorFlow Lite / ONNX Runtime with ASI Biont

The Raspberry Pi has become the default prototyping platform for edge machine learning. With TensorFlow Lite or ONNX Runtime, you can run object detection, pose estimation, or anomaly detection directly on the device. But there is a catch: after the inference finishes, the result is often just a print statement or a log file. The Pi doesn't know what to do with it, and integrating it with other systems takes time.

This guide shows how to take that final step with the ASI Biont AI agent. Instead of writing a custom Flask service, a database bridge, and a notification script, you simply describe your Raspberry Pi in a chat. ASI Biont writes the integration code, connects to your device via SSH or MQTT, and automates the response. Everything happens through the chat dialog: no dashboards, no "Add Device" buttons, no manual configuration files.

By the end of this article, you will have a clear picture of how on-device ML and a conversational AI agent combine to build powerful, responsive automation in minutes.

The device: Raspberry Pi as a low-cost inference engine

The Raspberry Pi 4 and Pi 5 are the go-to single-board computers for edge AI. They run the official tflite-runtime wheel from TensorFlow's documentation, and ONNX Runtime provides prebuilt ARM64 wheels in its official release pipeline. You can install either with a simple pip command. With a Pi 5, even the COCO-trained EfficientDet-Lite models run at several frames per second — sufficient for threshold-based monitoring. For heavier models, you can add a Coral USB accelerator or an Hailo chip, but the integration pattern stays the same.

Here is a quick reference for what you need to get started:

Why connect it to an AI agent?

Because detection without action is just a measurement. A camera sees a person — what happens next? You might want a notification, a door unlock, a light switch, or a recording in the cloud. If you hard-code these rules, every change requires a new release. An AI agent, on the other hand, can decide the action based on the detection and even ask for clarification. ASI Biont brings that reasoning layer to your edge device without being embedded on the Pi itself — it runs wherever your other automation tools run.

How ASI Biont connects to the Pi

ASI Biont supports a broad list of industrial and IT protocols: MQTT, Modbus/TCP, OPC-UA, SSH, HTTP API/WebSocket, BACnet, CAN bus, gRPC, CoAP, and COM ports via a Hardware Bridge. For a Raspberry Pi, the most practical are SSH and MQTT.

On-demand inference via SSH

Suppose your Pi is on 192.168.1.50, user pi, with a TFLite model model.tflite and a camera. In the ASI Biont chat you write:

Connect to my Pi at 192.168.1.50 via SSH. Run /home/pi/detect.py and tell me if there is a person in the latest camera frame.

Here is what ASI Biont does behind the scenes, in Python code:

import paramiko, json

client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect("192.168.1.50", username="pi", password="mypass")

stdin, stdout, stderr = client.exec_command("python3 /home/pi/detect.py")
result = json.loads(stdout.read().decode())
if result["person_detected"]:
    answer = "Yes, a person is present."
else:
    answer = "No person detected."

client.close()

The script on the Pi, detect.py, uses the TFLite runtime. A minimal but realistic version:

import tflite_runtime.interpreter as tflite
import json, cv2

interpreter = tflite.Interpreter(model_path="model.tflite")
interpreter.allocate_tensors()

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

frame = cv2.imread("latest.jpg")
resized = cv2.resize(frame, (input_details[0]["shape"][2], input_details[0]["shape"][1]))
input_data = resized.reshape(input_details[0]["shape"]).astype("float32") / 255.0

interpreter.set_tensor(input_details[0]["index"], input_data)
interpreter.invoke()

boxes, scores, classes, count = interpreter.get_tensor(output_details[0]["index"])[:4]
person_detected = any(s > 0.5 and label == 1 for s, label in zip(scores, classes))
print(json.dumps({"person_detected": bool(person_detected)}))

The integration is done in a few sentences. The AI agent chooses the library, handles errors, and returns a natural-language answer. In a conversation, you can now ask questions like "How confident was the model?" or "Run it every hour and give me a summary."

Continuous streaming with MQTT

For a security system, you want the Pi to detect events independently and send them to the agent continuously. The Pi runs its own process that publishes to an MQTT broker. The Pi code might look like this:

import paho.mqtt.client as mqtt
import json, time

client = mqtt.Client()
client.connect("192.168.1.10", 1883)  # broker on your LAN or cloud

while True:
    person_detected = run_inference_on_latest_frame()  # your TFLite/ONNX code
    payload = json.dumps({"ts": time.time(), "person": person_detected})
    client.publish("home/pi/detection", payload)
    time.sleep(1)

Note: this loop runs on the Pi, not inside ASI Biont's execute_python sandbox, which has a 30-second timeout. The Pi process is a normal systemd service. You can ask ASI Biont to generate the service file too.

In the chat, you tell the agent: "Subscribe to topic home/pi/detection on the broker at 192.168.1.10. Whenever a person is detected, send me a Telegram message with the payload." ASI Biont subscribes to the topic and processes each message. It is that simple.

ONNX Runtime inference on the Pi

TensorFlow Lite is not the only option. ONNX Runtime lets you run models trained in PyTorch, scikit-learn, or Keras. On the Pi, the code is very similar:

import onnxruntime as ort
import numpy as np, json

session = ort.InferenceSession("model.onnx")
input_name = session.get_inputs()[0].name

# preprocess your input as a numpy array
input_data = np.random.rand(1, 224, 224, 3).astype(np.float32)
output = session.run(None, {input_name: input_data})[0]

predicted_id = int(np.argmax(output))
print(json.dumps({"class_id": predicted_id, "confidence": float(output.max())}))

ASI Biont can execute this via SSH or receive the output over MQTT just as easily. The model format is irrelevant to the agent; what matters is the data arriving at the right topic or command output.

The universal execute_python connector

What if your Pi has a sensor that isn't supported by any built-in protocol? That's where execute_python comes in. You don't need a new module; the AI agent writes a Python script for your exact device. For example, a serial temperature sensor on /dev/ttyUSB0:

import serial, json

ser = serial.Serial('/dev/ttyUSB0', 115200, timeout=2)
line = ser.readline().decode().strip()
data = json.loads(line)
print(f"Temperature: {data['temperature']} C")

You describe this to the AI, and it generates, runs, and evaluates such a script in a sandbox. This is the "connect anything" escape hatch. Combined with the on-device model, it means you can build a custom edge computer and connect it to ASI Biont without waiting for platform features.

A typical chat session

The following table shows a realistic conversation between a user and ASI Biont while integrating the Pi:

Everything happens in natural language. No YAML configs, no webhooks setup, no deployment pipelines.

Real-world scenarios

Here are ten real-world scenarios where the pairing of a Raspberry Pi with TFLite/ONNX Runtime and ASI Biont changes the deployment effort.

1. Smart doorbell – A Pi at the front door detects a person, captures a photo, and sends it to the agent, which alerts the homeowner. The owner can then ask, "Was that the mailman?" and the agent re-runs classification for a package in the person's hand.
2. Industrial vibration anomaly detection – A Pi reads accelerometer data and runs an ONNX Runtime autoencoder. The agent monitors the anomaly score, and when it crosses the limit, writes an incident in the maintenance management system and stops the line via Modbus/TCP to a PLC.
3. Office occupancy and HVAC control – A Pi with a webcam counts people via TFLite. The agent calculates occupancy-based ventilation, communicates with the building's BACnet controller, and reduces airflow after everyone leaves.
4. Fall detection for elderly care – A Pi runs a pose-estimation model, detects falls, and sends the video fragment to the agent, which calls the caregiver's mobile phone via an HTTP API and sends the location.
5. Retail customer traffic – The Pi counts entries and exits; the agent cross-references with POS transactions to estimate conversion rate and sends shift-planning suggestions to the store manager.
6. Package theft detection – The Pi sees a package placed on the porch and then removed unexpectedly. The agent saves a video clip to an S3 bucket, sends a police notification, and arms an indoor camera.
7. Agricultural pest monitoring – A Pi on a sticky trap classifies pest species. The agent tracks the count over time and, when the threshold is reached, sends instructions to a drone service to release parasitic wasps.
8. Traffic light optimization – A Pi at an intersection counts vehicles; the agent adjusts a PLC's traffic light timings over Modbus/TCP, reducing congestion during rush hours.
9. Hospital PPE compliance – A Pi detects whether medical staff are wearing masks and caps. The agent logs violations in a compliance dashboard and sends reminders to the floor supervisor.
10. Smart parking – Several Pis monitor parking spaces. The agent receives availability updates over MQTT, updates a mobile app's map, and sends a message to the driver whose reservation expires.

Each of these follows the same pattern: the Pi runs a model, ASI Biont receives the result, and it reacts across the entire connected system. No custom integration platform is needed.

Security notes

When connecting via SSH, prefer key-based authentication over passwords. You can tell ASI Biont to use an SSH key by specifying the key's path in the chat. MQTT connections should always use TLS and, if possible, client certificates. The agent can generate a script with paho-mqtt that enables TLS and validates certificates. Since execute_python runs with a 30-second timeout, there's no risk of a runaway script; it is designed for finite operations.

Why this is better than the traditional path

A typical integration on a legacy platform involves writing a Python script with paho-mqtt, setting up a listener, configuring a dashboard, connecting a notification service, and then deploying everything. With ASI Biont, the conversation creates this stack automatically. The AI agent is fluent in the exact libraries you would use anyway — paramiko, paho-mqtt, pymodbus, aiohttp — so the results are production-quality code, but written in seconds instead of days.

The Raspberry Pi with TensorFlow Lite / ONNX Runtime and ASI Biont is a practical example of edge and agentic AI collaboration. The Pi offers low-latency, on-device inference; the agent offers the reasoning and orchestration. The connection between them is as simple as a sentence in the chat.

Try it on asibiont.com: deploy your Pi, open the chat, and ask the agent to connect to it. Your first edge-to-agent integration will be running before your coffee cools.