ROS / ROS2 Meets AI: How to Control Robots with Natural Language via ASI Biont

Introduction

Roboticists have long dreamed of a system where you can simply say "navigate to the kitchen table" or "pick up the red cube" and watch your ROS-based robot execute the task without writing a single line of C++ or Python. That dream is now a reality. ASI Biont, an AI agent designed for industrial and robotic automation, integrates directly with ROS (Robot Operating System) and ROS2 ecosystems, enabling operators to control navigation, manipulation, and data collection through plain-text commands.

In this article, we provide a practical, code-driven guide to connecting ROS/ROS2 robots to ASI Biont. You'll learn which connection methods the AI agent supports (WebSocket via rosbridge, SSH, MQTT, and HTTP), see real examples of autonomous navigation with RViz visualization, object grasping, and real-time telemetry, and understand why this integration slashes development time from weeks to minutes.

Why Connect ROS/ROS2 to an AI Agent?

ROS is the de facto standard for research and industrial robotics. It provides a modular architecture with topics (pub/sub), services (request/reply), and actions (goal-oriented tasks). However, building a natural language interface on top of ROS typically requires:
- A custom NLP pipeline to parse user commands.
- A state machine to map commands to ROS actions.
- A WebSocket or REST bridge to communicate with external systems.

ASI Biont eliminates all of this. The AI agent understands your robot's capabilities (provided via system prompts or YAML configuration), writes the integration code on the fly, and executes it in a secure sandbox. The result: you can command your robot in English, and the AI handles the ROS message serialization, service calls, and error handling.

Supported Connection Methods for ROS/ROS2

ASI Biont does not have a pre-built "ROS plugin". Instead, it connects to ROS/ROS2 systems through the same universal methods it uses for any device. The most relevant for ROS are:

Recommended approach: Use WebSocket via rosbridge_server for interactive control. The AI agent writes a Python script using aiohttp and websockets (both available in the ASI Biont sandbox) to connect to the rosbridge WebSocket endpoint, subscribe to topics (e.g., /camera/image_raw), publish commands (e.g., /cmd_vel), and call services (e.g., /gazebo/set_model_state).

Step-by-Step Integration: ROS2 + ASI Biont

Prerequisites

• A ROS2 (Humble or later) robot with rosbridge_server installed: sudo apt install ros-humble-rosbridge-server
• The robot's onboard computer (e.g., Raspberry Pi 5 or NVIDIA Jetson) must be reachable on the network.
• ASI Biont account (free tier works for testing).

1. User Describes the Robot in Chat

In the ASI Biont chat, the user provides:

"Connect to my ROS2 robot at 192.168.1.100. The rosbridge WebSocket is on port 9090. I want to:
- Subscribe to /scan (LaserScan) to get LiDAR data
- Publish to /cmd_vel (Twist) to move the robot
- Call /get_pose service to get current position
- When I say 'go forward 1 meter', make the robot drive forward 1 meter."

2. AI Writes the Integration Code

ASI Biont generates a Python script using aiohttp and websockets. The script runs inside the ASI Biont sandbox (execute_python) and connects to the rosbridge WebSocket. Here is a simplified version of the generated code:

import asyncio
import json
import websockets

ROSBRIDGE_URI = "ws://192.168.1.100:9090"

async def ros_call(ws, op, msg):
    """Send a rosbridge operation and wait for response if needed."""
    payload = {"op": op, **msg}
    await ws.send(json.dumps(payload))
    response = await ws.recv()
    return json.loads(response)

async def main():
    async with websockets.connect(ROSBRIDGE_URI) as ws:
        # Subscribe to /scan
        sub_msg = {"topic": "/scan", "type": "sensor_msgs/LaserScan"}
        await ros_call(ws, "subscribe", sub_msg)

        # Wait for one scan message
        scan_raw = await ws.recv()
        scan = json.loads(scan_raw)
        print(f"Received {len(scan['msg']['ranges'])} laser ranges")

        # Call /get_pose service (assuming a custom service)
        service_call = {
            "service": "/get_pose",
            "args": {},
            "type": "your_package/GetPose"
        }
        response = await ros_call(ws, "call_service", service_call)
        print(f"Robot pose: {response['values']['pose']}")

        # Publish a velocity command
        twist_msg = {
            "op": "publish",
            "topic": "/cmd_vel",
            "msg": {
                "linear": {"x": 0.5, "y": 0.0, "z": 0.0},
                "angular": {"x": 0.0, "y": 0.0, "z": 0.0}
            }
        }
        await ws.send(json.dumps(twist_msg))
        print("Published /cmd_vel: forward 0.5 m/s")

asyncio.run(main())

Note: The actual script generated by ASI Biont is more robust, with error handling, reconnection logic, and a command parser that maps natural language ("go forward 1 meter") to duration-based velocity commands.

3. User Executes the Integration

The AI agent runs the script in the sandbox. The output shows:
- "Received 360 laser ranges"
- "Robot pose: x=1.2, y=0.5, theta=0.0"
- "Published /cmd_vel: forward 0.5 m/s"

The robot starts moving. The user can now type:

"Turn left 90 degrees"

The AI generates a new command: publish a Twist with angular.z = 0.5 rad/s for 3.14 seconds (since 0.5 rad/s * 3.14 s ≈ π/2 rad).

4. Real-Time Telemetry with RViz

To visualize the robot's state, the user can ask:

"Show me the LiDAR scan in RViz"

ASI Biont cannot directly inject into RViz (which runs on the user's local machine), but it can:
- Publish a marker array topic that RViz subscribes to.
- Or instruct the user to run a provided RViz config that connects to the rosbridge topics.

The AI generates a launch file snippet:

<launch>
  <node name="rviz" pkg="rviz2" exec="rviz2" args="-d $(find your_package)/config/ai_control.rviz"/>
  <node name="rosbridge" pkg="rosbridge_server" exec="rosbridge_websocket"/>
</launch>

Real-World Use Cases

Case 1: Autonomous Navigation with Rviz

A TurtleBot3 robot equipped with LiDAR and a camera. The user commands:

"Navigate to coordinates (2.5, 1.0) while avoiding obstacles. Show me the costmap in RViz."

ASI Biont:
1. Calls /move_base_simple/goal (action) with the target pose.
2. Subscribes to /move_base/global_costmap and /move_base/local_costmap.
3. Publishes a visualization_msgs/MarkerArray to indicate the goal on RViz.
4. Continuously monitors /odom and /amcl_pose to report progress.

Case 2: Object Grasping with a Robotic Arm

A UR5e arm controlled via ROS2 MoveIt2. The user says:

"Pick up the green cylinder from the conveyor belt and place it in bin B."

ASI Biont:
1. Subscribes to /camera/depth_registered/points (PointCloud2).
2. Runs a segmentation model (via transformers or onnxruntime in the sandbox) to locate the green cylinder.
3. Computes the grasp pose using inverse kinematics (via /compute_ik service).
4. Executes the trajectory via /follow_joint_trajectory action.

Case 3: Telemetry Alerting

A fleet of ROS2 robots in a warehouse. The user says:

"Monitor battery levels on all robots. If any robot's battery drops below 20%, send me a Telegram message and command the robot to return to the charging station."

ASI Biont:
1. Subscribes to /battery_state on each robot (using separate rosbridge connections).
2. Stores the threshold and checks every 10 seconds.
3. When triggered, publishes to /cmd_vel to navigate the robot to a pre-defined dock pose, then uses the Twilio API (available in sandbox) to send an SMS.

Why ASI Biont Beats Custom Development

Limitations and Considerations

1. Latency: The sandbox runs in the cloud (on Railway). For real-time control loops (e.g., balancing robot), you need a local deployment. ASI Biont supports Hardware Bridge (bridge.py) which runs on your local PC and can communicate with the sandbox via HTTP long polling, but the round-trip is ~100-500 ms. For high-frequency control (100 Hz+), consider running the AI-generated code directly on the robot's onboard computer via SSH.

2. Security: The rosbridge WebSocket should be protected with a firewall or authentication token. In production, use rosbridge_server with SSL and a custom authentication plugin.

3. Sandbox Timeout: execute_python scripts have a 30-second timeout. For long-running tasks (e.g., monitoring battery for hours), use the industrial_command tool with a persistent connection (e.g., MQTT subscription) rather than a one-shot script.

How to Get Started

1. Install rosbridge_server on your robot: sudo apt install ros-<distro>-rosbridge-server
2. Launch the bridge: ros2 launch rosbridge_server rosbridge_websocket_launch.xml
3. Open ASI Biont at asibiont.com.
4. In the chat, describe your robot:

   "Connect to ROS2 at IP 192.168.0.42, port 9090. I have topics /scan, /cmd_vel, /odom. I want to control the robot with natural language."

5. The AI writes and executes the code. You can immediately send commands like "turn 45 degrees" or "show me the LiDAR data".

Conclusion

Integrating ROS/ROS2 with ASI Biont transforms the way roboticists interact with their machines. Instead of writing boilerplate bridge code, you simply describe your robot in plain English, and the AI agent handles all the protocol details—whether it's WebSocket via rosbridge, SSH for direct node execution, or MQTT for lightweight telemetry. The result is a natural language interface to your robot that works out of the box, with no custom NLP or state machine development.

The three use cases—autonomous navigation with RViz, object grasping with MoveIt2, and fleet telemetry alerting—demonstrate the breadth of what's possible. Whether you're a researcher prototyping a new algorithm or a DevOps engineer managing a fleet of warehouse robots, ASI Biont eliminates the integration bottleneck and lets you focus on what matters: making your robot smarter.

Try it yourself today. Go to asibiont.com, create an account, and tell the AI agent to connect to your ROS robot. You'll be controlling your bot with natural language in under 5 minutes.