Internet Search and Parsing JS Sites: web_search for Data, browser_scrape for SPA

## web_search and browser_scrape: How AI Extracts Data in the SPA Era The modern internet is not just HTML pages. Most websites today are built on JavaScript (SPA — Single Page Applications), where content loads dynamically, and standard HTTP request parsing is useless. For AI agents collecting data, this is a real challenge. But there is a solution: two tools — **web_search** for finding information through search engines and **browser_scrape** for extracting content from JS sites via headless Chromium on the server. Let's break down how they work and when to use which. ### What is web_search and why is it needed? **web_search** is an AI tool that automates internet searches. Instead of manually Googling and analyzing results, the neural network itself forms queries, follows links, and collects relevant data. This is ideal for: - Gathering news on a topic - Monitoring product prices - Analyzing reviews and ratings - Finding contact information Example: An AI agent searches for the latest articles on machine learning. It sends a query "ML 2026 trends," gets 10 links, parses them, and returns a brief summary. This is faster than manually flipping through pages. | Tool | Data Type | Example Use Case | |------------|------------|----------------------| | web_search | Text search results | News gathering, monitoring | | browser_scrape | Dynamic SPA content | Parsing React/Vue sites | ### The SPA Problem: Why Standard Parsing Doesn't Work SPA sites (e.g., Trello, Airbnb, many modern landing pages) load content via JavaScript after the initial HTML load. If you try to parse such a site with a simple HTTP request, you'll get an empty shell. You need a full browser engine that executes JS code. This is where **browser_scrape** comes into play. ### How browser_scrape Works with headless Chromium **browser_scrape** uses headless mode of Chromium — a browser without a graphical interface that runs on the server. It: 1. Loads the entire page (including JS) 2. Waits for dynamic content to load 3. Extracts needed elements (text, images, data) Example: You want to parse a list of courses on an SPA platform. headless Chromium opens the page, executes scripts, and after 2-3 seconds you get the full HTML. This is indispensable for **parsing** modern web applications. ### Practical Tips for AI Search - **Choose the right tool**: For static sites (blogs, Wikipedia), web_search is sufficient. For SPA (online stores, SaaS), use browser_scrape. - **Set delays**: The headless browser can be blocked for fast requests. Add pauses of 1-2 seconds. - **Use proxies**: To avoid blocks during mass data collection. - **Check robots.txt**: Respect the site's policy. ### Conclusion **web_search** and **browser_scrape** are two powerful tools for AI agents working with internet data. The first is for quick searches through search engines, the second for deep parsing of JS sites. Together, they cover 90% of information gathering tasks. If you want to learn how to use them in practice, check out the free courses on ASI Biont — there are modules on AI agents and web scraping. Start now, without restrictions or payment!

Internet Search and Parsing JS Sites: web_search for Data, browser_scrape for SPA

Comments

Recent articles

Vibe Coding for Entrepreneurs: Without Programmers in 60 Minutes

AI Agent as an Employee: Setting Up Autopilot for Your Business in 2026

Vibe Coding: How AI Helps Create Code Without Programming Skills on the ASI Biont Course

System Design: How AI and the ASI Biont Course Are Changing the Learning Approach

Excel and Google Sheets: A Course with AI on ASI Biont — How to Automate Work in 2026

AI Agent in Telegram: How @asibiont_bot Automates Replies to Subscribers

Course 'Copywriting and Content': Learning with AI on ASI Biont — How to Write Texts That Sell

SEO and Website Promotion: How AI Is Changing Learning in 2026

Course "Chinese from Scratch": Learning with AI on ASI Biont — From Characters to HSK