Crawl4AI - Agno

Crawl4aiTools enable an Agent to perform web crawling and scraping tasks using the Crawl4ai library.

Prerequisites

The following example requires the crawl4ai library.

pip install -U crawl4ai

Example

The following agent will scrape the content from the https://github.com/agno-agi/agno webpage:

cookbook/tools/crawl4ai_tools.py

from agno.agent import Agent
from agno.tools.crawl4ai import Crawl4aiTools

agent = Agent(tools=[Crawl4aiTools(max_length=None)], show_tool_calls=True)
agent.print_response("Tell me about https://github.com/agno-agi/agno.")

Toolkit Params

Parameter	Type	Default	Description
`max_length`	`int`	`1000`	Specifies the maximum length of the text from the webpage to be returned.

Toolkit Functions

Function	Description
`crawl`	Crawls one or more URLs using crawl4ai’s WebCrawler. Parameters include url for a single URL or list of URLs to crawl, and an optional search_query to filter content using the BM25 algorithm.

Developer Resources

View Tools
View Cookbook