> ## Documentation Index
> Fetch the complete documentation index at: https://docs-v1.agno.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Crawl4AI

**Crawl4aiTools** enable an Agent to perform web crawling and scraping tasks using the Crawl4ai library.

## Prerequisites

The following example requires the `crawl4ai` library.

```shell theme={null}
pip install -U crawl4ai
```

## Example

The following agent will scrape the content from the [https://github.com/agno-agi/agno](https://github.com/agno-agi/agno) webpage:

```python cookbook/tools/crawl4ai_tools.py theme={null}
from agno.agent import Agent
from agno.tools.crawl4ai import Crawl4aiTools

agent = Agent(tools=[Crawl4aiTools(max_length=None)], show_tool_calls=True)
agent.print_response("Tell me about https://github.com/agno-agi/agno.")
```

## Toolkit Params

| Parameter    | Type  | Default | Description                                                               |
| ------------ | ----- | ------- | ------------------------------------------------------------------------- |
| `max_length` | `int` | `1000`  | Specifies the maximum length of the text from the webpage to be returned. |

## Toolkit Functions

| Function | Description                                                                                                                                                                                      |
| -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `crawl`  | Crawls one or more URLs using crawl4ai's WebCrawler. Parameters include url for a single URL or list of URLs to crawl, and an optional search\_query to filter content using the BM25 algorithm. |

## Developer Resources

* View [Tools](https://github.com/agno-agi/agno/blob/main/libs/agno/agno/tools/crawl4ai.py)
* View [Cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/tools/crawl4ai_tools.py)