Skip to main content

​
Prerequisites

The following examples require the scrapegraph-py library. 
pip install -U scrapegraph-py
Optionally, if your ScrapeGraph configuration or specific models require an API key, set the SGAI_API_KEY environment variable: 
export SGAI_API_KEY="YOUR_SGAI_API_KEY"

​
Example

The following agent uses ScrapeGraphTools to extract specific information from a webpage using the smartscraper functionality. 
from agno.agent import Agent
from agno.tools.scrapegraph import ScrapeGraphTools

agent = Agent(
    tools=[ScrapeGraphTools(smartscraper=True)],
    show_tool_calls=True,
)

agent.print_response("""
    "Use smartscraper to extract the following from https://www.wired.com/category/science/:
- News articles
- Headlines
- Images
- Links
- Author
""",
)
View the Startup Analyst example & Agentic Deep Researcher Workflow

​
Toolkit Functions

FunctionDescription
smartscraperExtracts structured data from a webpage using an LLM and a natural language prompt. Returns a JSON string of the result or an error.
markdownifyConverts a webpage to markdown format. Returns the markdown string or an error.

​
Toolkit Parameters

These parameters are passed to the ScrapeGraphTools constructor.
ParameterTypeDefaultDescription
api_keyOptional[str]NoneAPI key for ScrapeGraph services. If not provided, it defaults to the SGAI_API_KEY environment variable.
smartscraperboolTrueEnable the smartscraper tool.
markdownifyboolFalseEnable the markdownify tool.

​
Developer Resources