Website Tools - Web Scraping and Content Analysis - Agno

This example demonstrates how to use WebsiteTools for web scraping and analysis. Shows enable_ flag patterns for selective function access. WebsiteTools is a small tool (<6 functions) so it uses enable_ flags.

"""
Website Tools - Web Scraping and Content Analysis

This example demonstrates how to use WebsiteTools for web scraping and analysis.
Shows enable_ flag patterns for selective function access.
WebsiteTools is a small tool (&lt;6 functions) so it uses enable_ flags.
"""

from agno.agent import Agent
from agno.tools.website import WebsiteTools

# ---------------------------------------------------------------------------
# Create Agent
# ---------------------------------------------------------------------------


agent = Agent(
    tools=[WebsiteTools()],  # All functions enabled by default
    description="You are a comprehensive web scraping specialist with all website analysis capabilities.",
    instructions=[
        "Help users scrape and analyze website content",
        "Provide detailed summaries and insights from web pages",
        "Handle various website formats and structures",
        "Ensure respectful scraping practices",
    ],
    markdown=True,
)

# Example usage

# ---------------------------------------------------------------------------
# Run Agent
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    print("=== Basic Web Content Search Example ===")
    agent.print_response(
        "Search web page: 'https://docs.agno.com/introduction' and summarize the key concepts",
        markdown=True,
    )

Run the Example

# Clone and setup repo
git clone https://github.com/agno-agi/agno.git
cd agno/cookbook/91_tools

# Create and activate virtual environment
./scripts/demo_setup.sh
source .venvs/demo/bin/activate

python website_tools.py

⌘I