Documentation Index
Fetch the complete documentation index at: /llms.txt
Use this file to discover all available pages before exploring further.
You are viewing v1 docs. For the latest documentation, visit docs.agno.com
from agno.agent import Agent
from agno.models.vllm import vLLM
from agno.tools.duckduckgo import DuckDuckGoTools
agent = Agent(
model=vLLM(
id="NousResearch/Nous-Hermes-2-Mistral-7B-DPO", top_k=20, enable_thinking=False
),
tools=[DuckDuckGoTools()],
show_tool_calls=True,
markdown=True,
)
agent.print_response("Whats happening in France?", stream=True)
Create a virtual environment
Terminal and create a python virtual environment.python3 -m venv .venv
source .venv/bin/activate
python3 -m venv .venv
.venv/scripts/activate
Install Libraries
pip install -U agno openai vllm duckduckgo-search
Start vLLM server
vllm serve NousResearch/Nous-Hermes-2-Mistral-7B-DPO \
--enable-auto-tool-choice \
--tool-call-parser hermes \
--dtype float16 \
--max-model-len 8192 \
--gpu-memory-utilization 0.9
Run Agent
python cookbook/models/vllm/tool_use.py
Was this page helpful?