๐Ÿ”Œ Creating Plugins

Build custom tools for the voice assistant

๐Ÿค–

ROBO CODED โ€” This documentation was made with AI and may not be 100% sane. But the code does work! ๐ŸŽ‰

๐Ÿ”Œ Creating Plugins

Extend the SIP AI Assistant with custom tools by creating Python plugins.

Plugin code

๐ŸŽฏ Plugin Basics

Plugins are Python files in src/plugins/ that define tool classes:

src/plugins/
โ”œโ”€โ”€ __init__.py
โ”œโ”€โ”€ weather_tool.py
โ”œโ”€โ”€ timer_tool.py
โ”œโ”€โ”€ callback_tool.py
โ””โ”€โ”€ my_custom_tool.py   # ๐Ÿ‘ˆ Your plugin here!

Each plugin:

  1. ๐Ÿ“ฆ Inherits from BaseTool
  2. ๐Ÿ“ Defines name, description, and parameters
  3. โšก Implements an execute() method

๐Ÿš€ Quick Start

Step 1: Create the Plugin File

Create src/plugins/hello_tool.py:

"""
Hello Tool
==========
A simple greeting tool.
"""

from typing import Any, Dict
from tool_plugins import BaseTool, ToolResult, ToolStatus


class HelloTool(BaseTool):
    # ๐Ÿท๏ธ Unique tool name (UPPERCASE)
    name = "HELLO"
    
    # ๐Ÿ“ Description shown to LLM
    description = "Say hello to someone"
    
    # โœ… Enable/disable
    enabled = True
    
    # ๐Ÿ“‹ Parameter definitions
    parameters = {
        "name": {
            "type": "string",
            "description": "Name to greet",
            "required": True
        }
    }
    
    async def execute(self, params: Dict[str, Any]) -> ToolResult:
        name = params.get("name", "friend")
        
        return ToolResult(
            status=ToolStatus.SUCCESS,
            message=f"Hello, {name}! Nice to meet you."
        )

Step 2: Register the Plugin

Edit src/tool_manager.py:

def _load_tools(self):
    """Load all tool plugins."""
    # Add import at top of method
    from plugins.hello_tool import HelloTool
    
    tool_classes = [
        # ... existing tools ...
        HelloTool,  # ๐Ÿ‘ˆ Add your tool here
    ]

Step 3: Restart and Test

# Restart the service
docker compose restart sip-agent

# Verify tool is loaded
curl http://localhost:8080/tools | jq '.[].name'

Expected output:

"WEATHER"
"SET_TIMER"
"CALLBACK"
...
"HELLO"    # ๐Ÿ‘ˆ Your new tool!

Step 4: Test Execution

curl -X POST http://localhost:8080/tools/HELLO/execute \
  -H "Content-Type: application/json" \
  -d '{"params": {"name": "World"}}' | jq

Expected output:

{
  "success": true,
  "tool": "HELLO",
  "message": "Hello, World! Nice to meet you.",
  "data": null,
  "spoken": false,
  "error": null
}
Plugin test

๐Ÿ“‹ Tool Class Structure

from typing import Any, Dict
from tool_plugins import BaseTool, ToolResult, ToolStatus


class MyTool(BaseTool):
    # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
    # ๐Ÿท๏ธ REQUIRED: Unique tool name (UPPERCASE)
    # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
    name = "MY_TOOL"
    
    # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
    # ๐Ÿ“ REQUIRED: Description (shown to LLM)
    # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
    description = "What this tool does"
    
    # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
    # โœ… OPTIONAL: Enable/disable (default: True)
    # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
    enabled = True
    
    # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
    # ๐Ÿ“‹ OPTIONAL: Parameter definitions
    # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
    parameters = {
        "param_name": {
            "type": "string",      # string, integer, number, boolean
            "description": "...",   # Help text for LLM
            "required": True,       # Required or optional
            "default": "value"      # Default if not provided
        }
    }
    
    # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
    # ๐Ÿ”ง OPTIONAL: Custom initialization
    # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
    def __init__(self, assistant):
        super().__init__(assistant)
        # Access config: self.config
        # Access assistant: self.assistant
    
    # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
    # โšก REQUIRED: Execute the tool
    # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
    async def execute(self, params: Dict[str, Any]) -> ToolResult:
        # Your tool logic here
        return ToolResult(
            status=ToolStatus.SUCCESS,
            message="Result to speak to user",
            data={"optional": "structured data"}
        )

๐Ÿ“Š Parameter Types

TypePython TypeExample Value
stringstr"hello"
integerint42
numberfloat3.14
booleanbooltrue / false

Parameters are automatically validated and converted.


๐Ÿ“ค Return Values

Always return a ToolResult:

from tool_plugins import ToolResult, ToolStatus

# โœ… Success
return ToolResult(
    status=ToolStatus.SUCCESS,
    message="Message to speak to user",
    data={"key": "value"}  # Optional structured data
)

# โŒ Failure
return ToolResult(
    status=ToolStatus.FAILED,
    message="Error message to speak"
)

# โณ Pending (for async operations)
return ToolResult(
    status=ToolStatus.PENDING,
    message="Working on it...",
    data={"task_id": "abc123"}
)

๐Ÿ”— Accessing Resources

Your tool has access to:

class MyTool(BaseTool):
    async def execute(self, params):
        # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
        # ๐Ÿ“‹ Configuration
        # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
        api_key = self.config.my_api_key
        base_url = self.config.my_base_url
        
        # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
        # ๐Ÿ“ž Current call info
        # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
        if self.assistant.current_call:
            caller_id = self.assistant.current_call.caller
            call_duration = self.assistant.current_call.duration
        
        # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
        # โฐ Schedule tasks
        # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
        task_id = await self.assistant.tool_manager.schedule_task(
            task_type="my_task",
            delay_seconds=60,
            message="Task complete"
        )
        
        # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
        # ๐ŸŒ HTTP requests
        # โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
        import httpx
        async with httpx.AsyncClient() as client:
            response = await client.get("https://api.example.com")
            data = response.json()

๐Ÿ“– Full Example: Stock Price Tool

"""
Stock Price Tool
================
Get current stock prices from an API.
"""

import logging
import httpx
from typing import Any, Dict

from tool_plugins import BaseTool, ToolResult, ToolStatus

logger = logging.getLogger(__name__)


class StockPriceTool(BaseTool):
    name = "STOCK"
    description = "Get the current price of a stock"
    enabled = True
    
    parameters = {
        "symbol": {
            "type": "string",
            "description": "Stock ticker symbol (e.g., AAPL, GOOGL)",
            "required": True
        }
    }
    
    def __init__(self, assistant):
        super().__init__(assistant)
        # ๐Ÿ”ง Check for required config
        self.api_key = getattr(self.config, 'stock_api_key', None)
        if not self.api_key:
            self.enabled = False
            logger.info("๐Ÿ“‰ Stock tool disabled - no API key configured")
    
    async def execute(self, params: Dict[str, Any]) -> ToolResult:
        symbol = params.get("symbol", "").upper()
        
        # โœ… Validate input
        if not symbol:
            return ToolResult(
                status=ToolStatus.FAILED,
                message="Please specify a stock symbol"
            )
        
        try:
            # ๐ŸŒ Fetch stock data
            async with httpx.AsyncClient(timeout=10.0) as client:
                response = await client.get(
                    f"https://api.stockprovider.com/quote/{symbol}",
                    headers={"Authorization": f"Bearer {self.api_key}"}
                )
                
                if response.status_code == 404:
                    return ToolResult(
                        status=ToolStatus.FAILED,
                        message=f"Stock symbol {symbol} not found"
                    )
                
                response.raise_for_status()
                data = response.json()
            
            # ๐Ÿ“Š Format result
            price = data.get("price", 0)
            change = data.get("change", 0)
            change_pct = data.get("change_percent", 0)
            
            direction = "up" if change >= 0 else "down"
            
            return ToolResult(
                status=ToolStatus.SUCCESS,
                message=f"{symbol} is at ${price:.2f}, {direction} {abs(change_pct):.1f}% today",
                data={
                    "symbol": symbol,
                    "price": price,
                    "change": change,
                    "change_percent": change_pct
                }
            )
            
        except httpx.TimeoutException:
            return ToolResult(
                status=ToolStatus.FAILED,
                message="Stock service is not responding"
            )
        except Exception as e:
            logger.error(f"Stock fetch error: {e}")
            return ToolResult(
                status=ToolStatus.FAILED,
                message="Error fetching stock data"
            )

๐Ÿ  Full Example: Home Assistant Integration

"""
Home Assistant Tool
===================
Control smart home devices via Home Assistant.
"""

import logging
import httpx
from typing import Any, Dict

from tool_plugins import BaseTool, ToolResult, ToolStatus

logger = logging.getLogger(__name__)


class HomeAssistantTool(BaseTool):
    name = "HOME"
    description = "Control smart home devices (lights, switches, etc.)"
    enabled = True
    
    parameters = {
        "action": {
            "type": "string",
            "description": "Action: on, off, toggle, status",
            "required": True
        },
        "device": {
            "type": "string",
            "description": "Device name or entity ID",
            "required": True
        }
    }
    
    # ๐Ÿ“‹ Device name mappings
    DEVICE_MAP = {
        "living room light": "light.living_room",
        "bedroom light": "light.bedroom",
        "kitchen light": "light.kitchen",
        "front door": "lock.front_door",
        "garage": "cover.garage_door",
    }
    
    def __init__(self, assistant):
        super().__init__(assistant)
        self.ha_url = getattr(self.config, 'home_assistant_url', None)
        self.ha_token = getattr(self.config, 'home_assistant_token', None)
        
        if not self.ha_url or not self.ha_token:
            self.enabled = False
            logger.info("๐Ÿ  Home Assistant tool disabled - not configured")
    
    async def execute(self, params: Dict[str, Any]) -> ToolResult:
        action = params.get("action", "").lower()
        device = params.get("device", "")
        
        # ๐Ÿ” Resolve device name to entity ID
        entity_id = self.DEVICE_MAP.get(device.lower(), device)
        
        headers = {
            "Authorization": f"Bearer {self.ha_token}",
            "Content-Type": "application/json"
        }
        
        try:
            async with httpx.AsyncClient(timeout=10.0) as client:
                if action == "status":
                    # ๐Ÿ“Š Get current state
                    response = await client.get(
                        f"{self.ha_url}/api/states/{entity_id}",
                        headers=headers
                    )
                    data = response.json()
                    state = data.get("state", "unknown")
                    
                    return ToolResult(
                        status=ToolStatus.SUCCESS,
                        message=f"The {device} is currently {state}"
                    )
                else:
                    # โšก Perform action
                    domain = entity_id.split(".")[0]
                    service = f"turn_{action}" if action != "toggle" else "toggle"
                    
                    response = await client.post(
                        f"{self.ha_url}/api/services/{domain}/{service}",
                        headers=headers,
                        json={"entity_id": entity_id}
                    )
                    
                    if response.status_code == 200:
                        return ToolResult(
                            status=ToolStatus.SUCCESS,
                            message=f"Done! The {device} is now {action}"
                        )
                    else:
                        return ToolResult(
                            status=ToolStatus.FAILED,
                            message=f"Couldn't {action} the {device}"
                        )
                        
        except Exception as e:
            logger.error(f"Home Assistant error: {e}")
            return ToolResult(
                status=ToolStatus.FAILED,
                message="Error communicating with Home Assistant"
            )

โœ… Best Practices

1. ๐Ÿ›ก๏ธ Handle Errors Gracefully

try:
    result = await some_api_call()
except Exception as e:
    logger.error(f"Error: {e}")
    return ToolResult(
        status=ToolStatus.FAILED,
        message="I couldn't complete that request"  # ๐Ÿ‘ˆ User-friendly!
    )

2. โœ”๏ธ Validate Input

async def execute(self, params):
    value = params.get("value")
    
    if not value:
        return ToolResult(
            status=ToolStatus.FAILED,
            message="Please provide a value"
        )
    
    if len(value) > 100:
        return ToolResult(
            status=ToolStatus.FAILED,
            message="Value is too long"
        )

3. ๐Ÿ“ Use Logging

from logging_utils import log_event

log_event(logger, logging.INFO, "Tool executed",
         event="my_tool_success", 
         data={"key": "value"})

4. ๐Ÿ—ฃ๏ธ Keep Messages Conversational

# โœ… Good - natural speech
message = "The temperature is 72 degrees"

# โŒ Bad - robotic
message = "Temperature: 72ยฐF | Humidity: 45%"

5. ๐Ÿ”ง Disable When Unconfigured

def __init__(self, assistant):
    super().__init__(assistant)
    if not self.config.required_setting:
        self.enabled = False
        logger.info("Tool disabled - missing config")

๐Ÿ“ Plugin Directory Structure

src/plugins/
โ”œโ”€โ”€ ๐Ÿ“„ __init__.py          # Package marker
โ”œโ”€โ”€ ๐Ÿ“„ README.md            # Plugin documentation
โ”œโ”€โ”€ ๐Ÿ“„ timer_tool.py        # โฒ๏ธ SET_TIMER
โ”œโ”€โ”€ ๐Ÿ“„ callback_tool.py     # ๐Ÿ“ž CALLBACK
โ”œโ”€โ”€ ๐Ÿ“„ weather_tool.py      # ๐ŸŒค๏ธ WEATHER
โ”œโ”€โ”€ ๐Ÿ“„ hangup_tool.py       # ๐Ÿ“ด HANGUP
โ”œโ”€โ”€ ๐Ÿ“„ status_tool.py       # ๐Ÿ“‹ STATUS
โ”œโ”€โ”€ ๐Ÿ“„ cancel_tool.py       # โŒ CANCEL
โ”œโ”€โ”€ ๐Ÿ“„ datetime_tool.py     # ๐Ÿ• DATETIME
โ”œโ”€โ”€ ๐Ÿ“„ calc_tool.py         # ๐Ÿงฎ CALC
โ”œโ”€โ”€ ๐Ÿ“„ joke_tool.py         # ๐Ÿ˜„ JOKE
โ””โ”€โ”€ ๐Ÿ“„ my_custom_tool.py    # ๐Ÿ”Œ Your plugins!

๐Ÿงช Testing Checklist

# 1. โœ… Tool appears in list
curl http://localhost:8080/tools | jq '.[].name' | grep MY_TOOL

# 2. โœ… Tool info is correct
curl http://localhost:8080/tools/MY_TOOL | jq

# 3. โœ… Execution works
curl -X POST http://localhost:8080/tools/MY_TOOL/execute \
  -H "Content-Type: application/json" \
  -d '{"params": {"param1": "value1"}}' | jq

# 4. โœ… Error handling works
curl -X POST http://localhost:8080/tools/MY_TOOL/execute \
  -H "Content-Type: application/json" \
  -d '{"params": {}}' | jq  # Missing required param

# 5. โœ… Voice test - make a call and try it!