VRChat Avatar Mediator via OSC

Introduction

Software agents designed to execute tasks based on user commands define the concept of a virtual assistant. This specific tool facilitates the connection between such sophisticated AI agents and the VRChat environment. By utilizing Open Sound Control (OSC), it allows an external system to dynamically control user representations and interactions within virtual reality, extending capabilities beyond simple text interaction.

Key Capabilities

• Avatar Manipulation: Adjusting parameters and expressive states of the user's in-world avatar.
• Locomotion Control: Issuing commands to govern avatar movement and spatial orientation.
• Communication: Directly injecting text messages into the VRChat display area.
• Interface Access: Programmatically toggling the main system menus.
• Data Retrieval: Querying the characteristics and current settings of the active avatar.
• Automatic Configuration: Effortlessly recognizing and adapting to existing avatar setups within VRChat.

System Prerequisites

• The runtime environment requires Node.js, version 18 or later.
• VRChat must be running with its native OSC functionality explicitly enabled.
• The controlling entity must be a compatible agent supporting the MCP framework, such as Claude Desktop.

Setup Procedure

Clone and npm link

To prepare the necessary components locally, first obtain the source repository and then register it for global npm use:

git clone https://github.com/Krekun/vrchat-mcp-osc
cd vrchat-mcp-osc
npm link

Configure Claude Desktop

Adjust the configuration file for the controlling application, specifically the claude_desktop_config.json file, to recognize this service:

{
  "mcpServers": {
    "vrchat-mcp-osc": {
      "command": "npx",
      "args": [
        "vrchat-mcp-osc"
      ]
    }
  }
}

Command Line Options

Customizing the server behavior is done via command-line arguments when launching the service. The following structure illustrates how various network and operational settings can be specified:

# Claude Desktop configuration
{
  "mcpServers": {
    "vrchat-mcp-osc": {
      "command": "npx",
      "args": [
        "vrchat-mcp-osc",
        "--websocket-port", "8765",
        "--websocket-host", "localhost",
        "--osc-send-port", "9000",
        "--osc-send-ip", "127.0.0.1",
        "--osc-receive-port", "9001",
        "--osc-receive-ip", "127.0.0.1",
        "--debug"             
      ]
    }
  }
}

Available Options

This table details the adjustable parameters for the server's operation:

Option	Description	Default	Notes
--websocket-port <port>	WebSocket port	8765	For WebSocket communication
--websocket-host <host>	WebSocket host	localhost	For WebSocket communication
--osc-send-port <port>	OSC send port	9000	Port for sending to VRChat
--osc-send-ip <ip>	OSC send IP	127.0.0.1	Address for sending to VRChat
--osc-receive-port <port>	OSC receive port	9001	Port for receiving from VRChat
--osc-receive-ip <ip>	OSC receive IP	127.0.0.1	Address for receiving from VRChat
--debug	Enable debug logging	false	Output detailed logs
--no-relay	Disable relay server	false	When not using relay server

Exposed MCP Functions

This component presents several discrete functions to connected AI assistants for environmental manipulation:

Tool Name	Description
get_avatar_name	Retrieves the current avatar's name
get_avatar_parameters	Lists available avatar parameters
set_avatar_parameter	Sets a specific avatar parameter
set_emote_parameter	Triggers avatar emotes
move_avatar	Moves the avatar in a specific direction
look_direction	Controls avatar's view direction
jump	Makes the avatar jump
menu	Toggles the VRChat menu
voice	Toggles voice features
send_message	Sends a message to the VRChat chatbox

Troubleshooting

Common Issues

1. VRChat not responding to commands
2. Confirm that OSC communication is toggled on within the VRChat settings menu.
3. Verify port numbers align precisely between the VRChat application and the MCP configuration settings.

4. Restarting both VRChat and the controlling assistant application can often resolve temporary connection issues.

5. MCP server not starting

6. Confirm that Node.js version 18 or newer is correctly installed on the system.
7. Carefully review the command line parameters provided during the launch sequence for syntactical errors.

8. Initiating execution with the --debug flag is recommended for obtaining verbose diagnostic output.

9. NPX execution issues

10. If command-line arguments appear ignored, attempt execution using the explicit separator: npx vrchat-mcp-osc -- --debug.
11. On systems running Windows, try executing the command shell with elevated administrative permissions.
12. If local linking proves problematic, the previously described local npm link method should be re-verified.

Project Structure

The software is organized into several distinct packages within the workspace definition:

vrchat-mcp-osc/
├── packages/
│   ├── mcp-server/    # MCP server implementation (main entry point)
│   ├── relay-server/  # WebSocket to OSC relay
│   ├── types/         # Shared TypeScript interfaces
│   └── utils/         # Common utilities
└── pnpm-workspace.yaml  # Workspace configuration

Development

Build From Source

Developers can rebuild the tool from the source code using these commands:

# Clone the repository
git clone https://github.com/Krekun/vrchat-mcp-osc
cd vrchat-mcp-osc

# Install dependencies
pnpm install

# Build all packages
pnpm -r build

# Development mode
pnpm -r dev

License Details

The distribution of this software is governed by two distinct licensing agreements. Users must adhere to the terms applicable to their usage scenario.

For Non-Commercial Use: This software component is provided under the terms of the MIT License, permitting modification and redistribution under those specific conditions. (Refer to the included MIT License file for complete stipulations.)

For Commercial Use: Any utilization intended for commercial purposes requires securing a separate, explicit commercial licensing agreement.

By employing this software for non-commercial activities under the MIT License, users formally accept the stated license conditions. Commercial entities must obtain the necessary commercial license first.

Acknowledgments

Special thanks are extended to several contributing entities:

• The VRChat development team for providing the foundational OSC interface capabilities.
• The Model Context Protocol group for standardizing the necessary interface structure for AI interaction.
• Anthropic for their implementation efforts regarding the MCP specification utilized by Claude.

Related Topics

• Open Sound Control (OSC) networking protocols for digital performance.
• Chatbot technology and natural language understanding capabilities.
• Virtual Reality (VR) user interface paradigms and immersion.
• Model Context Protocol (MCP) for standardized agent communication.
• Apple Siri, Amazon Alexa, and Google Assistant as examples of consumer VAs.

Extra Details

This tool acts as a critical translation layer, converting abstract AI intentions into precise network packets compatible with proprietary application protocols like OSC. While prominent consumer virtual assistants often focus on home automation or scheduling, this integration prioritizes real-time, sensory manipulation within persistent social virtual worlds. The underlying OSC protocol efficiently handles the low-latency data required for smooth avatar animation and expression updates.

Conclusion

This bridge successfully integrates advanced AI-driven processing power with rich, interactive virtual environments. It allows software agents, which interpret user needs via text or voice, to execute complex actions within simulated spaces. This capability advances the utility of virtual assistants by extending their functionality from simple data retrieval to direct, dynamic participation in shared digital realities.