Universal LLM Development Harness with Vendor Agnostic Backend

A robust Python implementation mirroring advanced code assistants, augmented with superior graphical feedback, expenditure tracking, and a full Model Context Protocol (MCP) server utility. This framework delivers an intuitive natural language interface tailored specifically for programming tasks, supporting numerous upstream LLM suppliers.

Core Capabilities

• Vendor Flexibility: Interoperates seamlessly with offerings from OpenAI, Anthropic, and auxiliary model services.
• MCP Compliance:
• Functions as a fully compliant MCP node for connection with specialized desktop environments (like Claude) or other client applications.
• Provides built-in client functionality to link with any established MCP node.
• Enables synchronous coordination among distinct operational agents for challenging undertakings.
• Live Execution Feedback: Provides immediate graphical representation of tool invocation sequences and resultant outputs.
• Financial Oversight: Maintains detailed ledgers of token consumption and imposes expenditure ceilings.
• Extensive Utility Set: Includes functionality for file manipulation (read/write), pattern searching, and arbitrary shell command execution.
• Refined User Interface: Features a rich, interactive terminal presentation with real-time status indicators and formatted code output.
• Contextual Pruning: Implements intelligent mechanisms for history summarization and memory conservation.
• Agential Teamwork: Supports role-specific computational entities that collaborate dynamically on defined objectives.

Deployment Instructions

1. Obtain a local copy of the source code repository
2. Resolve dependencies:

bash pip install -r requirements.txt

1. Establish a configuration file named .env containing necessary credentials:

Select one or more processing engines

OPENAI_API_KEY=your_secret_openai_key ANTHROPIC_API_KEY=your_secret_anthropic_key

Optional model selection overrides

OPENAI_MODEL=gpt-4o-mini ANTHROPIC_MODEL=claude-3-sonnet-20240229

Operational Modes

Command Line Interface (CLI)

Execute the main script in interactive mode, defaulting to the first available API key:

bash python main_script.py initiate

Specify the computation source and target model:

bash python main_script.py initiate --source openai --model gpt-4-turbo

Enforce budgetary restrictions:

bash python main_script.py initiate --cap 10.00

MCP Node Operation

Activate the process as a Model Context Protocol server:

bash python main_script.py start-node

Initiate in diagnostic mode with the integrated MCP Inspector:

bash python main_script.py start-node --debug

Customize network binding parameters:

bash python main_script.py start-node --interface 0.0.0.0 --port 9001

Incorporate external libraries at runtime:

bash python main_script.py start-node --extras requests beautifulsoup4

Load configurations from an alternate location:

bash python main_script.py start-node --config-file production.env

MCP Client Functionality

Establish a connection to a remote MCP entity utilizing the system as the inference engine:

bash python main_script.py connect-node /path/to/remote_definition.mcp

Designate a specific reasoning model for the client session:

bash python main_script.py connect-node /path/to/remote_definition.mcp --reasoner gpt-4o

Test with the supplied demonstration endpoint:

bash

Session 1 - Initiate the listener

python examples/listener.py

Session 2 - Connect the processing client

python main_script.py connect-node examples/listener.py

Synchronized Agent Cluster Mode

Launch an orchestrated client where multiple agents operate concurrently:

bash python main_script.py orchestrate /path/to/server_manifest.json

Utilize a custom coordination blueprint file:

bash python main_script.py orchestrate /path/to/server_manifest.json --blueprint custom_roles.yaml

Example utilizing the echo listener:

bash

Terminal A - Start the server

python examples/listener.py

Terminal B - Start the coordinated client

python main_script.py orchestrate examples/listener.py --blueprint examples/agent_spec.json

Integrated Capabilities

• Peek: Retrieve file contents, optionally restricting line count.
• Mutate: Alter file contents via targeted text replacement.
• Persist: Write new content or overwrite existing files.
• LocatorTool: Discover files matching specific path patterns.
• ScannerTool: Search within file data using regular expressions.
• DirectoryLister: Enumerate directory structures.
• ShellExecutor: Execute arbitrary operating system commands.

Runtime Directives

• /guide: Display available system commands.
• /trim: Condense dialogue history for token conservation.
• /info: Output current version details.
• /sources: List all active LLM suppliers.
• /finance: Report accumulated expenditure and usage statistics.
• /cap [value]: Define a maximum spend threshold.
• /exit, /shutdown: Terminate the application session.

System Blueprint

The PLSF structure is founded upon a modular decomposition:

/plsf_core/ /engines/ # LLM vendor specific implementations /utilities/ # Tool definitions and executors /memory/ # Context and state management /interface/ # User interaction components (TUI/Web) /metrics/ /workflows/ # Command handling definitions /configuration/ # Setting persistence /helpers/ # General utilities main_script.py # Primary CLI entry point mcp_node.py # Model Context Protocol server logic

Interfacing via Model Context Protocol (MCP)

Utilizing PLSF as an MCP Gateway

After initializing the MCP node, external MCP-aware clients can connect:

1. Run the server module: bash python main_script.py start-node

2. Access the configuration interface in a browser:

http://localhost:9001

1. Follow prompts to integrate with external applications (e.g., copy/download setup manifests).

Employing PLSF as an MCP Endpoint Client

To utilize PLSF to communicate with any standardized MCP hub:

1. Ensure required API credentials (e.g., Anthropic key) are set.
2. Verify the target MCP node is operational.

3. Connect using the client module: bash python main_script.py connect-node /path/to/remote_descriptor.mcp

4. Input requests into the interactive shell.

Utilizing Coordinated Agent Clusters

For highly complex tasks requiring specialized perspectives:

1. Prepare the orchestration configuration file.
2. Ensure the target MCP service is available.

3. Launch the synchronized client: bash python main_script.py orchestrate /path/to/server_manifest.json --blueprint examples/roles.json

4. Interact via the specialized directive system:

5. General input messages are broadcast to all agents.
6. Use /relay Agent_ID message for direct messaging.
7. Use /roster to list active agents.
8. Use /transcript to review recent interactions.

Contribution Guidelines

1. Fork the repository source.
2. Establish a dedicated branch for your feature development.
3. Implement changes, ensuring unit tests validate correctness.
4. Submit a comprehensive Merge Request.

Licensing

MIT License

Credits

This framework draws inspiration from Anthropic's established command-line tool, re-engineered in Python to incorporate enhanced observability, fiscal control, and full MCP server capability.# Multi-Vendor LLM Orchestrator (MVOLO)

A versatile command-line utility and software interface leveraging OpenAI's capabilities alongside function invoking and output streaming.

Features

• Interactive terminal for development support.
• Web-accessible interface for external application linking.
• Integrated Model Context Protocol (MCP) service daemon.
• System redundancy mechanisms for persistent uptime.
• Modular toolset architecture for adaptability.
• Optimization driven by adaptive feedback loops (RL).
• Browser-based front-end interface.

Setup

1. Clone the project repository.

2. Install necessary libraries: bash pip install -r requirements.txt

3. Export your primary API access token: bash export OPENAI_API_KEY=your_auth_token

Execution Profiles

Terminal Mode

Start the assistant interactively via the shell:

bash python terminal_interface.py run

Parameters: - --engine, -e: Designate the model engine (default: gpt-4o-turbo). - --creativity, -c: Set generation randomness factor (default: 0.1). - --diagnostic, -d: Activate verbose logging. - --activate-rl/--deactivate-rl: Toggle adaptive tool refinement. - --train-rl: Force an immediate update cycle for the optimization model.

Network Service Mode

Initiate the system as a background API endpoint:

bash python terminal_interface.py serve-api

Parameters: - --interface: Network interface address to bind (default: 127.0.0.1). - --port, -p: Listening socket number (default: 9000). - --threads, -t: Process concurrency level (default: 1). - --redundancy-mode: Enable state synchronization across nodes. - --master/--slave: Designate node role. - --link: Specify peer nodes for replication (address:port, repeatable).

MCP Daemon Mode

Run the orchestrator configured as an MCP communication hub:

bash python terminal_interface.py serve-mcp

Parameters: - --interface: Binding address (default: 127.0.0.1). - --port, -p: Listening port (default: 9000). - --audit: Enable heightened debugging trace output. - --inject-deps: List supplementary Python packages to load. - --context-file: Specify a file path for environment settings.

MCP Client Connector

Use the orchestrator to interface with a specified MCP node:

bash python terminal_interface.py connect-mcp /path/to/node_config.mcp

Parameters: - --engine, -e: Preferred reasoning model (default: gpt-4o-turbo). - --target-host: Address of the remote MCP server (default: 127.0.0.1). - --target-port, -p: Port of the remote MCP server (default: 9000).

Deployment Utility

For streamlined deployment, utilize the included launch script:

bash ./launcher.sh --interface 0.0.0.0 --port 9000 --threads 4

To enable high-availability state synchronization:

bash

Primary Node

./launcher.sh --redundancy-mode --port 9000

Replica Node

./launcher.sh --redundancy-mode --slave --port 9001 --link 127.0.0.1:9000

Web Frontend

Access the graphical interface by loading client/index.html in a compatible web browser once the API service is active.

Service Endpoints

Core API Endpoints

• POST /session: Establish a new dialogue context.
• POST /session/{id}/input: Submit user input to a dialogue context.
• POST /session/{id}/input/stream: Stream response from a dialogue context.
• GET /session/{id}: Retrieve dialogue context details.
• DELETE /session/{id}: Terminate a dialogue context.
• GET /system/health: Operational status check.

MCP Interface Endpoints

• GET /mcp/status: Health check (MCP standard).
• POST /mcp/query_context: Request context data for a prompt schema.
• GET /mcp/schemas: List available prompt definitions.
• GET /mcp/schemas/{id}: Fetch a specific prompt definition.
• POST /mcp/schemas: Define a new prompt schema.
• PUT /mcp/schemas/{id}: Update an existing prompt definition.
• DELETE /mcp/schemas/{id}: Remove a prompt definition.

State Synchronization

The replication architecture facilitates running clustered instances with shared operational state, ensuring:

• Increased uptime resilience.
• Load distribution capabilities.
• Robustness against single point failures.

To configure synchronization: 1. Deploy the primary instance using --redundancy-mode. 2. Deploy subsequent replicas using --redundancy-mode --slave --link [primary_address:port].

Toolkit Inventory

The orchestrator incorporates several specialized execution modules: - ExternalData: Fetch real-time environmental metrics. - FileReader: Access and parse local files. - FileWriter: Modify or introduce file content. - FileWriter: Overwrite content entirely. - ShellExecutor: Execute arbitrary shell commands. - PatternMatcher: Locate files based on wildcard expressions. - ContentScanner: Perform regex-based textual analysis across files. - DirLister: Inventory directory listings. - JinaSearch: Perform web queries via Jina. - JinaFactCheck: Verify claims using Jina's knowledge base. - WebReader: Ingest and summarize data from specified URLs.

Shell Directives

• /help: Display command reference.
• /summarize: Compact conversation history for memory optimization.
• /metrics: Output current token utilization and session metrics.
• /settings: Review active configuration parameters.
• /rl-report: Show status of adaptive tool selection (if active).
• /rl-execute: Trigger manual RL model refinement (if active).
• /rl-data: Display statistics on tool invocation frequency (if active).

Reference Knowledge (Cloud Computing)

Cloud computing is defined by ISO as "a framework for facilitating ubiquitous, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction."

NIST identified five key attributes in 2011:

On-demand self-service: Consumers provision resources autonomously without vendor staff intervention. Ubiquitous Network Availability: Services are accessible via standard protocols across diverse endpoint hardware. Resource Aggregation: The provider's infrastructure capacity is dynamically allocated across numerous tenants (multi-tenancy). Elastic Scalability: Capacity can expand or contract quickly, often automatically, appearing limitless from the user's viewpoint. Measured Consumption: Resource usage (storage, cycles, bandwidth) is automatically tracked, reported, and controlled, ensuring transparency.

Early concepts date to the 1960s with time-sharing systems facilitating shared mainframe access. The visual 'cloud' metaphor for networking abstraction originated around 1994 within General Magic's environment, later gaining commercial traction in 1996.