Web Intelligence Connector - MCP Endpoint

This repository furnishes a Model Context Protocol (MCP) server implementation designed to bridge AI assistants with diverse, third-party information retrieval mechanisms (e.g., Google, Bing) utilizing the SearchAPI.site infrastructure.

Steward: Claude

• Access Portal
• Source Repository
• Package Registry

Supported Data Retrieval Channels

• [x] Google Nexus - Surface Data
• [x] Google Nexus - Pictorial Indexing
• [x] Google Nexus - Video Stream Lookup
• [ ] Google Nexus - Geospatial Querying
• [x] Bing Index - General Information Retrieval
• [ ] Bing Index - Visual Indexing
• [ ] Reddit Discussion Aggregation
• [ ] X/Twitter Feed Synthesis
• [ ] Facebook Public Post Scrutiny
• [ ] Facebook Community Aggregation
• [ ] Instagram Media Examination
• [ ] TikTok Content Indexing

SearchAPI.site Ecosystem

• Primary Site
• Interface Documentation
• OpenAPI Specification Configuration
• Generate Your Search Credential Here
• Underlying Search API GitHub

Accepted Communication Protocols

• [x] Standard Input/Output (stdio) transport - Default for command-line execution environments
• [x] Continuous Data Flow HTTP (Streamable HTTP) transport - Optimized for web-based consumers
• [ ] Authentication implementation (Utilizing 'Authorization' headers with Bearer <token>)
• [x] ~~Server-Sent Events (sse) transport~~ (Retired)
• [ ] Development of comprehensive validation routines

Operational Guidance

Command Line Interface (CLI)

bash

Execute a general web lookup via CLI

npm run dev:cli -- search-google --query "desired search phrase" --api-key "your-access-token"

Execute an image discovery request via CLI

npm run dev:cli -- search-google-images --query "visual concept" --api-key "your-access-token"

Execute a video platform query via CLI

npm run dev:cli -- search-youtube --query "video topic" --api-key "your-access-token" --max-results 5

MCP Configuration Setup

For local integration leveraging the stdio transport:

{ "mcpServers": { "searchapi": { "command": "node", "args": ["/path/to/searchapi-mcp-server/dist/index.js"], "transportType": "stdio" } } }

For connecting to a remote HTTP endpoint:

{ "mcpServers": { "searchapi": { "type": "http", "url": "http://mcp.searchapi.site/mcp" } } }

Configuration Variables for HTTP Transport:

Use these environment variables to customize the HTTP server binding:

• MCP_HTTP_HOST: Network interface address to bind to (default: 127.0.0.1)
• MCP_HTTP_PORT: Network port to monitor (default: 8080)
• MCP_HTTP_PATH: The URI path for the endpoint (default: /mcp)

---

Source Code Structure Map

What is MCP?

Model Context Protocol (MCP) establishes an open standard enabling secure, context-aware interconnection between artificial intelligence frameworks and external utilities or data reservoirs.

This project furnishes an MCP specification implementation employing a modular, clean hierarchy designed for straightforward adaptation to support any arbitrary API or knowledge base.

Advantages of this Boilerplate

• Production-Grade Blueprint: Adheres to the established scaffolding utilized in certified MCP services, featuring clear delineation between CLI interactions, tool definitions, control logic, and external service interfacing.

• Strong Typing: Developed in TypeScript, ensuring enhanced developer experience, superior code fidelity, and simplified long-term maintenance.

• Operational Demonstration: Incorporates a fully functional utility demonstrating the entire pipeline, from command invocation to external API integration (e.g., IP lookup).

• Testing Suite: Includes comprehensive infrastructure for both isolated unit validation and end-to-end CLI integration testing, complete with coverage metrics generation.

• Development Toolchain: Pre-configured with essential quality assurance tools: ESLint, Prettier, TypeScript, and other standard utilities for MCP server development.

---

Initial Setup Procedures

Necessary Preconditions

• Node.js Runtime (Minimum version 18.x): Acquire Here
• Git Version Control System: Essential for source code management

---

Phase 1: Acquisition and Dependency Resolution

bash

Clone the project repository

git clone https://github.com/mrgoonie/searchapi-mcp-server.git cd searchapi-mcp-server

Install all required software packages

npm install

---

Phase 2: Launching the Development Instance

Invoke the server in a live development mode utilizing the default stdio transport:

bash npm run dev:server

Alternatively, initiate with the Streamable HTTP protocol support:

bash npm run dev:server:http

This action launches the MCP service with real-time code reloading and activates the MCP Inspector interface at http://localhost:5173.

⚙️ Proxy service actively monitoring port 6277 🔍 MCP Inspector operational at http://127.0.0.1:6274

When using the HTTP transport, the service endpoint will default to http://127.0.0.1:8080/mcp.

---

Phase 3: Validating the Sample Utility

Execute the integrated IP resolution utility from the CLI environment:

bash

Using CLI in active development mode

npm run dev:cli -- search-google --query "your search query" --api-key "your-api-key"

Or testing with explicit parameters

npm run dev:cli -- search-google --query "your search query" --api-key "your-api-key" --limit 10 --offset 0 --sort "date:d" --from_date "2023-01-01" --to_date "2023-12-31"

---

System Architecture

This framework employs a rigorously layered architectural pattern to ensure separation of concerns and promote high maintainability.

Directory Organization

src/ ├── cli/ # Command-line interface handlers ├── controllers/ # Core operational logic implementation ├── resources/ # MCP resource definitions: expose data content to LLMs ├── services/ # Modules responsible for external API communication ├── tools/ # MCP tool specifications and schemas ├── types/ # Global type definitions and interfaces └── index.ts # Primary execution manifest

Functional Layer Responsibilities

CLI Subsystem (src/cli/*.cli.ts)

• Role: Define entry points for command-line invocation, handling argument parsing and delegating tasks to controllers.
• Naming Convention: Files must follow the <feature>.cli.ts convention.
• Validation: Corresponding integration tests are located in <feature>.cli.test.ts.

Tools Subsystem (src/tools/*.tool.ts)

• Role: Define the structure (schemas) and descriptions of MCP tools intended for AI consumption.
• Naming Convention: Files should be named <feature>.tool.ts, with associated types in <feature>.types.ts.
• Schema Handling: Utilizes Zod library exclusively for argument validation.

Controllers Subsystem (src/controllers/*.controller.ts)

• Role: Encapsulate the main business processing steps, manage error propagation, and structure the final output format.
• Naming Convention: Files should be named <feature>.controller.ts.
• Output Standard: Must return standardized ControllerResponse objects.

Services Subsystem (src/services/*.service.ts)

• Role: Handle direct, low-level interactions with external web services or databases.
• Naming Convention: Files should be named <feature>.service.ts.
• Principle: Should contain minimal application logic, focusing purely on API interaction.

Utilities Subsystem (src/utils/*.util.ts)

• Role: Provide globally shared, reusable functionality throughout the application.
• Key Utilities:
  • logger.util.ts: Structured logging mechanisms.
  • error.util.ts: Standardization of error management.
  • formatter.util.ts: Helpers for generating Markdown representations.

---

Development Workflow

Execution Scripts

bash

Initiate development server (auto-reload & Inspector enabled)

npm run dev:server

Invoke CLI within the development environment

npm run dev:cli -- [command] [arguments]

Compile source code to distributable assets

npm run build

Start the server using production optimization settings

npm run start:server

Invoke CLI using production-compiled assets

npm run start:cli -- [command] [arguments]

Testing Procedures

bash

Execute the entire test suite

npm test

Execute tests targeting a specific file or directory

npm test -- src/path/to/test.ts

Generate a detailed test coverage report

npm run test:coverage

Evaluation Frameworks (evals)

The integrated evals package automatically loads an MCP client configured to interact with the compiled index.ts. Recompilation between evaluation runs is unnecessary. Environment variables for external services (like OpenAI keys) should be prepended to the npx command. Comprehensive instructions are available here.

bash OPENAI_API_KEY=your-key npx mcp-eval src/evals/evals.ts src/tools/searchapi.tool.ts

Code Hygiene

bash

Check code against style rules

npm run lint

Apply automatic formatting via Prettier

npm run format

Verify all TypeScript types are sound

npm run typecheck

---

Blueprint for Custom Utility Addition

To integrate a novel data retrieval tool into this server, follow these systematic steps:

1. Establish the Service Interface

Develop a dedicated service module within src/services/ to manage direct interaction with your external API:

typescript // src/services/example.service.ts import { Logger } from '../utils/logger.util.js';

const logger = Logger.forContext('services/example.service.ts');

export async function retrieveData(identifier: string): Promise

2. Construct the Controller Handler

Create a controller within src/controllers/ to govern the business process flow:

typescript // src/controllers/example.controller.ts import { Logger } from '../utils/logger.util.js'; import * as exampleService from '../services/example.service.js'; import { formatMarkdown } from '../utils/formatter.util.js'; import { handleControllerError } from '../utils/error-handler.util.js'; import { ControllerResponse } from '../types/common.types.js';

const logger = Logger.forContext('controllers/example.controller.ts');

export interface RetrievalParameters { identifier?: string; }

export async function processDataRetrieval( onfig: RetrievalParameters = {}, ): Promise { try { logger.debug('Processing data request with configuration', config);

    const rawData = await exampleService.retrieveData(config.identifier || 'default-id');

    const formattedContent = formatMarkdown(rawData);

    return { content: formattedContent };
} catch (e) {
    throw handleControllerError(e, {
        entityType: 'CustomData',
        operation: 'processDataRetrieval',
        source: 'controllers/example.controller.ts',
    });
}

}

3. Define the MCP Tool Specification

Implement the tool definition file inside src/tools/:

typescript // src/tools/example.tool.ts import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js'; import { z } from 'zod'; import { Logger } from '../utils/logger.util.js'; import { mapToolError } from '../utils/error.util.js'; // Renamed for clarity import * as exampleController from '../controllers/example.controller.js';

const logger = Logger.forContext('tools/example.tool.ts');

const DataFetchSchema = z.object({ identifier: z.string().optional().describe('The unique ID required for fetching data.'), });

type DataFetchArgs = z.infer;

async function executeDataFetch(args: DataFetchArgs) { try { logger.debug('Tool invocation: execute_data_fetch', args);

    const response = await exampleController.processDataRetrieval({
        identifier: args.identifier,
    });

    return {
        result: [{ type: 'text' as const, value: response.content }],
    };
} catch (err) {
    logger.error('Tool execution failed: execute_data_fetch', err);
    return mapToolError(err);
}

}

export function register(server: McpServer) { server.tool( 'execute_data_fetch', Retrieves specialized information using the provided identifier. Essential for accessing custom data sets. Output is rendered in Markdown format., DataFetchSchema.shape, executeDataFetch, ); }

4. Integrate CLI Entry Point

Design the command-line interface module in src/cli/:

typescript // src/cli/example.cli.ts import { program } from 'commander'; import { Logger } from '../utils/logger.util.js'; import * as exampleController from '../controllers/example.controller.js'; import { handleCliError } from '../utils/error-handler.util.js';

const logger = Logger.forContext('cli/example.cli.ts');

program .command('fetch-custom-data') .description('Triggers the custom data retrieval operation via CLI') .option('--id ', 'Mandatory unique identifier') .action(async (opts) => { try { logger.debug('CLI command triggered: fetch-custom-data', opts);

        const output = await exampleController.processDataRetrieval({
            id: opts.id,
        });

        console.log(output.content);
    } catch (err) {
        handleCliError(err);
    }
});

5. Register Components Globally

Ensure your new modules are declared in the main entry files:

typescript // In src/cli/index.ts import '../cli/example.cli.js';

// In src/index.ts (for tool registration) import customToolDefinition from './tools/example.tool.js'; // Subsequently, in the tool registration function: customToolDefinition.register(server);

---

Debugging Utilities

Visual MCP Inspector

Utilize the graphical interface for testing tool execution and monitoring data flow:

1. Execute: npm run dev:server
2. Navigate to http://localhost:5173 in a modern web browser.
3. Test your defined utilities and inspect request/response payloads in real-time.

Verbose Server Logging

Activate detailed logging output during development sessions:

bash

Set the DEBUG environment variable

DEBUG=true npm run dev:server

Or configure persistent debug flags in ~/.mcp/configs.json

---

Deployment and Distribution

Once the service is finalized and validated:

1. Revise metadata within package.json.
2. Finalize documentation in README.md.
3. Generate optimized assets: npm run build.
4. Verify production runtime behavior: npm run start:server.
5. Publish the package to the npm registry: npm publish.

---

Licensing Information

ISC License

{ "searchapi": { "runtimeOptions": { "VERBOSE_MODE": "true", "KEY_GOOGLE_SEARCH": "secret_value_123" } } }

Compatibility Note: For legacy compatibility, the server will also recognize configuration maps keyed by the full name (searchapi-mcp-server) or the unscoped name if the preferred short key (searchapi) is absent. Adopting the concise searchapi key is the recommended practice for all novel configurations.

Contributors

• Claude AI Model (Code Generation)
• Goon (Primary Maintainer)

WIKIPEDIA_REFERENCE: Asynchronous JavaScript and XML (AJAX) relies on the XMLHttpRequest (XHR) interface—a standardized JavaScript object—to dispatch HTTP requests from a client browser to a remote server. This methodology permits web applications to fetch and update data post-page load without necessitating full page refreshes, which was the norm before AJAX. XHR was instrumental in shifting web interaction paradigms away from traditional form submissions and link navigation.

== Chronology == The foundational concepts for XHR were formalized in 2000 by developers working on Microsoft Outlook. This idea first materialized in the Internet Explorer 5 browser release (1999). Crucially, the initial implementation did not use the modern XMLHttpRequest naming convention, relying instead on COM object instantiations: ActiveXObject("Msxml2.XMLHTTP") or ActiveXObject("Microsoft.XMLHTTP"). By the time Internet Explorer 7 arrived (2006), universal support for the standardized XMLHttpRequest identifier was achieved across all major rendering engines, including Mozilla's Gecko (2002), Safari 1.2 (2004), and Opera 8.0 (2005).

=== Standardization Efforts === The World Wide Web Consortium (W3C) initiated the formal standardization of the XMLHttpRequest object with a Working Draft published on April 5, 2006. A subsequent Level 2 Working Draft appeared on February 25, 2008, introducing capabilities for progress monitoring, enabling cross-origin requests, and managing binary data streams. By the conclusion of 2011, the Level 2 feature set was integrated back into the main specification. Since 2012, the WHATWG has assumed stewardship, maintaining the specification as a living document defined using Web IDL.

== Standard Operational Flow == Utilizing XMLHttpRequest generally follows a sequential programming pattern:

1. Instantiation: Create an instance of the XMLHttpRequest object via its constructor.
2. Configuration: Invoke the open() method to define the request type (GET, POST, etc.), specify the target resource URI, and set the operational mode (synchronous or asynchronous).
3. Asynchronous Setup: If operating asynchronously, attach an event handler function to monitor state transitions.
4. Transmission: Trigger the request transmission by calling the send() method.
5. Response Handling: Process state changes within the listener. Upon successful completion (state 4, "done"), the server's textual response is typically available in the responseText attribute.

Beyond these core steps, XHR offers extensive configuration options for request control and response processing. Custom request headers can be injected, data can be uploaded within the send() argument, and the received text can be automatically parsed from JSON into native JavaScript objects or processed in chunks as data arrives. Requests can also be externally terminated or set to time out.