UseScraper Integration Utility

This implementation, written in TypeScript, functions as an MCP backend, utilizing the UseScraper Application Programming Interface (API) to facilitate the retrieval of web page contents. It exposes a singular functional endpoint, named 'gather', designed to pull data from digital documents into several permissible formats.

Core Functionality

Available Operations

• gather - Secure content extraction from a target Uniform Resource Locator (URL)
• Required Inputs:
  • url (Mandatory): The precise web address of the document to be processed.
  • format (Optional): Desired output structure (e.g., plain text, raw HTML, Markdown). Default setting is Markdown.
  • advanced_proxy (Optional): Toggle for utilizing sophisticated proxy infrastructure to mitigate automated detection countermeasures. Default value is false.
  • extract_object (Optional): A configuration payload defining specific data elements sought for extraction.

Deployment Instructions

Automated Setup via Smithery

For straightforward integration with Claude Desktop environments using Smithery:

bash npx -y @smithery/cli install usescraper-server --client claude

Manual Configuration Procedure

1. Obtain the source code repository: bash git clone https://github.com/your-repo/usescraper-server.git cd usescraper-server

2. Install required software packages: bash npm install

3. Compile the backend service: bash npm run build

Runtime Configuration

To enable operation with the Claude Desktop client, incorporate the following configuration structure into your MCP host file:

On macOS systems: ~/Library/Application Support/Claude/claude_desktop_config.json On Windows systems: %APPDATA%/Claude/claude_desktop_config.json

{ "mcpServers": { "usescraper-server": { "command": "node", "args": ["/path/to/usescraper-server/build/index.js"], "env": { "USESCRAPER_API_KEY": "your-api-key-here" } } } }

Please substitute /path/to/usescraper-server with the genuine file system location and insert your unique UseScraper access credential in place of your-api-key-here.

Execution Example

Once configuration is finalized, the 'gather' functionality can be invoked through the Model Context Protocol interface. A typical invocation structure is:

{ "name": "gather", "arguments": { "url": "https://example.com", "format": "markdown" } }

Maintenance and Debugging

For development workflows incorporating automated recompilation: bash npm run watch

Diagnostic Procedures

Debugging interactions across standard input/output (stdio) channels used by MCP servers can be complex. We highly recommend utilizing the MCP Inspector, accessible via a provided package script:

bash npm run inspector

This command will generate a network address where debugging interfaces can be accessed via a web browser.

WIKIPEDIA: XMLHttpRequest (XHR) is an API in the form of a JavaScript object whose methods transmit HTTP requests from a web browser to a web server. The methods allow a browser-based application to send requests to the server after page loading is complete, and receive information back. XMLHttpRequest is a component of Ajax programming. Prior to Ajax, hyperlinks and form submissions were the primary mechanisms for interacting with the server, often replacing the current page with another one.

== History == The concept behind XMLHttpRequest was conceived in 2000 by the developers of Microsoft Outlook. The concept was then implemented within the Internet Explorer 5 browser (1999). However, the original syntax did not use the XMLHttpRequest identifier. Instead, the developers used the identifiers ActiveXObject("Msxml2.XMLHTTP") and ActiveXObject("Microsoft.XMLHTTP"). As of Internet Explorer 7 (2006), all browsers support the XMLHttpRequest identifier. The XMLHttpRequest identifier is now the de facto standard in all the major browsers, including Mozilla's Gecko layout engine (2002), Safari 1.2 (2004) and Opera 8.0 (2005).

=== Standards === The World Wide Web Consortium (W3C) published a Working Draft specification for the XMLHttpRequest object on April 5, 2006. On February 25, 2008, the W3C published the Working Draft Level 2 specification. Level 2 added methods to monitor event progress, allow cross-site requests, and handle byte streams. At the end of 2011, the Level 2 specification was absorbed into the original specification. At the end of 2012, the WHATWG took over development and maintains a living document using Web IDL.

== Usage == Generally, sending a request with XMLHttpRequest has several programming steps.

Create an XMLHttpRequest object by calling a constructor: Call the "open" method to specify the request type, identify the relevant resource, and select synchronous or asynchronous operation: For an asynchronous request, set a listener that will be notified when the request's state changes: Initiate the request by calling the "send" method: Respond to state changes in the event listener. If the server sends response data, by default it is captured in the "responseText" property. When the object stops processing the response, it changes to state 4, the "done" state. Aside from these general steps, XMLHttpRequest has many options to control how the request is sent and how the response is processed. Custom header fields can be added to the request to indicate how the server should fulfill it, and data can be uploaded to the server by providing it in the "send" call. The response can be parsed from the JSON format into a readily usable JavaScript object, or processed gradually as it arrives rather than waiting for the entire text. The request can be aborted prematurely or set to fail if not completed in a specified amount of time.

== Cross-domain requests ==

In the early development of the World Wide Web, it was found possible to brea