Pipedream serves as an integration platform tailored for software developers.

Pipedream furnishes a complimentary, managed environment for orchestrating connections between applications and engineering reactive automations. The platform boasts a repository exceeding 1,000 pre-configured integrations, enabling rapid deployment of components for tasks such as dispatching notifications to Slack or appending records to Google Sheets. For requirements necessitating bespoke logic, users are empowered to execute arbitrary code in Node.js, Python, Golang, or Bash. Pipedream maintains demonstrated SOC 2 compliance, with the Type 2 audit report accessible upon request via support@pipedream.com.

This repository houses essential project assets, including:

The source code for all standardized integration modules
The publicly visible product development plan
The comprehensive Pipedream documentation assets
Additional source materials pertaining to the Pipedream ecosystem.

This README outlines the core capabilities of the platform and guides users through initial setup.

For technical assistance, kindly navigate to https://pipedream.com/support.

Core Capabilities

Workflow Orchestration - Workflows execute automation sequences. They consist of discrete stages—either utilizing pre-built actions or custom implementations in Node.js, Python, Golang, or Bash—initiated by a specific trigger event (e.g., an incoming HTTP call, a scheduled time, a new entry in Google Sheets, etc.).
Event Triggers - Sources are the initiators of workflow execution. They capture events originating from external services such as GitHub, Slack, Airtable, RSS feeds, and numerous others. When an external service generates relevant activity, an event source invokes the corresponding workflow.
Component Actions - Actions represent encapsulated, pre-written code units designed to perform standardized operations across Pipedream's library of over 1,000 integrated APIs. Examples include sending emails or appending data records to a Google Sheet, among many other functions.
Custom Scripting - Complex integration requirements often necessitate tailored logic. Pipedream facilitates the execution of any Node.js, Python, Golang, or Bash code within a workflow step. Users can seamlessly import external packages from language-specific registries, interface with authenticated Pipedream applications, and more. Pipedream embodies "low-code" principles by providing modular components for common tasks while retaining the power of full-spectrum custom coding when needed.
Data Sinks - Destinations manage the reliable, batched, and asynchronous delivery of processed events to external systems such as Amazon S3, Snowflake, or via HTTP endpoints or email.
Cost Structure - A robust free tier is available for individual developers (subject to specified usage quotas).

Demonstration

Select the image below to view a concise demonstration video on YouTube.

Workflow Execution

Workflow executions are sequences of linear operations that commence in response to a preceding event (such as an API invocation or a record creation in Google Sheets). These workflows enable the rapid development of sophisticated automations capable of interfacing with over 1,000 connected applications.

Consult our workflow quickstart guide to begin implementation.

Event Ingestion Sources

Event Sources are listeners that monitor services like GitHub, Slack, Airtable, RSS feeds, and more for novel data occurrences. Upon detecting an event, the source publishes it, which subsequently triggers any associated workflows.

You can also retrieve events emitted by sources programmatically using Pipedream's REST API or via a persistent, private SSE stream.

Should a required pre-built source not exist, you possess the capability to construct your own. The following snippet illustrates the most minimal event source: it exposes an HTTP endpoint ready to receive payloads, logging the request details upon invocation:

javascript export default { name: "http", version: "0.0.1", props: { http: "$.interface.http", }, run(event) { console.log(event); // event contains the method, payload, etc. }, };

The codebase for all standard event sources resides in the components directory. To report defects or propose enhancements, please review our contribution guidelines.

Action Components

Actions are encapsulated functional units executable within a workflow, designed to abstract common data manipulation tasks across Pipedream's more than 500 integrated API connectors. For instance, an action can be used to dispatch electronic mail or append a new record into a Google Sheet, among other operations.

You are able to engineer your own custom actions for internal reuse across workflows. Furthermore, you may opt to publish your actions to the broader Pipedream community, making them available for general consumption.

This action serves as a basic example: it accepts a 'name' input parameter and outputs it to the workflow's execution log:

javascript export default { name: "Action Demo", description: "This is a demo action", key: "action_demo", version: "0.0.1", type: "action", props: { name: { type: "string", label: "Name", }, }, async run() { return hello ${this.name}!; }, };

The implementation code for all standard actions is located in the components directory. For reporting issues or contributing features, refer to our contribution guidelines.

Custom Code Execution

Most non-trivial integrations necessitate custom implementation logic. Code offers the most expressive medium for this, thus Pipedream supports executing custom scripts within a workflow via:

You can incorporate dependencies from the respective language package repositories simply by declaring imports directly within your code; Pipedream handles the parsing and retrieval of necessary libraries.

javascript // Node.js import axios from "axios";

python

Python

import pandas as pd

golang // Go import ( "fmt" pd "github.com/PipedreamHQ/pipedream-go" )

Moreover, custom code steps allow secure authentication and interaction with any connected Pipedream application. For example, to post a message to Slack:

javascript import { WebClient } from "@slack/web-api";

export default defineComponent({ props: { // Defines a connection named "slack" tied to the Slack application credentials. slack: { type: "app", app: "slack", }, }, async run({ steps, $ }) { const web = new WebClient(this.slack.$auth.oauth_access_token);

return await web.chat.postMessage({
  text: "Hello, world!",
  channel: "#general",
});

}, });

Data Delivery Sinks (Destinations)

Destinations, analogous to actions, abstract the complexity associated with managing connections, data batching, and reliable transfer to services like Amazon S3, or endpoint targets such as HTTP servers and email services.

For instance, transmitting data to an Amazon S3 bucket is concisely achieved by invoking $send.s3():

javascript $send.s3({ bucket: "your-bucket-here", prefix: "your-prefix/", payload: event.body, });

Pipedream provides native support for the following data sinks:

Contributors

Our sincere gratitude to every individual who has enriched the Pipedream source code. Your involvement is highly valued!

Costing Model

Pipedream features a highly accommodating free usage tier. Users can execute sources and workflows at no charge, contingent upon the established limitations of this tier. Should usage exceed these allowances, progression to one of our subscription plans is available.

Operational Constraints

The Pipedream execution environment imposes certain runtime boundaries on both event sources and workflows. Detailed documentation regarding these constraints is available in our official documentation.

Issue Reporting & Feature Requests

Prior to submitting a new issue, please conduct a thorough search of existing reported issues or contact our support team to ascertain if a similar topic has already been raised.

If an existing report is found, we request that you either apply a reaction or add a comment elaborating on your specific integration scenario.

If no prior issue covers your concern, kindly utilize the provided issue creation templates when logging a new report.

Security Posture

Information regarding our platform's security practices and data privacy protocols can be reviewed at this location.

If you suspect a security vulnerability or have specific inquiries about product security, please contact our dedicated security team at security@pipedream.com.