# Github Sync

When Github Syncing is enabled on your project, Pipedream will serialize your workflows and synchronize changes to a GitHub repo.

Capabilities include:

Bi-directional GitHub sync (push and pull changes)
Edit in development branches
Track commit and merge history
Link users to commits
Merge from Pipedream or create PRs and merge from GitHub
Edit in Pipedream or use a local editor and synchronize via GitHub (e.g., edit code, find and replace across multiple steps or workflows)
Organize workflows into projects with support for nested folders

# Getting Started

# Create a new project and enable GitHub Sync

Projects are a new concept we are introducing to Pipedream. A project may contain one or more workflows and may be further organized using nested folders. Each project may be synchronized to a single GitHub repo.

Go to https://pipedream.com/projects
Create a new project
Enter a project name and check the box to Configure GitHub Sync
- To use OAuth
  - Select a connected account, GitHub scope and repo name
  - Pipedream will automatically create a new, empty repo in GitHub
- To use Deploy Keys
  - Create a new repo in GitHub
  - Follow the instructions to configure the deploy key
  - Test your setup and create a new project

# Create a branch to edit a project

Branches are required to make changes

All changes to resources in a project must be made in a development branch.

Examples of changes include creating, editing, deleting, enabling, disabling and renaming workflows. This also includes changing workflow settings like concurrency, VPC assignment and auto-retries.

To edit a git-backed project you must create a development branch by clicking Edit > Create Branch

Creating a new git backed development branch in a workflow

Next, name the branch and click Create:

Enter your desired branch name, and click create to create the branch

To exit development mode without merging to production, click Exit Development Mode:

Exiting the branch without committing the changes

Your changes will be saved to the branch, if you choose to revisit them later.

# Merge changes to production

Once you have committed your changes, you can deploy your changes by merging them into the production branch. There are two ways to merge into production, through the Pipedream UI or through merging on Github.

Changes are not singluar

When you deploy a workflow in non-git mode, only the single workflow is affected. However, when you merge a git-backed project to production ALL modified resources in the project will be merged. I.e., multiple workflows may be deployed to, deleted from or modified in production through a single merge action.

# Merge via the Pipedream UI

To merge changes to production, click on Merge to production:

Click on the Merge to production button in the top right of the UI to merge the changes

Pipedream will present a diff between the development branch and the production. Validate your changes and click Merge to production to complete the merge:

In the confirmation modal, click Merge to production to confirm the changes

# Create a Pull Request in Github

To create a pull request in GitHub, either choose Open GitHub pull request from the git-actions menu in Pipedream or in GitHub:

You can also review and merge changes directly from GitHub using the standard pull request process.

Pull request reviews cannot be required

PR reviews cannot be required. That feature is on the roadmap for the Business tier.

# Commit changes

To commit changes without merging to production, select Commit Changes from the Git Actions menu:

Select commit changes from the dropdown menu to make a commit

You can review the diff and enter a commit message:

Preview your changes and approve them with a commit message

# Pull changes and resolve conflicts

If remote changes are detected, you'll be prompted to pull the changes:

Click the Pull to pull in the latest changes to your current branch

Pipedream will attempt to automatically merge changes. If there are conflicts, you will be prompted to manually resolve it:

Example of a commit that requires manual resolution to continue with the commit

# Move existing workflows to projects

Not available for v1 workflows

Legacy (v1) workflows are not supported in projects. Follow this guide to migrate your v1 workflows to v2 workflows.

First, select the workflow(s) you want to move from the workflows listing page (opens new window) and click Move in the top action menu:

Select your workflows you'd like to transfer to a project, then click the Move button in the top right

Then, select the project to move the selected workflows to:

Select which project to move the selected workflows into in the dropdown in the top right of the screen

Undeployed changes are automatically assigned a development branch

If any moved workflows have undeployed changes, those changes will staged in a branch prefixed with undeployed-changes (e.g., undeployed-changes-27361).

# Use the changelog

The changelog tracks all git activity (for projects with GitHub sync enabled). If you encounter an error merging your project, go to the changelog and explore the log details to help you troubleshoot issues in your workflows:

Opening the changelog of commits on the left hand menu

# Local development

Projects that use GitHub sync may be edited outside of Pipedream. You can edit and commit directly via GitHub’s UI or clone the repo locally and use your preferred editor (e.g., VSCode).

To test external edits in Pipedream:

Commit and push local changes to your development branch in GitHub
Open the project in Pipedream’s UI and load your development branch
Use the Git Actions menu to pull changes from GitHub

# Known Issues

Below are a list of known issues that do not currently have solutions, but are in progress:

Project branches on Pipedream cannot be deleted at this time. Even if they are deleted on the Github
If a workflow uses an action that has been deprecated, merging to production will fail
Legacy (V1) workflows are not supported in Projects and there is no option to automatically convert them to the new format (V2)
Self-hosted Github Server instances are not yet supported in-product. Please contact us for help (opens new window).

# Github Enterprise Cloud

If your repository hosted on an Github Enterprise account, you can allow the static Pipedream IP address to sync your project changes.

Follow the directions here (opens new window) to add an IP address.

Then add this static IP address 3.214.142.179 to allow Pipedream to sync changes.

Github Sync is available on Business and above plans

To use this public IP address and connect to Github Enterprise Cloud hosted repositories, you'll need to have a Pipedream Business plan. View our plans (opens new window).

# Frequently Asked Questions

# How are Pipedream workflows synchronized to Github?

Pipedream will serialize your project's workflows and their configuration into a standard YAML format for storage in Github.

Then Pipedream will commit your changes to your connected Github account.

# Do you have a definition of this YAML?

Not yet, please stay tuned!

# Can I sync multiple workflows to a single Github Repository?

Yes, projects are synced to a single Github Repository which allows you to store multiple workflows into a single Github Repository for easier organization and management.

# Can I use this feature to develop workflows locally?

Yes, you can use the Github Syncing feature to develop your workflows from YAML files checked into your Pipedream connected Github Repository.

Then pushing changes to the production branch will trigger a deploy for your Pipedream workflows.

# Why am I seeing an error when trying to merge a branch that includes a connected account in a workflow?

Make sure that all connected accounts in use are accessible to the entire workspace. Private accounts not shared with the workspace will not be able to be merged.

← Projects Workspaces →