Sippy

Sippy is a data migration service that allows you to copy data from other cloud providers to R2 as the data is requested, without paying unnecessary cloud egress fees typically associated with moving large amounts of data.

Migration-specific egress fees are reduced by leveraging requests within the flow of your application where you would already be paying egress fees to simultaneously copy objects to R2.

​​ How it works

When enabled for an R2 bucket, Sippy implements the following migration strategy across Workers, S3 API, and public buckets:

• When an object is requested, it is served from your R2 bucket if it is found.
• If the object is not found in R2, the object will simultaneously be returned from your source storage bucket and copied to R2.
• All other operations, including put and delete, continue to work as usual.

​​ When is Sippy useful?

Using Sippy as part of your migration strategy can be a good choice when:

• You want to start migrating your data, but you want to avoid paying upfront egress fees to facilitate the migration of your data all at once.
• You want to experiment by serving frequently accessed objects from R2 to eliminate egress fees, without investing time in data migration.
• You have frequently changing data and are looking to conduct a migration while avoiding downtime. Sippy can be used to serve requests while Super Slurper can be used to migrate your remaining data.

If you are looking to migrate all of your data from an existing cloud provider to R2 at one time, we recommend using Super Slurper.

​​ Get started with Sippy

Before getting started, you will need:

• An existing R2 bucket. If you don’t already have one, refer to Create buckets.
• API credentials for your source object storage bucket.
• (Wrangler only) Cloudflare R2 Access Key ID and Secret Access Key with read and write permissions. For more information, refer to Authentication.

​​ Enable Sippy via the Dashboard

1. From the Cloudflare dashboard, select R2 from the sidebar.
2. Select the bucket you’d like to migrate objects to.
3. Switch to the Settings tab, then scroll down to the Incremental migration card.
4. Select Enable and enter details for the AWS / GCS bucket you’d like to migrate objects from. The credentials you enter must have permissions to read from this bucket. Cloudflare also recommends scoping your credentials to only allow reads from this bucket.
5. Select Enable.

​​ Enable Sippy via Wrangler

​​ Set up Wrangler

To begin, install npm Open external link . Then install Wrangler, the Developer Platform CLI.

​​ Enable Sippy on your R2 bucket

Log in to Wrangler with the wrangler login command. Then run the r2 bucket sippy enable command:

$ npx wrangler r2 bucket sippy enable <BUCKET_NAME>

This will prompt you to select between supported object storage providers and lead you through setup.

​​ Enable Sippy via API

For information on required parameters and examples of how to enable Sippy, refer to the API documentation Open API docs link . For information about getting started with the Cloudflare API, refer to Make API calls.

​​ Disable Sippy on your R2 bucket

​​ Dashboard

1. From the Cloudflare dashboard, select R2 from the sidebar.
2. Select the bucket you’d like to disable Sippy for.
3. Switch to the Settings tab and scroll down to the Incremental migration card.
4. Press Disable.

​​ Wrangler

To disable Sippy, run the r2 bucket sippy disable command:

$ npx wrangler r2 bucket sippy disable <BUCKET_NAME>

​​ API

For more information on required parameters and examples of how to disable Sippy, refer to the API documentation Open API docs link .

​​ Supported cloud storage providers

Cloudflare currently supports copying data from the following cloud object storage providers to R2:

• Amazon S3
• Google Cloud Storage (GCS)

​​ R2 API interactions

When Sippy is enabled, it changes the behavior of certain actions on your R2 bucket across Workers, S3 API, and public buckets.

Actions not listed above have no change in behavior. For more information, refer to Workers API reference or S3 API compatibility.

​​ Create credentials for storage providers

​​ Amazon S3

To copy objects from Amazon S3, Sippy requires access permissions to your bucket. While you can use any AWS Identity and Access Management (IAM) user credentials with the correct permissions, Cloudflare recommends you create a user with a narrow set of permissions.

To create credentials with the correct permissions:

1. Log in to your AWS IAM account.
2. Create a policy with the following format and replace <BUCKET_NAME> with the bucket you want to grant access to:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:Get*",
                "s3:List*"
            ],
            "Resource": [
                "arn:aws:s3:::<BUCKET_NAME>",
                "arn:aws:s3:::<BUCKET_NAME>/*"
            ]
        }
    ]
}

3. Create a new user and attach the created policy to that user.

You can now use both the Access Key ID and Secret Access Key when enabling Sippy.

​​ Google Cloud Storage

To copy objects from Google Cloud Storage (GCS), Sippy requires access permissions to your bucket. Cloudflare recommends using the Google Cloud predefined Storage Object Viewer role.

To create credentials with the correct permissions:

1. Log in to your Google Cloud console.
2. Go to IAM & Admin > Service Accounts.
3. Create a service account with the predefined Storage Object Viewer role.
4. Go to the Keys tab of the service account you created.
5. Select Add Key > Create a new key and download the JSON key file.

You can now use this JSON key file when enabling Sippy via Wrangler or API.

​​ Caveats

​​ ETags

While R2’s ETag generation is compatible with S3’s during the regular course of operations, ETags are not guaranteed to be equal when an object is migrated using Sippy. Sippy makes autonomous decisions about the operations it uses when migrating objects to optimize for performance and network usage. It may choose to migrate an object in multiple parts, which affects ETag calculation.

For example, a 320 MiB object originally uploaded to S3 using a single PutObject operation might be migrated to R2 via multipart operations. In this case, its ETag on R2 will not be the same as its ETag on S3. Similarly, an object originally uploaded to S3 using multipart operations might also have a different ETag on R2 if the part sizes Sippy chooses for its migration differ from the part sizes this object was originally uploaded with.

Relying on matching ETags before and after the migration is therefore discouraged.