Stitch Documentation
has moved!

Please update your bookmarks to https://www.stitchdata.com/docs

If you're not automatically redirected after 5 seconds, click here.

Backing Up Data for Migrating to Redshift v2

If you've connected data sources that don't make historical data available, you'll have to make a back up of your existing data to complete the Redshift data warehouse migration process.

Affected Integrations

The following integrations don't have the ability to replay historical data:

  • Import API
  • Mandrill
  • Segment
  • SendGrid
  • Data sources you no longer have access to. For example: you once used Intercom but now use Drift.

Backing Up Your Data

Important!
This process requires you to be comfortable with running command line scripts. If you don't know what this means or how to proceed, please loop in a developer to help you before reaching out to Stitch support.

Incorrectly running the script can create some funky data in the backed up table, which could cause issues if inserted into the new v2 table.

The script in this GitHub repository will copy the integration data from its existing view-based representation into a backup table that's compatible with the v2 version of the data warehouse. For example: if the original table was called segment_page, the backed up table will be called _sdc_backup_segment_page.

When your point-of-contact gives you the go-ahead, run the script once for every schema created by one of the affected integrations. This script should be run as the user you use to connect to Stitch. To confirm the correct user, refer to the Stitch Destination Settings page.

Backfilling Your Data

Only when the following items have been completed will you be able to safely combine the backup table with the new table.

  1. The other steps in the migration process (dropping the _rjm schema, resetting historical syncs, and unpausing the other integrations) have been completed.
  2. New Segment, SendGrid, Mandrill, and/or Import API data has been successfully replicated.

In most cases, the data you backed up into the _sdc_backup_* tables can be combined directly with the new data. However, we urge you to review the data types of each backup/new column pairing for compatibility if you plan on using something like this:

INSERT INTO segment_page (fieldA, fieldB, ...)
SELECT fieldA, fieldB, ... FROM _sdc_backup_segment_page

RELATED

Was this article helpful?
0 out of 0 found this helpful

Comments

Questions or suggestions? If something in our documentation is unclear, let us know in the comments!