Stitch Documentation
has moved!

Please update your bookmarks to https://www.stitchdata.com/docs

If you're not automatically redirected after 5 seconds, click here.

Getting Started with the Import API

The Stitch Import API - or IAPI for short - is a REST API that allows you to push arbitrary data into your data warehouse. Once the data enters the Import API, it’ll be processed and sent through the data pipeline like data from any other integration.

The Import API accepts JSON or Transit and returns JSON for all of its methods. Each method uses a standard HTTP verb (GET/POST) and standard HTTP response codes for returning statuses.

Before you get started, consider where the data that you want to push currently lives: is it already in a database or SaaS integration we support? A few of our larger SaaS integrations allow you to sync data at the tabular level; all database integrations can be synced at the columnar level. In addition, you can define replication methods for tables that come from a database integration.

In short, unless you want complete control over data replication, it’s usually easier to use the native integration instead of using the Import API.

Requirements

To use the Import API, you’ll need:

  • A little bit of technical know-how. If you’re comfortable writing and maintaining a small Ruby or PHP script, you’ll be more than qualified.
  • A Stitch account. Free and paid accounts both have access to the Import API.

Generate an API Access Token

The first step in pushing data to the Import API is to generate an access token.

  1. From the Stitch Dashboard page, click the Add an Integration button.
  2. Click the Stitch Import API icon.
  3. Enter a name for the integration. This is the name that will display on dashboard page; it’ll also be used to create the schema in your data warehouse.

    For example, the name “Stitch CRM” would create a schema called stitch_crm in the data warehouse.
  4. Click the Save and Generate Token button.

This token has write access to your Stitch account. Do not distribute it to untrusted third parties.

Locate Your Client ID

Look at your URL while on the Dashboard page to find your client ID. It’s the four-digit number between client and pipeline:

https://app.stitchdata.com/v2/client/XXXX/pipeline/connections

Verify Stitch is Operational

If you’re writing an automated routine for pushing data to the Import API, you may want to have it check the status of the Import API before proceeding:

curl -i https://api.stitchdata.com/v2/import/status

If the response has an HTTP status code other than 200, data should not be pushed.

Choose a Sequence

Every data point pushed to the Import API must have a sequence property. Sequence properties communicate the order in which data points should be considered – newer data points can replace older ones, but not vice versa.

A simple solution is just to use the current timestamp, but before doing so, consider the following:

Are the rows being considered frequently updated?
Rows that are updated every few milliseconds can result in failure if records with identical key values are pushed simultaneously. This means that records with the same key values cannot be sent during the same clock resolution.

For example: if the resolution is measured in milliseconds, records with identical key values cannot be sent during the same millisecond.

Are the records coming from multiple sources?
If records from multiple sources will be sent to the Import API, the time clocks of these sources must be synchronized. This is especially important if different sources are pushing rows to the same table.

Validate Your Requests

During development, use the validate endpoint to verify that the requests being generated are valid:

curl -X POST https://api.stitchdata.com/v2/import/validate \
	-H 'Content-Type: application/json' \
	-H 'Authorization: Bearer < access-token >' \
	-d @filename

Upsert the Data

Now you’re ready to write some data. Note that request bodies have a maximum size of 4MB.

curl -X POST https://api.stitchdata.com/v2/import/push \
	-H 'Content-Type: application/json' \
	-H 'Authorization: Bearer < access-token >' \
	-d @filename

You can push more than one table using the same API access token. Think of it this way: one schema for each API access token. All tables pushed using the same API access token will be housed in the same schema in your data warehouse.

The body of the request must contain the following values:

  • client_id - your client ID
  • table_name - name of the destination table
  • sequence - property that communicates the order in which data points are considered. See the Sequence section for more info on how to define this property.
  • action - “upsert”
  • key_names - array of field names that uniquely identifies the row that this record belongs to. This is the Primary Key, or Keys, for the table being created. 
  • data - contains data to be upserted into the database

   Example Request Body

[
 {
  "client_id": 3,
  "table_name": "users",
  "sequence": 10000,
  "action": "upsert",
  "key_names": [
   "id"
  ],
  "data": {
   "id": 10,
   "email": "john@smith.com",
   "name": "John Smith"
  }
 },
 {
  "client_id": 12,
  "table_name": "dogs",
  "sequence": 300,
  "action": "upsert",
  "key_names": [
   "dog_id"
  ],
  "data": {
   "dog_id": 40,
   "name": "Brody",
   "breed": "corgi"
  }
 },
 {
  "client_id": 3,
  "table_name": "users",
  "sequence": 10001,
  "action": "upsert",
  "key_names": [
   "id"
  ],
  "data": {
   "id": 11,
   "email": "jane@doe.com",
   "name": "Jane Doe"
  }
 }
]

RELATED

Was this article helpful?
0 out of 0 found this helpful

Comments

Questions or suggestions? If something in our documentation is unclear, let us know in the comments!