SFTP Integrations

SFTP (Secure File Transfer Protocol) enables you to export data out of Folio or import data into Folio through automated files that are either placed by Folio in an SFTP location or collected by Folio from an SFTP location. This enables Folio to provide data to another software system or receive data from another system as frequently as an overnight process.

Folio provides exported files in a comma separated variable (csv) format and expects files that it imports to be in a comma separated variable (csv) format.

Export: Allows you to extract any data out of Folio using Folio reports, so any data that can be extracted in csv format via reports is available for SFTP
Import: Allows you to import data for the following purposes:

• Create new Folio records and update existing Folio records
• Create new users and update existing users
• Create new entities and update existing entities
• Create new contacts and update existing contacts

Note: SFTP Imports follow the same rules as a normal import, but you are able to set up a field mapper so the columns do not need to be an exact match.

In order to set up an SFTP integration you will need to set up your own SFTP server or contact the Folio support team who can provide you with a quote for an SFTP server hosted by Folio. Once you have the server, you will be able to follow the steps below to set up the integration in Folio.

This article contains the following:

How to setup your SFTP Server details

Creating an Import SFTP

Creating an Export SFTP

How to setup your SFTP Server details

Please note: In order to set up an SFTP Integration to Folio you can set up your own SFTP server or contact the Folio support team who can provide you for a quote for an SFTP server hosted by Folio.

Before you can create an SFTP Integration you will first need to setup the SFTP server, you will need to have administrator rights to do this. You should also check that you have whitelisted Folio’s background servers if you are hosting your own sftp server (please contact support for more information about this)

To get started making a new SFTP server click on the admin gearwheel

Then click on Integration

 

Select either Import or Export depending on what you want to do with the integration (For setting up the server initially either of these options will do)

On the Import Screen click on +Create Import or + Create Export and the Select Integration modal will appear.

Then click on Create a SFTP Server

You will then see the New Integration screen, you will then need to fill in the following fields:

Name	This is the name for the SFTP, we would recommend picking a descriptive name as this is the name that will be shown in the server dropown while creating a new integration
Server	This is the URL of the server
Username	This is the username that Folio will use to access the server
Use Private Key	If the security of your server requires the use of an private key you can tick this, then fill in the private key field with an private key that Folio can use to access the server - Note the private key should be an OpenSSH Private key without a passphrase
Password	If your server setup requires a password to access the server fill in the password here
Private Key	If your server setup requires a private key fill the key in here

Once filled in click Create and you will be prompted to create a new Import/Export (depending on what screen you created the server from)

Creating an Import SFTP

Setting up the Import

On the admin screen click Integrations

 

Then click Imports

Click on + Create Import you will then see the server list dropdown where you can select the server you setup previously (if you haven't done this yet read the instructions above for info about how to do that)

You will then see the New SFTP Import modal, This is how you set up where Folio is to look for the Import file, and also what Folio will import to. 

The fields on the import modal are as follows:

Name	This is the name for the import, we would recommend picking a descriptive name as this is the name that will be shown on the SFTP imports list screen
Prefix	If you are going to have multiple imports to the same folder the prefix can be used to help identify which field Folio should take.  It is an optional field, but if you fill in the prefix then Folio will look for a csv file in the folder that starts with whatever you specify
Import File Location	The folder location on the SFTP server where Folio can find the input csv file
Import To	This is a dropdown which lets you select where Folio will import the data to.  You can choose from: Users, Entities, Contacts or a specific Folio Template
Create Records	This toggle controls if Folio will create a new Folio Record/User/Entity/Contact if there is no matching record in Folio that identifier (The setup of the identifiers is done after you create this, and you can change this option later)
Create Table Rows	This toggle controls if Folio will create a new table row if a row with the matching identifiers is not found
Create Answers	This toggle controls if Folio will create new answers in Single or Multi-select or Hierarchy fields if the a matching answer is not already found
Import Start Date	The date that you want the import to start from
Execute Hour	The hour that the Import will be run at, you can choose any time from 6pm to 5am, but it is important that you pick some time after the file has been created and is in the server
Import As	This is a dropdown list of all super users, you have to select one
Use Import as User for Person Responsible	This option allows you to have Folio automatically set the Person Responsible for any newly created Folio records to the Import as user you selected above - This is useful if you are unable to insert a users name into the import file as it can be assigned automatically
Send Failed Notification To	This is a dropdown list of all users, whenever something goes wrong with the import (for example: the server cannot be connected to, the file has some errors, there is no file) the selected users will receive an email informing them of the error - note that if the error persists they will receive an email every time the import fails
User Meta

Click Save when you are done. Now select the import that you have just created and you can begin mapping fields for the import. At the top you will see details of the import that you just created:

On using Identifiers

If the import you are setting up is going to be updating existing records in Folio then you will need to consider what you are going to use as an Identifier for the import to be able to find the matching record in Folio. 

Folio allows you to use one or more Short Text or Numeric fields to identify the record. For Folios you can also use the Key (which is set up as the default identifier) which will expect the exact Folio key. For other imports there is not a default identifier so you will need to add one if you are going to update records. 

Recommendations about the Identifier:

• If you are importing from another system where that other system is the single source of truth then you should have a field marked as read only (so it cannot be tampered with) which stores the unique identifier for that record in the other system, and then that single field is used as the identifier
• If you do not have a single identifier you can use multiple, but the import will fail if it finds more than one record where the values in the identifier fields match the import file. 
• If you are re-importing into Folio from another system we would recommend using the Folio Key as that is guaranteed to be unique. 

Setting up the Mapping

Next you will set up the Field Mapping. If at this point you are not sure what the columns of your file are going to be then you can come back and complete this later. But once you have locked in the columns you can proceed with the mapping. 

The field mapper shows all of the fields that are on the Folio Template/User/Entity/Contact. The mapper will contain different fields based on the import type, but for the sake of this guide we are going to show a mapper that uses an example Incidents Template. 

Inside the field mapper you will see an empty table with three columns

• Identifier - The field that will be used to identify different folios, must be unique. By default the Key field is used. (See above for an explanation of the identifier)
• Import File - The field(column) name in the import file (csv) that is being imported into Folio.
• <Template Name> - The list of fields in the template that you are importing into.

To start mapping you just need to click on the Not Mapped in the first column of the table, by default the name will be the same as the column (see below) 

But you can change the name to whatever you want, so for example below the Folio field Your Name is mapped to a column with the same name but the Type of Business Incident is mapped to a column called Incident Type

Repeat this until you have setup all of the relevant mapping for the import. 

Note that what you put under Import File must match exactly what that column is called in the csv that you are importing. That column is then mapped to the correct field in the Folio template.

If you have tables in the template, they can also be mapped. Scroll further down the page and you will find each table on the template with their own section:

Click on the table name to expand the section, then you can map the fields in the same way:

Note: Each table needs its own identifier, separate from the overall identifier. This is so that the overall identifier can be used to select the Folio, then the table identifier is used to select the correct row on the table

What to do next

Once you are finished with the setup you can click Activate, this will make the integration active. It will then follow the Import Schedule and run the import on the import start date at the time you have specified. 

You can also manually trigger the import by clicking on the Import button, then Folio will go into the server and try to retrieve the file and import it if it is found immediately. You can do this if you would like to test the integration as well while it is inactive. 

Import Logs

Once you have run at least one import you will see a new tab appear on your import called History. 

Clicking on this will show the history of all of the imports that have been run on this import. 

So in my example below the first import failed because Folio was unable to connect to my SFTP server, the next one failed because there was no file on there, The next one did succeed but there were some rows with errors so i fixed those on the csv and re-did the import and now all the records were imported successfully. 

You are also able to click on the File Name in the first column to download the file 

The table below explains the status columns

Status	Description
Initiated	The Import has been started, but Folio has not connected to the SFTP server yet
Connection Failed	Folio was unable to connect to the SFTP server, hover on the status for a more detailed error message (note: the most common case for this is that you have not whitelisted Folio’s IP address - contact support for more information about this)
Invalid	There is some issue with the CSV that Folio is unable to parse it to be imported
Downloaded	Folio has downloaded a file from the SFTP folder and is preparing to import it
No Files	There was no csv file found in the SFTP folder setup on the import
Imported	Folio imported all rows of the csv successfully. You can click on this to view the import data, however this data is eventually removed and you will only be able to see the keys that were updated
Validated	Folio has imported all successful rows that it can but there are rows that contain errors that Folio was unable to import. You can click on the validated badge to see the import and more information about the rows with errors. You can fix them in the table view and click import again to try and import those rows again or re-do the import entirely
Failed	Something went wrong while importing the file

Creating an Export SFTP

Setting up an Data Integration Report

Folio Integration Exports use Reports to get the data, so to start you will need to set up a custom report with the Fields and Filters you are wanting to Export out of Folio. 

Once you have saved a base report as a custom report scroll down to report settings

At the bottom is the Data Integration Report, ticking this and then saving the report will allow you to pick the report to be used on an SFTP Export. 

Setting up the Export

Once you have a Report set up you can get started with the SFTP Export. 

On the admin screen click Integrations

Then click Exports

Click on + Create Export you will then see the server list dropdown where you can select the server you setup previously (if you haven't done this yet read the instructions above for info about how to do that)

You will then see the New SFTP Export modal, This is how you set up where Folio is going to export the csv file amongst other options. 

The fields on the export modal are as follows:

Name	This is the name for the export, we would recommend picking a descriptive name as this is the name that will be shown on the SFTP imports list screen
Prefix	This is an optional field, but if you write a prefix here Folio will add what you wrote to the beginning of the exported file name.  Usually you are using this if you are doing multiple exports to the same folder and you want to be able to differentiate between the files.
Export File Location	The folder location on the SFTP server where Folio will place the csv file
Export From	This is a dropdown which lets you select which Data Integration Report you want to use for the report (info about how to set these up is above)
Execute Hour	The hour that the Export will be run at, you can choose any time from 6pm to 5am.
Export As	This is a dropdown list of all super users, select who you would like Folio to export the report as. The report will use the users date format for any date fields in the report
Send Failed Notification To	This is a dropdown list of all users, whenever something goes wrong with the import (for example: the server cannot be connected to) the selected users will receive an email informing them of the error - note that if the error persists they will receive an email every time the export fails
Export Start Date	This is the date that Folio will start running the export From

After filling out these details and hitting Save, you will then be taken to the newly created Export where you can Activate it, or click on Export to trigger it manually:

Export Logs

Once you have run at least one export you will see a new tab appear on your import called History. 

Clicking on this will show the history of all of the imports that have been run on this Export. 

You are able to click on the File Name in the first column to download the file

The table below explains the status columns

Status	Description
Initiated	The Export has been started, but Folio has not connected to the SFTP server yet
Connection Failed	Folio was unable to connect to the SFTP server, hover on the status for a more detailed error message (note: the most common case for this is that you have not whitelisted Folio’s IP address - contact support for more information about this)
Invalid	Something went wrong while trying to export the report
Uploaded	The report has been exported and has been successfully uploaded to the SFTP folder.