Box Importer

Introduction

The box importer takes files processed in migration-center and imports them into the target box site.

Starting with migration-center 23.2, the box Importer uses the box API 4.2.1 to communicate with the box cloud through the respective box web services.

Starting with migration-center 23.3, the box importer can now import and update folder objects.

Limitations and Known Issues

• The character \ is not recognized as a path delimiter

• The folder_path system attribute must start with /

• Sometimes a newly created folder in Box cannot be found by the importer in the first 10 minutes after creation

• Cannot set Created and Modified date on Box Folders (#69471)

• Update imports cannot change the content of an existing document.

Prerequisites

In order to use the Box Importer it is necessary to create a Box Application within the developer console because the adapter uses the JWT authentication.

More about JWT with SDKs at https://developer.box.com/guides/authentication/jwt/with-sdk/

Create a Box Application

In order to create a Box Application that can be used with Migration-Center Box Importer, please follow the steps:

• Log into Box and go to the Developer Console(https://app.box.com/developers/console)

• Select Create New App

• Choose Custom App

• Set the App Name and Purpose to Automation

• Optional: Set the Description and Who is building this application

• Click Next and choose Server Application (with JWT) as Authentication Method in the next Dialog.

• Click Create App

Now the application is created and some additional configurations must be done:

• On the Configuration tab go to App Access Level and choose App Access Only

• On the Configuration tab go to Application Scopes and under Content Actions section select Read all files and folders stored in Box

• On the Configuration tab go to Advance Features and select Make API calls using the as-user header and Generate user access tokens

Create configuration file

The Box Importer needs some information about the Box Application in a JSON file. To generate this file the following steps are necessary:

• On Configuration tab go to Add and Manage Public Keys and click on Generate a Public/Private Keypair. Note: The logged in user is required to have MFA authentication setup on his account

• After security check, a JSON file containing authentication data is downloaded in the browser

• Save this file, it will be used by Migration-Center Box Importer to connect to the Box API

• Ensure the both checkboxes on Advanced Feature are still checked

• Click on Save Changes

Approve Box Application

• On the Authorization tab click Review and Submit

• Click Submit on the new the Review App Authorization Submission tab

• After submitting the Application needs to be Authozired by an admin (or co-admin) account

• Before the application can be used, a Box Admin needs to authorize the application within the Box Admin Console.

  • If not already connected, login to Box with an admin or co-admin account

  • Open Admin Console -> Apps -> Custom Apps Manager

  • The previously created app is pending authorization approval

  
• Click on the three dots and select Authorize App

• Verify Application Scopes and confirm by clicking on Authorize.

Importer Configuration

To create a new box Importer job, specify the respective adapter type in the Importer Properties window – from the list of available connectors “Box” must be selected. Once the adapter type has been selected, the Parameters list will be populated with the parameters specific to the selected adapter type.

The Properties window of an importer can be accessed by double-clicking an importer in the list or by selecting the Properties button or entry from the toolbar or context menu.

Importer parameters

The common adaptor parameters are described in Common Parameters.

The configuration parameters available for the Box importer are described below:

• configFileJson* File containing the Box Application's private key and other details. Those details are obtained when the application is created.

  JSON file containing clientID, clientSecret and appAuth(publicKeyID, privateKey, passphrase)

• userIds* The Box user ids used to establish the API connections. It is possible to use multiple users in order to improve the import performance.

• numberOfThreadsPerUser Maximum number of parallel threads allowed to use simultaneous the same Box API user connection. The allowed values are between 1 and 4. The importer working threads number will be equal to numberOfThreadsPerUser*count(userIds)

• numberOfBoxApiRetries The maximum number of retries the importer will make for any one box API call.

• importLocation Box folder path where documents will be imported. This must start with '/' The value from the folder_path attribute will be concatenated at the end of this path.

• autoCreateFolders Flag indicating if the missing folders will be created automatically based on the values provided in the system rule folder_path.

• checkContentHash Requests the checksum computed by Box when the file is imported to be compared with the checksum computed by mc before import. The checksum must be computed using the SHA-1 algorithm because it is the algorithm used by Box. If the checksum was not computed before, at the scanning phase, the importer will compute the file checksum before importing it. If the two checksums do not match, an appropriate error will be logged, and the affected documents will be moved to the “Import error” state.

• loggingLevel* See Common Parameters.

Working with documents

The box importer can import files to Box. All files imported to box will have their own permissions, so only the box user whose credentials were used for the import can access the imported files on the box site.

Migration Sets

Documents targeted at a box site will have to be added to a migration set. Create a new migration set and set the <source object type>ToBox(document) object type in the Type drop-down. This is set in the -Migration Set Properties- window which appears when creating a new migration set (the type of object can no longer be changed after a migration set has been created).

The migration set is now set to work with box documents.

Transformation Rules

As with other target systems, migration-center objects targeted at box have some predefined system rules available in Transformation Rules. For box, these are “file_name”, “folder_path”, "mc_content_location" and “target_type”. Additional transformation rules for other box document attributes can of course be defined by the user.

If rules for specific attributes are defined, the values need to be associated with the target attributes of a box_document. This is done on the Associations tab of the –Transformation Rules- window by selecting the box_document object type and pointing the target_type rule at it. In this case the value output by the target type rule should be “box_document”, as it must match the object type selected below.

Collaborators

Collaborators can be set by making a transformation rule with the following format <emailaddress>;<role>.

i.e. [email protected];editor or [email protected];viewer.

This transformation rule needs to be associated to the collaborators attribute of the box_document object type.

Comments

Comments can be set by associating a multi-value rule to the comments attribute of the box_document object type.

The user is allowed to set multiple values to comments, the attribute is defined as repeating.

The following format should be used @[user_id:name] (i.e. @[12345:Cristina]) to mention a user in a comment.

Tags

Tags can be set by associating a multi-value rule to the tags attribute of the box_document object type.

The user can set multiple values as tags, the attribute is defined as repeating.

Tasks

Tasks can be set by associating valid rules for the following target attributes:

• task_users: needs to contain the valid email of a user

• task_comment: needs to contain a valid text comment

• task_date: (optional) a valid future date for when the task should finish

Multiple tasks can be set by having repeating values on the 3 attributes. All 3 attributes will need to contain the same number of repeating values for the import to succeed.

Note that if the task_users attribute is set, then task_comment needs to be set as well for the task to be successfully imported.

Template metadata

Template metadata can be set using Box Importer.

The user should define a new object type having the same name as the Template Key on Box.

The template metadata fields are defined as object type attributes having the name as the one on Box and the type mapped to the ones available on Migration-Center.

The dropdown field will be mapped to a String attribute.

The dropdown multivalue field will be mapped to a repeating String attribute.

After the object is defined, it should be added to the target_type system attribute after box_document value and associated in the Association tab similar to box_document.

Custom metadata

Setting custom metadata is possible by adding the metadata name as an attribute to the box_document object type.

The metadata value will be specified using a rule in the migset.

The last step is to associate the attribute from box_document with the rule defined in migset.

Partially Imported Objects

If errors or throttling issues happen during an import some objects might be imported to Box in an incomplete state if the rollback process is also interrupted by the issues.

In this case, the document will be set to Partially Imported in Migration Center, as only an incomplete document was imported to Box.

Reimporting Partially Imported Objects

Partially Imported objects can be reimported if the underlying issue has been resolved, i.e. throttling issues no longer happen and the object itself has valid attributes. In this case no manual action on Box needs to be taken

Partially Imported objects cannot be reimported if the initial error happened because of incorrect configuration (i.e. invalid attributes set from the transformation rules). In this case the object needs to be reset and transformed again before being imported again.

Working with folders

The Box Importer can import folder objects that have their own metadata templates and attributes.

To import folder objects you need to create a <source object type>ToBox(folder) migset.

System Attributes

folder_path - The path to where the folder will be imported. The last folder in the path will be the Folder Object itself.

target_type - Multivalue attribute holding the Object Type and any Metadata Template being used.

Collaborators

You can set Collaborators by making a transformation rule with the following format <emailaddress>;<role>.

i.e. [email protected];Editor or [email protected];Viewer.

You then need to associate the rule to the collaborators attribute of the box_folder object type.

Tags

You can set Tags by associating a multi-value rule to the tags attribute of the box_folder object type.

You can set multiple values as tags, the attribute is defined as repeating.

Migrating Updates (“Update” or “Delta” Migration)

Objects that have changed in the source system since the last scan, are (re-)scanned as update objects. Whether an object in a migration set is an update or not can be seen by checking the value of the Is_update column – if it’s 1, the current object is an update to a previously scanned object (the base object). There are some things to consider when working with the update migration feature:

• An update object cannot be imported unless its base object has been imported previously.

• Objects deleted from the source after having been migrated are not detected and will not be deleted in the target system. This is by design (due to the added overhead, complexity and risk involved in deleting customer data).

• Updates/changes to primary content will be detected and updated accordingly.