OpenText Importer

Introduction

The OpenText Importer takes the objects processed in migration-center and imports them into an OpenText Content Server.

Supported versions

The importer is compatible with the following versions of OpenText Content Server: 10.5, 16.0, 16.4, 20.2, 20.4, 21.4, 22.4 and 23.3.

Known issues and limitations

Importing not allowed items in the physical item container is permitted during delta migration (#50978)
RM classifications for physical objects are not removed during delta migration (#50979)
Physical objects properties of type date are not updated during delta migration (#50980)

Prerequisites

Server configuration

The les-services for v10.5 or the Content Web Services for v10.5+ must be installed on the Content Server for migration-center to connect to it.

The Classification Webservice must be installed on the Content Server to be able to set classifications to documents or folders.

The Record Management Webservice must be installed on the Content Server to be able to set Record Management Classifications.

To be able to import larger .txt files to OTCS, the WebService configuration file (OPENTEXT\webservices\dotnet\cws\Web.Config) needs to contain the following configuration:

<system.web>
 <httpModules>
  <add type="OpenText.Livelink.Service.Core.LogModule" name="LogModule" />
 </httpModules>
 
</system.web>

Content server Patch

Some importer features require installing of some of the provided patches on the Content Server.

The patches are in the migration-center kit in the following folder: ..\ServerComponents\Jobserver\lib\mc-otcs-importer\cspatches

To deploy the patches, copy the provided files to the folder .\patch on the Content Server and restart it.

Content Server Patch pat10000001

This patch extends the OpenText DOCMANSERVICE.Service.DocumentManagement.CreateSimpleFolder method.

The patch allows setting of custom CreateDate, ModifyDate, FileCreateDate and FileModifiyDate for nodes and versions.

Importer Configuration

To create a new OpenText Importer job specify the respective adapter type in the importer’s

properties window from the list of available adapters “OpenText” must be selected. Once the adapter type has been selected, the Parameters list will be populated with the parameters specific to the selected adapter type, in this case OpenText.

The -Properties window- of an importer can be accessed by double-clicking an importer in the list, by selecting the [Properties] button from the toolbar or from the context menu.

Importer parameters

The common adaptor parameters are described in Common Parameters.

The configuration parameters available for the OpenText Content Server Importer are described below:

username* The OpenText Content Server user with “System Administration Rights”
Note:
- “System Administration Rights” are required to internally allow “Impersonating” other users as the individual owners of the objects to be imported.
- Use either built-in user otadmin@otds.admin if Content Server is deployed under OpenText “Runtime and Core Services” or user “Admin” which have already “System Administration Rights”. If you have to use another user, set this privilege within the OpenText Content Server User Administration for that user.
- This user is also internally used for assigning permissions based on the MC System Rule “ACLs”
password* The user’s password.
authenticationMode* The OpenText Content Server authentication mode.
Valid values are:
- CWS for regular Content Server authentication
- RCS for authentication of OpenText Runtime and Core Services
- RCSCAP for authentication via Common Authentication Protocol over Runtime and Core Services
Note: If this version of OpenText Content Server Import Adaptor is used together with together with “Extended ECM for SAP Solutions”, then ‘authenticationmode’ has to be set to “RCS”, since OpenText Content Server together with “Extended ECM for SAP Solutions” is deployed under “Runtime and Core Services”. For details of the individual authentication mechanisms and scenarios provided by OpenText, see appropriate documentation at OpenText KnowledgeCenter.
webserviceURL* The URL of the OpenText Content Web Services. Ex: http://server:port/les-services/services/Authentication http://server:port/cws/Authentication.svc
classificationsWebServiceURL The URL of the Classification WebService. This is necessary when need to set classifications to the imported objects. Ex: http://server:port/les-classifications/services/Classifications http://server:port/Services/Classifications.svc
rmWebserviceURL The URL of the Record Management WebService. This is necessary when need to set records management classifications to the imported objects. Ex: http://server:port/les-recman/services/Classifications
rcsAuthenticationWebserviceURL The URL of the Authentication Service for RCS. It must be set only when RCS or RCSCAP is used. Ex: http://server:port/ot-authws/services/Authentication
physicalObjectsWebserviceUrl The URL of the Physical Objects WebService. This is necessary when need to import physical items. Ex: http://server:port/les-physicalObjects/services/PhysicalObjects
rootFolder* The internal node id of the Content Server root folder where the content of the imported objects will be stored. Note: The individual location for each object to be imported below this “rootfolder” is defined by MC System Rule “ParentFolder”
autoCreateFolders Check this if you want to automatically create missing folders while importing objects to OpenText Content Server.
inheritFolderCategories When enabled, the imported folders will inherit the categories from the parent folder. When the folders are created by autoCreateFolders functionality, the auto created folders will also inherit the categories from the parent folder. When not enabled the categories will not be inherited by the created folders.
inheritDocumentCategories When enabled, the categories from the parent folder will be assigned to the imported documents. The following rules applies:
- The importer will not inherit the categories that have been assigned to the documents in the migration-center.
- The folder categories (other than the ones defined for documents in migration-center) will be inherited by the imported documents.
excludeFolderCategories The list of folder categories (list the names of the categories) that will not be inherited from the parent folder by the documents and folders when “inheritFolderCategories” or “inheritDocumentCategories” parameter is activated. If the parameter is left empty, all folder categories will be inherited from the parent folder.
inheritFolderPermissions When enabled, the folder permissions together with the permissions set in the “ACLS” rule will applied to the documents. If set to false, only the permissions set in the “ACLs” rule will be set to the documents.
classRootFolder The internal node id of the Classification root folder. This is required when setting classifications.
extractMetadataFromEmail When checked, the related metadata of email type object will be extracted from the msg file. If not, the email metadata will be mapped from source attributes.
extractAttachmentsFromEmail When checked, the email attachments will be extracted as XReference into the content server. The attachment will be created as distinct objects into the same place with the email.
crossRefCode The value of parameter must match set in the Content Server Configuration. (Records Management -> Records Management Administration -> System Settings -> RM Settings -> Email Cross-Reference). It becomes mandatory if extractAttachmentsFromEmail is checked.
checkContentIntegrity When checked the content integrity will be checked after the import based on the checksum calculated during scan.
hashAlgorithm Specify the hash algorithm that will be used for checking the content integrity. It is mandatory when “checkContentIntegrity” is checked, so it should be set with the same algorithm that was used at scanning phase. Valid values are: MD2, MD5, SHA-1, SHA-224, SHA-256, SHA-384, SHA-512
hashEncoding Specify the hash encoding that will be used for checking the content integrity. It is mandatory when “checkContentIntegrity” is checked, so it should be set with the same encoding that was used at scanning phase. Valid values are: HEX, Base32, Base64
moveDocumentsToCD When checked, the documents that are children of compound documents are moved the folder where they have been initially imported to the compound document. When not checked, the child document remains in the original location but a shortcut pointing to the compound document is created.
numberOfThreads The number of threads that will be used for importing the documents. Maximum allowed is 20. Note: Due to their hierarchical structure the Folders will be created using a single thread.
loggingLevel* See Common Parameters.

Parameters marked with an asterisk (*) are mandatory.

OpenText Features

The OpenText importer can import different object types. This is specified by migset type i.e.<source type>ToOpenText(document) and the value set in the target_type system rule.

The target_type system rule can be set with multiple repeating values. The order of these values is important and is as follows:

Object Type (i.e. opentext_document, opentext_physical_item etc.)
Optional feature for some types (such as opentext_rm_classification)
Multiple Categories

Below you can find details on each of the types.

Folders

There are two ways to create folders in the OpenText repository with the importer:

On the fly when importing documents When creating a new migration set choose the “<source type>ToOpenText(document)“ type – this will create migration set containing documents targeted at OpenText. Use the “autoCreateFolders” parameter from the OpenText Importer configuration to generate the folder structure based on values extract in the system rule “ParentFolder”. No categories, classification or permissions can be set on the created folders.
Using a dedicated migration set for folders When creating a new migration set choose the “<source type>ToOpenText(container)“ type – this will create a migration set containing folders targeted at OpenText.

Now only the scanner runs containing folder objects will be displayed on the |Filescan Selection| tab. Note that the number of objects contained in the displayed scanner runs now indicates folders and not documents, which is why the number on display (folders) can be different from the total number of objects processed by the scan (if it contains other types of objects besides folders). When creating transformation rules for the migration set, keep in mind that folder-only migration sets have folder-specific attributes to work with, in this case attributes specifically targeted at OpenText folder objects. You can set permissions, categories and classifications to the imported folders.

When importing folder migration set, in case an existing folder structure is already in place an error will be thrown for the folder objects that already exist. It is not possible to avoid this behavior unless you skip them manually by removing them from the migset or putting them in an invalid state for import.

The importer parameter “autoCreateFolders” applies to both documents and folders migration sets.

Projects

OpenText importer supports importing project objects that were scanned from an OpenText source environment. After scanning Project objects these objects will be seen as containers with the Type "Project" in migration center. The functionality of this feature is similar to the importing of Folders using a dedicated migration set. When creating a new migration set choose the “<source type>ToOpenText(folder)“ type – this will create a migration set containing folders or projects targeted at OpenText. When selecting a scan run only scan runs containing folders and projects will be available. The scan run containing the objects of the Type "Project" needs to be selected. Project specific metadata can be set using the migration set transformation rules.

Documents

OpenText importer allows importing documents from any supported source system to OpenText. For that a migset of type “<Source>toOpenTextDocument” has to be created.

Importing documents with multiple versions is supported by OpenText importer. The structure of the versions tree is generated by the scanners of the systems that support this feature and provide means to extract it. Although the version tree is immutable (i.e. the ordering of the objects relative to their antecedents cannot be changed) the type of the versions (linear, minor or major) can be set in the system attribute “VersionControl” (see next sections for more detailed)

All objects from a version structure must be imported since each of them reference its antecedent, going back to the very first version. Therefore, it is advised to not drop the versions of an object between the scan and the import processes as this will generate inconsistencies and errors. If an object is intended to be migrated without versions (i.e. only the current version of the object needs to be migrated) than the affected objects should be scanned without enabling the respective scanner’s versioning option.

Content validation

The Content Validation functionality is based on checksums computed for the document’s contents during scan (by the Scanner itself) and after import (by the Importer). The two checksums are then compared - for any given piece of content its checksums from before and after the migration should be identical.

If the two checksums differ in any way, this is an indication that the content has been corrupted/modified during/after migration. Modifications can happen on purpose or simply by user or environment error. Actual data corruption rarely happens and is usually due to software and/or hardware errors on the systems involved during content transfer or storage.

Validating content is always performed after the content has been imported to the target repository, thus adding another step to the migration process. Accordingly, using this feature may significantly increase import time due to having to read back every piece of content for every document and compute its checksum in order to compare it against the initial checksum computer during scan.

This feature is controlled through the checkContentIntegrity parameter in the OpenText Importer (disabled by default).

This feature work only in tandem with a Scanner that supports it: Documentum Scanner, Filesystem Scanner, Database Scanner and SharePoint Scanner.

Compound documents

This current version of the importer allows importing virtual documents scanned from Documentum as compound document in OpenText. For creating a compound document, the first value of the system attribute “target_type” must be “opentext_compound”. Setting the categories and classifications to the compound documents are supported as they are supported for normal documents. The children of the VD will be imported inside the compound documents in a “box-in-box” manner, similar to the way VDs, children, content, and VDs in VDs are represented in Documentum Webtop.

Importing multiple versions of compound documents is not supported. Therefore, the option for scanning only the last version of the virtual documents from Documentum should be activated. For more details please check the Documentum Scanner user guide.

Importing virtual documents as compound documents is done by OpentText Importer in two steps:

Import all virtual documents as empty CD and all non-virtual documents as normal documents. They are all imported within the folder specified in the "ParentFolder" attribute.
Add the virtual documents children to the compound documents based on the VDRelation relations scanned from Documentum in the following way:

if the child is linked in the folder where it was imported in the step 1 and the importer parameter “moveDocumentsToCD” is checked, then the child document is moved to the compound document. If “moveDocumentsToCD” is not checked then a shortcut to the child is added inside the compound document.
if the child is already linked in a compound document then a shortcut is created in the current compound document (the case when a document is child of a multiple VDs)

If the virtual documents being imported as compound documents have content, the content is imported as the first child of the compound document having the same name as the compound document itself.

Emails

OpenText importer allows importing emails scanned as MSG files from Outlook, Exchange or from other supported systems. In order to be imported as emails, the objects in migration center need to be associated with the target type “opentext_email”.

If the importer parameter “extractAttachmentsFromEmail” is checked the importer will extract the email attachments and import them as cross references for the imported email. In this case the parameter “crossRefCode” should be set with the value of Email Cross-Reference in the RM settings (Records Management -> Records Management Administration -> System Settings -> RM Settings -> Email Cross-Reference) .

For the case when MSG files are scanned from other source system than Outlook or Exchange, the importer allows extracting the email properties (subject, from, to, cc, body, etc) from the email file. This is done during the import.

Physical objects

Starting with version 3.7 the importer allows users to import the scanned documents and folders as Physical Items. All predefined types of physical items (Non Container, Container and Box) are supported. For importing physical objects, the user needs to configure the webservice URL in the “physicalObjectsWebserviceURL”.

Physical Items can be imported with migsets having the appropriate processing type: <SourceType>toOpentext(physicalObj). Physical objects migration sets have a predefined set of rules for system attributes listed under - Rules for system attributes- in the -Transformation rules- window. The system attributes have a special meaning for the importer, so they have to be set with valid values as it is described below.

Attribute Name Multivalue: No Mandatory: No Description: Responsible for assigning an existing area code to the physical item.
Area Multivalue: No Mandatory: No Description: Responsible for assigning an existing area code to the physical item.
Client Multivalue: No Mandatory: No Description: The username to be set as Client to the physical item.
Facility Multivalue: No Mandatory: No Description: Responsible for assigning a valid facility code (that allow users to track boxes that are sent to off-site storage facilities) to the physical item.
HomeLocation Multivalue: No Mandatory: Yes Description: Responsible for assigning a value to attribute 'HomeLocation'. This value can be from location table or a new value specified by user.
Keywords Multivalue: Yes Mandatory: No Description: Responsible for assigning values to target object's 'Keywords' attribute.
Locator Multivalue: No Mandatory: No Description: Responsible for assigning a valid Locator name to the physical item. The Locator should be predefined in Content Server
LocatorType Multivalue: No Mandatory: No Description: Responsible for assigning an existing locator type code from Locator Type Table to the physical item.
Name Multivalue: No Mandatory: Yes Description: Responsible for assigning the Name of the physical item.
OffsiteStorageID Multivalue: No Mandatory: No Description: Responsible for assigning a valid value to 'Offsite Storage ID' attribute.
ParentFolder Multivalue: No Mandatory: Yes Description: Responsible for holding the folder path where the physical item will be imported.
PhysicalBoxPath Multivalue: No Mandatory: No Description: Responsible for holding the path to the Physical Box where the item will be assigned. If it’s not set, the physical object is imported to the folder specified in the “ParentFolder” but it is not assigned to any box.
PhysicalItemType Multivalue: No Mandatory: Yes Description: This attribute specifies the Physical Item Type for physical object. The value should be already defined in Content Server.
ReferenceRate
Multivalue: No
Mandatory: No
Description: Responsible for assigning a valid reference rate code to the target type. The value should be already defined in Reference Rate table.
TemporaryID
Multivalue: No
Mandatory: No
Description: Responsible for assigning a valid value to 'TemporaryID' attribute.
UniqueId
Multivalue: No
Mandatory: No
Description: Responsible for assigning a unique value to 'Unique ID' attribute. This value should be unique in entire Opentext Content Server.

Assigning Physical Items in Box containers

Until version 3.12 of migration-center assigning physical items into physical box items was done by setting the ParentFolder system rule to point to the box item. This would import the physical item in the parent folder of the box item and assign it to the box item.

Starting from version 3.13 a new parameter was added, PhysicalBoxPath, for specifying which box item the object should be assigned to. This allows setting the ParentFolder to any other location, in order to better replicate the OTCS functionality.

Rule for System Attribute - target_type

The first value of the target type should be set with a valid Physical Object Type that is defined in the section Manage/Object types…. The accepted values should start with one of the following value: “opentext_physical_item”, “opentext_physical_box”, “opentext_physical_container”. For distinguish between different Item type you can create new types like: “opentext_physical_item_notebook”, “opentext_physical_container_records”. The provided object types can be extended with the custom physical properties that are defined in the Content Server for every Physical Item Type (Physical Properties).

Starting with the second value of “target_type” rule the user is allowed to optionally set one or multiple categories and records management classifications. See the dedicated chapters for more details regarding these features.

RM classifications

Starting with version 3.7 the Records Management Classifications are not handled anymore as system rules, but they are handled internally as a dedicated and predefined object type: “opentext_rm_classification”.

For assigning the record management classification to the imported objects (Documents, Containers and Physical Items) the “target_type” system rule should have one value (after the first value) equal with: “opentext_rm_classification”. For setting the specific attributes of the classification the association with the predefined type “opentext_rm_classification” and its attributes should be associated.

Only the attributes of “opentext_rm_classification” object type provided with the installation are accepted by the importer.

OpenText documents and folders features handled by rules for system attributes

Both folders and documents migration sets have a predefined set of rules for system attributes listed under -Rules for system attributes- in the -Transformation rules- window. The system attributes have a special meaning for the importer, so they have to be set with valid values as it is described in the next sections.

The following list has all provided system attributes available for documents and folders.

ACLs Migset Type: OpenText(container), OpenText(document) Multivalue: Yes Mandatory: No
Classifications Migset Type: OpenText(container), OpenText(document) Multivalue: Yes Mandatory: No
ContentName Migset Type: OpenText(container) Multivalue: No Mandatory: Yes
Impersonate-User Migset Type: OpenText(container), OpenText(document) Multivalue: No Mandatory: No
mc_content_location Migset Type: OpenText(document) Multivalue: No Mandatory: No
Name Migset Type: OpenText(container), OpenText(document) Multivalue: No Mandatory: Yes
ParentFolder Migset Type: OpenText(container), OpenText(document) Multivalue: No Mandatory: Yes
RenditionTypes Migset Type: OpenText(container) Multivalue: Yes Mandatory: No
Shortcuts Migset Type: OpenText(container), OpenText(document) Multivalue: Yes Mandatory: No
target_type Migset Type: OpenText(container), OpenText(document) Multivalue: Yes Mandatory: Yes
VersionControl Migset Type: OpenText(document) Multivalue: No Mandatory: Yes

Rule for System Attribute – ACLs

The system attribute ACLs (Access Control List) is responsible for optionally assigning permissions based on the source data to the target object.

Each value to be assigned consists of three main elements separated by #. The third value for individual permissions itself is separated by |.

<ACLType#RightName#Permission-1|Permission-2|Permission-n>

Sample ACLs value string: ACL#csuser#See|SeeContents

The following table describes all valid values for defining a correct ACLs value:

Element Possible Values: Owner, OwnerGroup, Public, ACL Description: Owner is responsible for setting owner permissions of the Content Server object OwnerGroup is responsible for setting owner group permissions of the Content Server object Public is responsible for setting public permissions of the Content Server object ACL is responsible for setting specific permissions for a user or group of the Content Server object
RightName Possible Values: Content Server User Login Name, Content Server Group Name, -1 Description: Use a valid Content Server User Login Name or Group Name for ACLType, "Owner", "OwnerGroup" and "ACL". Use -1 for ACLType "Public"
Permissions Possible Values: See, SeeContents, Modify, EditAtts, CreateNode, Checkout, DeleteVersions, Delete, EditPerms Description: Use a valid combination of permissions concatenated by |. Note: You must ensure that that given combination of permissions is valid. This means setting only Delete would not be valid for Content Server since for Delete Permission at least See, SeeContents, Modify and DeleteVersions is required too.

You may use as many individual entries as individual permissions required at OpenText Content Server for this MC System Rule.

Example:

Rule for System Attribute – Classifications

The system attribute Classifications is responsible for optionally assigning one or more Classifications to the target object.

Each value for a Classification to be assigned must be an existing Content Server Classification path, divided with forward slashes (/), located under the node specified in the importer parameter "classRootFolder" (see OpenText Importer parameters).

Rule for System Attribute – ContentName

The system attribute “ContentName” is responsible for assigning the internal target version filename for the source document to be uploaded.

OpenText Content Server uses this internal version filename also for its mimetype recognition, so it is required to always build “ContentName” together with a valid extension.

Rule for System Attribute – ImpersonateUser

The system attribute ImpersonateUser is responsible for assigning the correct owner of the source object to the imported target object.

Notes: If authenticated via RCS add the appropriate domain to the user.

Since the creation of the object to be imported is done with the context of this assigned user, this user needs at least "Add Items" permissions assigned for the target location defined in MC System Rule 'ParentFolder'

Rule for System Attribute – mc_content_location

This attribute can be used to import the content of the document from another place than the location where the scanner exported the document. It should be set with a valid file path. If it’s not set the content will be picked up from the original location.

Rule for System Attribute – Name

The system attribute Name is responsible for assigning a Name to the target document or folder. If a node with the same name already exists, the object will be skipped and the importer will throw an error.

Rule for System Attribute – ParentFolder

The system attribute ParentFolder will be set with the full path of the container where the current object will be imported.

Notes: The adaptor internally uses forward slash (/) as the path delimiter. Make sure to consider this in your rules.

If a folder name in the path contains a forward slash (/) that should be escaped with the character sequence: %2F

Rule for System Attribute – RenditionTypes and RenditionPaths

The system rules RenditionPaths and RenditionTypes can be used for importing renditions that had been exported from the source system.

RenditionTypes is multiple-value rule that will be set with the rendition types that will be imported to the content server (Ex: “pdf”, “png”). If no values are set for this attribute there will be no imported rendition.

RenditionPaths is multiple-value rule that will be used to set the paths where renditions exported from source system are located. If the rendition paths will not be set, the importer will ask the content server to generate the rendition on the file based on the document content.

RenditionTypes and RenditionPaths work in pairs in the following way:

when RenditionTypes has one or more values and the corresponding rendition paths are readable, the renditions are imported
when a RenditionTypes value is missing but a rendition path value is present, renditions are ignored
If a RenditionTypes value is duplicated the first pair of rendition type-rendition path are take in the consideration, the second pair being ignored.

Rule for System Attribute – Shortcuts

The system attribute Shortcuts is responsible for optionally creating shortcuts to the imported document or folder in the folder specified as value of this attribute. One or more folder paths can be specified. If the specified folder does not exist, it will be created automatically if “autoCreateFolders” is enabled in the importer.

If a shortcut cannot be created when importing a document an appropriate error will be reported by the importer and the document will be skipped.

If a shortcut cannot be created when importing a folder, the folder will be created but its status in migration center will be partially imported.

The name of the shortcut is taken from the system rule “Name”.

Rule for System Attribute – target_type

The first value of this attribute must be the type of object to be imported. For documents migsets the value must be “opentext_document”, for folders migsets the value must be “opentext_folder”.

The next values of this attribute are reserved to the categories that will be assigned to the imported document or folder.

Rule for System Attribute – VersionControl

The system attribute VersionControl is used for controlling the creation of versions at Content Server side with the required versioning method (Linear Versioning or Advanced Versioning with Minor or Major Versions).

The valid values are:

“0” - for creating a linear version at Content Server side
“1” - for creating minor version at Content Sever side
“2” - for creating a major version at Content Server side

You cannot mix linear versioning with advanced versioning (minor or major) for the versions belonging to the same version tree.

Delta Migration

Objects that have been changed in the source system since the last scan are scanned as update objects. Whether an object in a migration set is an update or not can be seen by checking the value of the Is_update column – if it’s 1, the current object is an update to a previously scanned object (the base object). An update object cannot be imported unless its base object has been imported previously.

Depending on the source systems the object comes from, the method of obtaining the update information will differ but the objects behavior will stay the same once scanned. See the documentation of the scanners in case you need more information about the supported updates and how they are detected.

In order for delta migration to work properly it is essential to not reset the migration sets (objects) after they have been imported.

When updating the documents or versions, the importer may need to delete some documents or versions that where imported previously. This is because of a limitation of the Content Webservice that does not allow updating the content of existing objects. If Records Manager is installed on the Content Server, importing documents updates may not work since the Content Server does not allow deleting documents and versions.