OpenText Scanner

Introduction

The OpenText Scanner allows extracting objects such as documents, folders compound documents and saves this data to migration-center for further processing. The key features of OpenText Scanner are:

  • Supports OTCS versions: 9.7.1, 10.0, 10.5, 16.0, 16.2, 16.2.4, 20.4

  • Export documents content and their metadata (versions, ACLs, categories, attribute sets, classifications, renditions)

  • Export folders and their metadata (categories, attribute sets, classifications)

  • Export shortcuts of documents and folders

  • Scale the performance by using multiple threads for scanning data

Scanner is the term used in migration-center for an input adapter. Using a scanner such as the OpenText Scanner to extract data that needs processing in migration-center is the first step in a migration project, thus scan also refers to the process used to input data to migration-center.

Scanners and importers work as jobs that can be run at any time and can even be executed repeatedly. For every run a detailed history and log file are created. Multiple scanner and import jobs can be created or run at a time, each being defined by a unique name, a set of configuration parameters and a description (optional).

Known issues & limitations

  • User fields from a category that contain deleted users are scanned as IDs only with a warning in the log (#52982)

Exporting from OpenText

Exporting objects

The OpenText scanner connects to the OpenText Content Server via the specified “webserviceURL” set in the scanner properties and can export folders, documents or compound documents. The account used to connect to the content server must have System Administration Rights to extract the content. All subfolders of the specified folder(s) will automatically be processed as well; an option for excluding select subfolders from scanning is also available. See OpenText Scanner Properties below for more information about the features and configuration parameters available in the OpenText scanner. Email folders are supported so the documents and emails located in email folders are scanned as well.

In addition to the documents themselves, renditions and properties can also be extracted. The content and renditions are exported as files on disk, while the properties are stored to the mc database, as it is the standard with all migration-center scanners.

After a scan has completed, the newly scanned documents and their properties are available for further processing in migration-center.

Below is a list of object types, which are currently supported by the OpenText scanner:

  • Any type of container nodes (we have successfully tested so far the following types: Folder, Email Folder, Project, Binder, EcmWorkspace)

  • Document (ID: 144)

  • Compound Document (ID: 136)

  • Email (ID: 749)

  • Shortcut

  • Generation

  • CAD Document (ID: 736) - scanned as a regular document

Any objects with different types will be ignored during initialization. A list of all node types of scanned containers is provided in the run log as well as all node types in the scope of scanning that were not exported because they are not supported.

Exporting the ACL metadata for folders and documents

The ACLs of the scanned folders and documents will be exported automatically as source attribute ACLs.

ACLs attribute can have multiple values, so that each value has the following format:

<ACLType#RightName#Permission-1|Permission-2|Permission-n>

The following table describes all valid values for defining a correct ACLs value:

Element

Possible Values

Description

ACLType

· Owner

· OwnerGroup

· Public

· ACL

This refers to the default and assigned access.

RightName

· Content Server User Login Name

· Content Server Group Name

· -1

It’s set with the owner name or owner access, with the username or group name for the assigned access and with “-1” for the public access.

Permissions

· See

· SeeContents

· Modify

· EditAtts

· CreateNode

· Checkout

· DeleteVersions

· Delete

· EditPerms

The granted permissions separated by |.

Ex:

ACL#csuser#See|SeeContents Owner#manager1#See|SeeContents|Modify

OpenText Scanner Properties

To create a new OpenText Scanner job, specify the respective adapter type in the Scanner Properties window from the list of available adapters, OpenText must be selected. Once the adapter type has been selected, the Parameters list will be populated with the parameters specific to the selected adapter type, in this case the OpenText parameters.

The Properties window of a scanner can be accessed by double-clicking a scanner in the list or selecting the [Properties] button or entry from the toolbar or context menu.

Common scanner parameters

Configuration parameters

Values

Name*

Enter a unique name for this scanner

Mandatory

Adapter type*

Select the “OpenText” adapter from the list of available adapters

Mandatory

Location*

Select the Job Server location where this job should be run. Job Servers are defined in the Jobserver window. If there are no Job Server defined, migration-center will prompt the user to define a Job Server Location when saving the Importer.

Mandatory

Description

Enter a description for this job (optional)

OpenText scanner parameters

Configuration parameters

Values

username*

The OpenText Content Server user with "System Administration Rights". This privilege is required so the scanner can access all objects in the scanning scope.

Mandatory

password*

The user password

Mandatory

webserviceUrl*

The URL to the .NET Content Web Services.

http://server:port/cws/Authentication.svc" Mandatory

authenticationWebserviceUrl*

The URL to a valid authentication webservice. Currently, CWS and OTDS authentication webservices are accepted.

Mandatory

authenticationType*

The authentication type. Valid values are CWS for standard content server authentication and OTDS for OpenText Directory Services authentication.

Mandatory

classificationsWebserviceUrl

The URL to a valid classification webservice or similar. Use only for Content Server 10.0.0 or later.

Ex: http://server:port/les-services-classifications/Classifications.svc

sessionLength*

The length of the session which is set into Content Server. The length is represented in minutes and has to be bigger or equal than 16 minutes. This is required for refreshing the authentication token.

Mandatory

rootFoldeIds*

The IDs of the nodes whose containers and documents will be scanned. The IDs can be provided either as a list of node IDs separated by a comma or as a CSV file path that contains a node ID value on each row. The CSV path must start with "@". By default, the value is 2000. Can be set with any folder IDs.

Mandatory

scanFolderPaths

The list of folder paths where the scanner looks for objects (documents and folders) relative to the specified root folders. Each of the specified paths must be in at least one of the root folders. The paths must start with "/". Multiple paths can be separated using the “|” character. If empty, the scanner will scan the entire folder structure under each specified root folder.

The following wildcards are allowed in the folder path:

  • * - replace zero, one or multiple characters

  • ? - replace a single character

Examples of using wildcards:

/Shelf/Drugs/Drug No. ?

/Shelf/Drugs/*

/Shelf/Drugs/Drug No. ?/Test

/Shelf/Drugs/*end/Ultimate

excludeFolderPaths

The list of folder paths to be excluded from the scan. Paths must start with "/" and must be relative to at least one ID specified in rootFolderIds.

exportDocuments

Flag indicating if the documents will be exported. When exportDocuments is enabled, the scanner will scan all the documents and their versions linked under the folders specified in rootFolderIds and scanFolderPaths The documents are exported as OTCS(document) objects to MC database.

exportLatestVersions

A number specifying how many versions from every version tree will be exported starting from the latest version to the older versions. If it is empty, 0 or negative, all versions will be exported.

exportCompoundDocuments

Flag indicating if the compound documents will be exported. When enabled the scanner will scan the latest version of the compound document together with its children. There will be no migration-center relations between the compound document and its children. The children are related to the compound document through the parent folder attribute.

The parameter exportCompoundDocuments can be enabled only when exportDocuments is enabled.

exportClassifications

Flag indicating if the classifications will be exported. When exportClassifications is enabled, the scanner will export the classifications of the scanned folders and documents in the source attribute "Classifications". Each classification will be a distinct value of the attribute "Classifications". The classification values will be saved as paths. The export of classifications will be available only for CS 10, 10.5 and 16 since CS 9.7.1 does not provide this functionality.

Ex: Corporate Information/News/Newsletter

exportShortcuts

Flag indicating if the shortcuts of scanned documents and folders will be exported. The full path of every shortcut pointing to the current object (document or folder) is scanned in the source attribute “Shortcuts”.

exportShortcutsAsObjects

Flag indicating if the shortcuts of scanned documents and folders will be exported as independent objects. The shortcuts and generations are exported as OTCS(shortcut) objects to MC database.

exportURLs

The flag indicating if the URLs will be exported as OTCS(object) to MC database. If the flag is not set, the URLs will not be exported.

exportRenditions

Flag indicating if the renditions will be exported.

exportFolderStructure

Flag indicating if the structure of folders will be exported. When enabled the scanner will scan entire folder structure under the folders configured by parameters rootFolderIds and scanFolderPaths. The scanner will export all folders as OTCS(container) objects to MC database.

exportLocation*

The location where the exported object content should be saved. It can be a job server local folder or a shared folder. It must exist and it should be writable.

Mandatory

computeChecksum

When it's checked the checksum of scanned files will be computed. Useful for determining whether files with different names and from different locations have in fact the same content, as can frequently happen with common documents copied and stored by several users in a file share environment.

Do not enable this option unless necessary, since the performance impact is significant due to the scanner having to read the full content for each and compute the checksum for it.

hashAlgorithm

Specifies the algorithm that will be used to compute the Checksum of the scanned objects.

Possible values are "MD2", "MD5", "SHA-1", "SHA-224", "SHA-256", "SHA-384" and "SHA-512". Default value is MD5.

hashEncoding

Specifies the encoding that will be used to compute the Checksum of the scanned objects.

Possible values are "HEX", "Base32" and "Base64". Default value is HEX.

numberOfThreads

The number of threads that will be used for scanning the documents. Maximum allowed is 20.

loggingLevel*

The scanner logging level to use: 1-Error, 2-Warn, 3-Info, 4-Debug.

Mandatory

Known issues

When extracting the Owner, Version Created By, Created By and any other attribute the references a user it may happen the internal id of the user cannot be resolved by the Content Server and therefore instead of the username the scanner will extract its id. When this happens, a warning is logged in the report log.

Email folder can only be extracted from the Content Server 16 and later.

Wildcards feature is working properly in most cases, but it has some limitations:

  • Folders which contain the character “?” in the name will not be scanned using “*” wildcard. Example: Ex: "/*" will not scan "/test?name"

  • The wildcards in the middle of the path do not work: “/*/Level2” will scan no documents under the “Level2” folder

  • “/Level1/*exactname*” will scan no documents which are located in “exactname” folder

When scanning documents by using “scanFolderPaths”, if two or more paths overlap, the documents which are located under them will be scanned twice or more. However, this issue will not affect the content integrity of the documents. Each duplicate document will have a separate content location.

Example:

/folder1/folder2/doc1

/folder1/folder3/doc2

/folder1/folder3/doc3

scanFolderPaths: “/folder1 | /folder1/folder3”

Considering the above scenario, the documents “doc2” & “doc3” will be scanned twice.

When at least two ids overlap with their children when setting rootFolderIds the documents will be scanned twice or more.

Example:

rootFolderIds* : 2000, 2001

If the root folder with id 2000 contains as children root folder 2001, all the objects under the root folder 2001 will be scanned twice.

Last updated