D2 Importer
Introduction
The D2 Importer currently supports the following D2 versions: 4.7, 16.5, 16.6, 20.2, 20.4, 21.4, 22.4. For using the importer with a D2 version older than 20.4, some additional configurations are required (see chapter D2 Configuration for older versions).
Since D2 itself is still based on the Documentum platform, D2’s requirements regarding Documentum product also apply to the migration-center D2 Importer – namely Documentum Content Server 6.6 and higher as well as DFC version 6.6 or higher. When migrating between different Documentum and Documentum-based systems, it is recommended to use the DFC version matching the Content Server being accessed. Should different versions of Documentum Content Server be involved in the same migration project at the same time, it is recommended to use the DFC version matching the latest Content Server in use.
Known Issues and Limitations
The validation against a property page is skipped in the following cases and therefore invalid attribute values are imported successfully:
The property page contains a label field that has a value assistance. Workaround: Remove the value assistance from every label fields in the property page.
The property page contains a combo field that is not linked to a property and "Auto select single value list" is checked. Workaround: Uncheck "Auto select single value list" for combo files that are not linked to a property.
The D2 Importer does not validate values for Editable ComboBox if they are not in the dictionary due to D2 API limitation (#49115)
Prerequisites
DFC configuration
Starting from version 3.9 of migration-center additional configurations need to be made for the Documentum connector to be able to locate Documentum Foundation Classes. This is done by modifying the dfc.conf file, located in the Job Server installation folder.
There are two settings inside the file that by default match the paths of a standard DFC install. One needs to have the path for the config folder of DFC and the other needs the path to the dctm.jar.
Example:
The dfcConfig parameter must point to the configuration folder.
The dfcDctmJar parameter must point to the dctm.jar file!
DFC compatibility issues
If the DFC version used by the migration-center Jobserver is not compatible with the Java version or the Content Server version it is connecting to, errors might be encountered when running a Documentum connector.
When encountering the following error, the first thing to check is the DFC - Java - DCTM compatibility matrixes.
Older D2 versions
For using the D2 Importer with a D2 Content Server older than 20.4 some additional steps must be performed:
Ensure the Job Server is stopped
Go to the ...\lib\mc-d2-importer folder
Remove all jar files (either by moving them outside the Job Server folder, or by deleting them)
If your D2 version is 4.7 unzip all files from ...\D2-4.7 \d2-4.7-importer into the folder...\lib\mc-d2-importer folder
If your D2 version is greater then 4.7 but less or equal with 20.2 unzip all files from ...\D2-20.2 \d2-20.2-importer into the folder...\lib\mc-d2-importer folder
Start the Job Server service again
Lockbox configuration
If your D2 environment has a lockbox configured additional steps need to be performed for the D2 Importer to work properly. The D2 lockbox files must be configured on the machine where the Job Server will perform the import job.
Before running the D2 installer please make sure that Microsoft Visual C++ 2010 Service Pack 1 Redistributable Package MFC Security Update - 32 bit is installed.
Run the D2 installer according to the D2 Installation Guide, using the same java version as on the D2 environment:
select Configure Lockbox
select Lockbox for – Other Application Server
set the install location to …\lib\mc-d2-importer\Lockbox of the Job Server folder location.
set the correct password and passphrase as it was configured on the D2 Server
restart the Job Server
Note that if a different location is selected for the Lockbox installation the wrapper.conf file must be change to reflect the new location:
wrapper.java.classpath.14=./lib/mc-d2-importer/LockBox/lib/*.jar
wrapper.java.classpath.15=./lib/mc-d2-importer/LockBox
wrapper.java.additional.3=-Dclb.library.path=./lib/mc-d2-importer/LockBox/lib/native/win_vc100_ia32
Importer Configuration
To create a new D2 Importer job specify the respective adapter type in the New Importer Properties window – from the list of available connectors “D2” must be selected. Once the adapter type has been selected, the Parameters list will be populated with the parameters specific to the selected adapter type, in this case the D2 connector’s parameters.
The Properties window of an importer can be accessed by double-clicking an importer in the list, or selecting the Properties button or entry from the toolbar or context menu.
Importer parameters
The common adaptor parameters are described in Common Parameters.
The configuration parameters available for the D2 importer are described below:
username* User name for connecting to the target repository. A user account with superuser privileges must be used to support the full Documentum functionality offered by migration-center.
password* The user’s password.
repository* Name of the target repository. The target repository must be accessible from the machine where the selected Job Server is running.
importObjects Selected by default; if NOT checked, documents and folders will NOT be imported. The reason for NOT importing folders and documents is to allow importing only relations between already imported folders/documents.
importRelations Determines whether to import relations between objects. In this case a relation means both dm_relations and dmr_containment objects. Hint: Depending on project requirements and possibilities, it can make sense to import folders and documents first, and add relations between these objects in a second migration step. For such a two-step approach the importer can be configured accordingly using the importObjects and importRelations parameters. It is possible to select both options at the same time as well and import everything in a single step is the migration data is organized suitably. It doesn’t make sense, however, to deselect both options.
importAuditTrails Determines whether Documentum(audit trail) migsets are imported or not. If the setting is false, any Documentum(audit trail) migsets added to the Importer will be ignored (but can be imported later, after enabling this option)
importLocation The path inside the target repository where objects are to be imported. It must be a valid Documentum path. This path will be appended in front of each dctm_obj_link (for documents) and r_folder_path (for folders) before linking objects if both the importLocation parameter and the dctm_obj_link/r_folder_path attribute values are set. If the attributes mentioned previously already contain a full path, the importLocation parameter does not need to be filled.
autoCreateFolders Flag indicating if the folder paths defined in "dctm_obj_link" attribute should be automatically created using the default folder type.
defaultFolderType The Documentum folder type name used when automatically creating the missing object links. If left empty, dm_folder will be used as default type.
applyD2Autonaming Enable or disable D2’s autonaming rules on import. See Autonaming for more details.
applyD2Autolinking Enable or disable D2’s autolinking rules on import. See Autolinking for more details.
applyD2Security Enable or disable D2’s Security rules on import. See Security for more details.
applyD2RulesByOwner Apply D2 rules based on the current document’s owner, rather than the user configured to run the import process. See Applying D2 rules based on a document’s owner for more details.
throwD2LifecycleErrors Specifies the behaviour when an error occurs during the application of D2 lifecycle actions:
checked - the lifecycle error is reported and the affected object is moved to the error state
unchecked - the error is reported as warning and the object is set as partially imported so next time when it will be imported only the lifecycle actions will be performed.
applyD2Lifecycle Apply the appropriate D2 Lifecycle, or the one specified in the d2_lifecycle_name system rule. See D2 Lifecycle for more details.
reapplyAttrsAfterD2Lifecycle Comma separated list of attributes existing in the transformation rules to be reapplied after attaching lifecycle. This is useful when some attributes are overwritten by the lifecycle state actions.
numberOfThreads The number of threads on which the import should be done. Each document will be imported as a separate session on its own thread. Max. no. is 20
checkContentIntegrity If enabled will check compare the checksums of imported objects with the checksum computed during scan time. Objects with a different checksum will be treated as errors. May significantly increase import time due to having to read back every document and compute its checksum after import.
ignoreRenditionErrors Determines whether errors affecting individual renditions will also trigger an error for the entire object.
checked - renditions with errors will be reported as warnings; the objects and other renditions will be imported normally
unchecked - renditions with errors will cause the entire object, including other renditions to fail on import. The object will be transitioned to the "Import error" status and will not be imported.
loggingLevel*
See Common Parameters.
Parameters marked with an asterisk (*) are mandatory.
The autoCreateFolders parameter familiar from the Documentum Importer has been dropped from the D2 Importer due to D2‘s Autolinking feature, which is the preferred way for creating folders in D2.
Working with features specific to D2
Many of the new features available through D2 have been implemented in the migration-center D2 connector in addition to the basic functionalities of importing documents and setting attributes. Features such as Autolinking, Autonaming and Security are all available. In addition, more features such as validating attribute values obtained from transformation rules using D2 dictionaries and taxonomies, using D2 templates for setting predefined default values, or applying D2 rules based on a document’s owner rather than the user the import process is configured to run with are provided.
Autonaming
The D2 autonaming feature is fully supported by the D2 Importer. The feature can be toggled on or off through the applyD2Autonaming parameter present in a D2 Importer’s properties.
If the applyD2Autonaming parameter is checked, D2’s autonaming rules will take effect as documents are imported; this also means any value set in Transformation Rules for the object_name attribute will be ignored and overridden by D2’s autonaming rules.
If the applyD2Autonaming parameter is unchecked (which is the default state the parameter is set to), D2’s autonaming rules will not be used. Instead the value set for object_name will be set for the imported documents, as is the case currently when using the standard Documentum Importer
Autolinking
The D2 autolinking feature is fully supported by the D2 Importer. The feature can be toggled on or off through the applyD2Autolinking parameter present in a D2 Importer’s properties.
If the applyD2Autolinking parameter is checked, D2’s autolinking rules will take effect as documents are imported, with the documents being arranged in the folder structure imposed by the rules applicable to the imported document(s).
If the applyD2Autolinking parameter is unchecked (which is the default state the parameter is set to), D2’s autolinking rules will not be used. Instead the dctm_obj_link system rule will be used as the target path for the imported documents, as is the case currently when using the standard Documentum Importer
Even when using D2 autolinking in the importer, a valid path must be provided as a value for the dctm_obj_link system rule. This is because of the way migration-center works – documents will be linked to the path specified by the dctm_obj_link first and re-linked to the proper (autolink) paths later if the applyD2autolinking parameter is enabled.
Migrating folder objects using the D2 Importer is not supported currently. Currently the only possible way of creating folders in D2 is by using the Autolinking feature or by checking the AutoCreateFolders parameter in the D2 Importer properties.
Security
The D2 Security feature is fully supported by the D2 Importer. The feature can be toggled on or off through the applyD2Security parameter present in a D2 Importer’s properties.
If the applyD2Security parameter is checked, D2’s Security rules will take effect as documents are imported; this also means any value set in Transformation Rules for the acl_name attribute will be ignored and overridden by D2’s Security rules which will set the appropriate ACLs.
If the applyD2Security parameter is unchecked (which is the default state the parameter is set to), D2’s Security rules will not be used. Instead the value set for acl_name will be used to set the ACL for the imported documents, as is the case currently when using the standard Documentum Importer. If neither the acl_name rule nor the applyD2Security parameter has been set, then the documents will fall back to the Content Server’s configuration for setting a default ACL; depending on the Content Server configuration this may or may not be appropriate, therefore please setting permissions explicitly by using either the acl_name rule or enabling the applyD2Security parameter.
D2 Lifecycle
The D2 Importer can set D2 Lifecycle states and run the state actions on documents. Lifecycles can be applied on documents by checking the applyD2Lifecycle property in the D2 Importer configuration. The lifecycle to be used can either be specified in the d2_lifecycle_name system rule, or not specified at all. Specifying it should increase performance as the importer does not need to search for it.
If no state was specified in the a_status system rule, the importer will apply the initial state as defined in D2-Config. The importer is able to set states that are only on the first 2 or 3 levels in the lifecycle. The third level is available only if the first state automatically transitions to the second one (see (init) -> Draft states). Note that state entry conditions are not checked upon setting the state.
When applying a lifecycle some of the attributes set in migration-center may get overwritten by the state actions. In order to work around this issue, the reapplyAttrsAfterD2Lifecycle property can be set. Here you can specify which attribute values to be reapplied after setting the lifecycle state.
The throwD2LifecycleErrors parameter can be used to specify whether the objects should be set to Error or Partially Imported when an error occurs during the application of lifecycle actions.
D2 default templates
The D2 Importer can use D2 templates to set default values for attributes. The template to be used can be specified through the d2_default_template system rule. If a valid template has been set through the, all attributes configured with a default value in the respective template will get the default value set during import.
It is possible to override the default value even if the d2_default_template system rule has been set to a valid D2 template by creating a transformation rule for the respective attributes. Thus if both a transformation rule and a default value via D2 template apply to a given attribute, the value resulting from the transformation rule will override the default value resulting from the template.
Validating attribute values based on D2 dictionaries
Certain attributes associated with D2 dictionaries or taxonomies can be validated to make sure the value set is among the permissible values defined in the associated D2 dictionary/taxonomy.
D2 creates such an association through a property page. Migration-center can be configured to read a D2 property page, identify attributes associated with dictionaries and taxonomies and validate the values resulting from the transformation rules of the respective attributes against the values defined in the associated dictionary/taxonomy.
One property page can be defined per migration set through the d2_property system rule; the resulting value must be the name of a valid D2 property page.
Failure to validate an attribute value in accordance with its associated dictionary/taxonomy will cause the entire document to fail during import and transition to the Import error status.
Applying D2 rules based on a document’s owner
The D2 importer allows D2 rules to be applied to objects during import based on the object’s owner, rather than the user running the import process. This makes sense, as the import process would typically be running using the same user all the time, while real-life usage scenarios with D2 would involve different users connecting using their own accounts, based on which D2 rules would then apply. Migration-center can simulate this behavior by passing on the document owner as the user based on which rules should be applied instead of the currently connected user the import process is running with.
The feature can be enabled through the ApplyD2RulesbyOwner parameter in the D2 Importer. In order for the feature to work, a rule for the owner_name attribute must also be defined and reference a valid user. Should the parameter be enabled without an owner having been set, it will have no effect.
This feature will not work if the D2 variable $USER is used to reference users in D2 rules. D2 will always resolve this variable to the currently connected user’s name, i.e. the user running the import process, and migration-center cannot override this behavior.
Working with features specific to Documentum
In addition to the dedicated D2 features presented in the previous chapter, the D2 Importer also supports basic Documentum features used by D2, such as versions, renditions, virtual documents, audit trails, and so on. These features generally work the same as they do in the migration-center Documentum Importer and should be instantly familiar to users of the Documentum connector.
Versions
Versions (including branches) are supported by the D2 Importer, including custom version labels. Version information is generated during scan, be it a Documentum Scanner or other scanners supporting systems where version information can be extracted from. This version information (essentially the Documentum r_version_label) can be changed using the transformation rules prior to import. The version structure (i.e. the ordering of the objects relative to their antecedents) cannot be changed using migration-center.
If objects have been scanned with with version information, all versions must be imported as well since each object references its antecedent, going back to the very first version. Therefore it is advised not to drop the versions of an object between the scan and the import processes since this will most likely generate inconsistencies and errors. If an object is intended to be migrated without versions (i.e. only the current version of the object needs to be migrated), than the affected objects should be scanned without enabling the respective scanner’s versioning option.
Renditions
Renditions are supported by the D2 Importer. Rendition information is typically generated during scan, be it by the Documentum Scanner or other scanners supporting systems where rendition information or other similar features can be extracted. Should rendition information not be obtainable using a particular scanner, or if the respective source system doesn’t have renditions at all, it is still possible to add files as renditions to objects during the transformation process. The renditions of an object can be controlled through the migration-center system attribute called dctm_obj_rendition. The dctm_obj_rendition attribute appears in the “System attributes” area of the Transformation Rules window. If the object contains at least one rendition in addition to the main content, the source attribute dctm_obj_rendition will be available for use in transformation rules. To keep the renditions for an object and migrate them as they are, the system attribute dctm_obj_rendition must be set to contain one single transformation function: GetValue(dctm_obj_rendition[all]). This will reference the path of the files where the corresponding renditions have been exported to; the Importer will pick up the content from this location and add them as renditions to the respective documents.
It is possible to add/remove individual renditions to/from objects by using the provided transformation functions. This can prove useful if renditions generated by third party software need to be appended during migration. These renditions can be saved to files in any location which is accessible to the Job Server where the import will be run from. The paths to these files can be specified as values for the dctm_obj_rendition attribute. A good practice for being able to assign such third party renditions to the respective objects is to name the files with the objects’ id and use the format name as the extension. This way the dctm_obj_rendition attributes’ values can be built easily to match external rendition files to respective D2 documents.
Other properties of renditions are also available to be set by the user through a series of rendition-related system attribute which are available automatically in any migset targeting a Documentum, D2 or FirstDoc system:
dctm_obj_rendition Specify a file on disk (full path) to be used as the content of that particular rendition
dctm_obj_rendition_format Specify a valid Documentum format to be set for the rendition Leave empty to let Documentum decide the format automatically based on the extension of the file specified in dctm_obj_rendition (if the extension is not known to Documentum the rendition’s format will be unknown) Will be ignored if dctm_obj_rendition is not set
dctm_obj_rendition_modifier Specify a page modifier to be set for the rendition. Any string can be set (must conform to Documentum’s page_modifier attribute, as that’s where the value would end up) Leave empty if you don’t want to set any page modifiers for renditions. Will be ignored if dctm_obj_rendition is not set
dctm_obj_rendition_page The page number where the rendition is linked. A repeating attribute to allow multiple values. Used for multiple page content. Leave empty if you don’t want to set any page for renditions. Will be ignored if dctm_obj_rendition is not set
dctm_obj_rendition_storage Specify a valid Documentum filestore where the rendition’s content file will be stored. Leave empty to store the rendition file in Documentum’s default filestore. Will be ignored if dctm_obj_rendition is not set
All dctm_obj_rendition* system attributes are repeating attributes and accept multiple values, allowing multiple renditions to be added to the same object. Normally the number of values for all dctm_obj_rendition* attributes should be the same and equal the maximum number of renditions one would like to set for an object. E.g. if three renditions should be set, then each of the dctm_obj_rendition* attributes should have three values for each of the three renditions. More values will be ignored, missing values will be filled in with whatever Documentum would use as default in place of that missing value.
Relations
Relations are supported by D2 Importer. Relations for import to D2 can currently be generated only by the Documentum Scanner, but technically it is possible to customize any scanner to generate relation information that can be processed by the D2 Importer if information similar or close to Documentum relations needs to be extracted from non-Documentum system. The option named exportRelations in the Documentum scanner’s configuration determines if relations are scanned and exported to migration-center.
Relations cannot be altered using transformation rules; migration-center will manage relations automatically if the appropriate options in the scanner and importer have been selected. Relations will always be connected to the parent object of the relation and can be viewed in migration-center by right-clicking on an object in any view of a migration set and selecting <View Relations> from the context menu. All relations with the selected object as the parent object are listed with their associated metadata, such as relation name, child object, etc.
An option corresponding to the scanners’ must be selected in the importer as well to restore the relations between objects on import; the option is labeled importRelations. The importer can be configured to import objects and relations together or independently of one another. This can be used to migrate only the objects first and attach relations to the imported objects later.
Relations will always be connected to the parent object of the relation, which is why importing a relation will always be attempted when importing its parent object and the importRelations option mentioned above is selected as well. Importing a relation will fail if its child object is not present at the import location. This is not to be considered a fatal error. Since the relation is connected to the parent object, the parent object itself will be imported successfully and marked as “Partially Imported”, indicating it has one or more relations which could not be imported (because the respective child object could not be found). After the child object gets imported, the import for the parent object can be repeated. The already imported parent object will not be touched, but its missing relation(s) will now be created and connected to the child object. Once all relations have been successfully created, the parent object’s status will change from “Partially imported” to “Imported”, indicating a fully migrated object, including all its relations. Should some objects remain in a “Partially imported” state because the child objects the relation depends on are not migrated for a reason, then the objects can remain in this state and this state can be considered a final state equivalent to “imported” in such a case. The “Partially imported” state does not have any adverse effects on the current or future migrations even if these depend on the respective objects.
migration-center’s Documentum connectors support relations between folders and/or documents only (i.e. dm_folder and dm_document objects, as well as their respective subtypes) dm_subscription type objects for example, although relations from a technical point of view, will be ignored by the scanner because they are relations involving a dm_user object.
Custom relation objects (i.e. relation-type objects which are subtypes of dm_relation) are also supported, including any custom attributes they may have. The restriction mentioned above regarding the types of objects connected by such a relation also apply to custom relation objects.
Virtual Documents
Documentum Virtual Documents are supported by the D2 Importer. The option named exportVirtualDocs in the configuration of the scanner determines if virtual documents are scanned and exported to migration-center.
Another option related to virtual documents, named maintainVirtualDocsIntegrity is recommended when scanning VDs. This option will allow the scanner to include children of VDs which may be outside the scope of the scanner (paths to scan or dqlString) in order to maintain the integrity of the VD. If this option is not turned on, the children in the VD that are out of scope (they are not linked under the scanned path or they are not returned by dqlString) will not be scanned and the VD may be incomplete. This option can be enabled/disabled based on whichever setting makes sense for the migration project.
The VD binding information (dmr_containment objects) are always scanned and attached to the root object of a VD regardless of the maintainVirtualDocsIntegrity option. This way it is possible to scan any missing child objects later on and still be able to restore the correct VD structure based on the information stored with the root object.
The ExportVersions option needs to be checked for scanning Virtual Documents (i.e. if the ExportVirtualDocuments option is checked) even if the virtual documents themselves do not have multiple versions, otherwise the virtual documents export might produce unexpected results. This is because the VD parents may still reference child objects which are not current versions of those respective objects. This is not an actual product limitation, but rather an issue caused by this particular combination of Scanner options and Documentum’s VD features, which rely on information related to versioning.
The Snapshot feature of virtual documents is not supported by migration-center.
Audit Trails
The D2 Importer also supports migrating audit trails of documents and folders. Audit Trails can be scanned using the Documentum Scanner (see the Documentum Scanner user guide for more information about scanning audit trails), added to a DctmToD2(audittrail) migset and imported using the D2 Importer.
DctmtoD2(audittrail) migsets are subject to the exact same migration procedure as Documentum documents and folders. DctmtoD2(audittrail) migsets can be imported together with the document and folder migsets they are related to, or on their own at any time after the required document and folder objects have been migrated. It is of course not possible to import any audit trails if the actual object the audit trails belong to hasn’t been migrated.
Importing audit trails is controlled in the D2 Importer via the importAuditTrails parameter (disabled by default).
A typical workflow for migrating audit trails consists of the following main steps:
Scan folders/documents with Documentum Scanner by enabling the “exportAuditTrail” parameter. The scanning process will create tree kind of distinct objects in migration-center: Documentum(folder), Documentum(document) and Documentum(audittrail).
Assign audit trail objects to (a) DctmToD2(audittrail) migset(s) and follow the regular migration-center workflow to promote the objects through transformation rules to a “Validated” state required for the objects to be imported.
Import audit trails objects using D2 Importer by assigning the respective migset(s) to a D2 Importer job with the importAuditTrails parameter enabled (checked)
Preparing a Migration Set Containing Audit Trails
In order to prepare audit trail for import, first create a migset containing audit trail objects (more than one migset containing audit trails can be created, just like for documents or folders). For a migset to work with audit trails, the type of object must be set “DctmToD2(audittrail)” accordingly. After setting the migset type to “DctmToD2(audittrail)” the | Filescan | tab will display only scans which contain Documentum audit trail objects. Any of these can be added/removed to/from the migration set as usual.
Transformation Rules for a Migration Set Containing Audit Trails
Transformation rules allow setting values for the attributes audit trail entries on import to the target system. Values can be simply carried over unchanged from the source attributes, or they can be transformed using the available transformation functions. All attributes that can be set and associated are defined in the target type “dm_audittrail”.
As with other migration-center objects, audit trail objects have some predefined system attributes:
audited_object_id this should be filled with the corresponding value that comes from source system. No transformation or mapping is necessary because the importer will translate that id into the corresponding id in target repository.
r_object_type must be set to a valid Documentum audit trail object type. This is normally “dm_audittrail” but custom audit trails object types are supported as well.
The following audit trail attributes don’t need to be set through the transformation rules because they are automatically taken from the corresponding audited object in target system: chronicle_id, version_label, object_type.
All other “dm_audittrail” attributes that refer to the audited object (acl_domain, acl_name, audited_obj_vstamp, controlling_app, time_stamp, etc) can be set either to the values that come from the source repository or not set at all, case in which the importer will set the corresponding values by taking them from the audited object located in the target repository.
The source attributes “attribute_list” and “attribute_list_old” may appear as multi-value in migration-center. This is because their values may exceed the maximum size of a value allowed in migration-center (4000 bytes), case in which migration-center handles such an attribute as a multi-value attribute internally. The user doesn’t need to take any action to handle such attributes; the importer knows how to process and set these values correctly in Documentum.
Not setting an attribute means not defining a rule for it or not associating an existing rule with any object type definition or attribute.
Working with rules and associations is core product functionality and is described in detail in the Client User Guide.
Migrating Updates (“Update” or “Delta” Migration)
The D2 Importer does also support “Update” or “Delta” migration if the source data is proved through a scanner which also supports the feature, such as the Documentum Scanner or File System Scanner.
Objects that have changed in the source system since the last scan are scanned as update objects. Whether an object in a migration set is an update or not can be seen by checking the value of the Is_update column – if it’s 1, the current object is an update to a previously scanned object (the base object). Some things that need to be considered when working with the update migration feature in combination with the Documentum Scanner will be illustrated next:
An update object cannot be imported unless its base object has been imported previously.
Updated objects are detected based on the r_modify_date and i_vstamp attributes. If one of these attributes has changed, the object is considered to have been updated and will be registered accordingly. By default actions performed in Documentum change at least one if not both of these attributes, offering a reliable way to detect whether an object has changed since the last scan; on the other hand, objects changed by third party code/applications which do not touch these attributes might not be detected by migration-center as having changed.
Objects deleted from the source after having been migrated are not detected and will not be deleted in the target system. This is by design (due to the added overhead, complexity and risk involved in deleting customer data).
Updates/changes to primary content, renditions, metadata, VD structures, and relations of objects will be detected and updated accordingly.
Extracting an object type from Documentum Content Server and preparing it for migration-center 3.x
migration-center Client, which is used to set up transformation and validation rules does not connect directly to any source or target system to extract this information. Object type definitions can be exported from the respective systems to a CSV file which in turn can be imported to migration-center.
One tool to easily accomplish this for Documentum object types is dqMan, which is used in the following steps to illustrate the process. dqMan is an administration Tool for EMC Documentum supporting DQL queries and API commands and much more. dqMan can be purchased at https://dqman.com. Other comparable administration tools can also be used, provided they can output a compatible CSV file or generate some similar output which can be processed to match the required format using other tools.
Start dqMan and connect to the target DOCUMENTUM repository. dqMan normally starts with the interface for working with DQL selected by default. Press the [DQL] button in the toolbar if not already selected.
In the “DQL Query” -box paste the following command and replace dm_document with the targeted object type: select distinct t.attr_name, t.attr_type, '0' as min_length, t.attr_length, t.attr_repeating, a.not_null as mandatory from dm_type t, dmi_dd_attr_info a where t.name=a.type_name and t.attr_name=a.attr_name and t.name='dm_document' enable(row_based); Press the [Run] button.
Click somewhere in the “Results” box. Use {CTRL+A} to select all. Right-click to open the context menu and choose <Export to> <CSV>.
The extracted object type template is now ready to be imported to migration-center 3.x as described in the Client User Guide.
Content Validation
The Content Validation functionality is based on checksums (an MD5 hash) computed for the document’s contents during scan (by the Scanner itself) and after import (by the Importer). The two checksums are then compared - for any given piece of content its checksums from before and after the migration should be identical.
If the two checksums differ in any way, this is an indication that the content has been corrupted/modified during/after migration. Modifications can happen on purpose or simply by user error and can usually be traced back based on the r_modifier and r_modify_date attributes of the affected object. Actual data corruption rarely happens and is usually due to software and/or hardware errors on the systems involved during content transfer or storage.
Validating content is always performed after the content has been imported to the target repository, thus adding another step to the migration process. Accordingly, using this feature may significantly increase import time due to having to read back every piece of content for every document and compute its checksum in order to compare it against the initial checksum computer during scan.
This feature is controlled through the checkContentIntegrity parameter in the D2 Importer (disabled by default).
Note: Currently only documents scanned by Documentum Scanner or Filesystem Scanner can be used in a migration workflow involving Documentum Content Validation. The D2 Importer does support Content Validation when used with the previously mentioned Scanners.
Renditions
The Documentum Content Validation also supports renditions in addition to a document’s main content. Renditions are processed automatically if found and do not require further configuration by the user.
There is one limitation that applies to renditions though: since the Content Validation functionality is based on a checksum computed initially during scan (before the migration), renditions are supported only if scanned from a Documentum repository using the Documentum Scanner. Currently this is the only scanner aware of calculating the required checksums for renditions; other scanners, even though they may provide metadata pointing to other content files which, may become renditions during import, do not handle the content directly and therefore do not compute the checksum at scan time needed by the Content Validation to be compared against the imported content’s checksum.
Rollback script
Each job run of the importer generates along with its log a rollback script that can be used to remove all the imported data from the target system. This feature can be very useful in the testing phase to clear the resulting items or even in production in case the user wants to correct the imported data and redo the import.
The name of the rollback script is build based on the following formula:
<Importer name>(<run number>)_<script generation date time>_ rollback_script.api
Its location is the same as the logs location:
<Server components installation folder>/logs/DCTM-Importer/
Composed by a series of Documentum API commands that will remove in the proper order the items created in the import process, the script should look similar to the following example:
//Links:
//Virtual Document Components:
//Relations:
//Documents:
destroy,c,090000138001ab99,1
destroy,c,090000138001ab96,1
destroy,c,090000138001ab77,1
destroy,c,090000138001ab94,1
//Folders:
destroy,c,0c0000138001ab5b,1
You can run it with any applications that supports Documentum API scripting this includes the fme dqMan application and the IAPI tool from Documentum.
The rollback script is created at the end of the import process. This means that it will not be created if the job run stops before it gets to this stage, this doesn’t include manual stops done directly from the client.