Sizing Guide
Introduction
The present document offers a guideline for sizing and configuring migration-center components in different migration scenarios. Due to the complexity of the migration projects, enterprise environments and applications migration-center interacts with, this guide does not provide exhaustive information, but it offers best practice configurations accumulated in more than 100 migration projects.
The Installation Guide and Database Administrator’s Guide contain a full description of the system requirements and installation steps for all migration-center components and should be consulted first. This document does not substitute the documents mentioned above but completes it with information that might be of importance for getting the best migration-center performance.
Targeted audience
This document is targeted specifically to the technical persons that are responsible with installation and configuration of migration-center components.
Common configurations and common sizing guidelines
This chapter describes the configurations and sizing guidelines that apply to any kind of migration project.
Client
The following hardware and software resources are required by MC client in any of the migration scenarios described in the next chapters.
Operating system: | Windows Server 2003 , 2008, 2012, or 2016 Windows 7, 8, or 10 |
Required software: | Oracle Client 11g R2 - 19c, 32-bit |
CPU: | Any CPU that is supported by OS will be supported by MC client |
RAM: | Usually it doesn’t consume more the 200 MB of RAM. This threshold might be overtaken when loading more than 100,000 objects in a grid. |
HDD: | 10MB |
Server components (Job Server)
Operating system: | Windows Server 2003 , 2008, 2012, or 2016 Windows 7, 8, or 10 |
Required software: | Oracle / OpenJDK 8 or Oracle / OpenJDK 11, 32-bit or 64-bit For migration to/from Documentum DFC 5.3 or later needs to be installed |
CPU: | Dual or Quad core processor min. 2.5 GHz |
RAM: | The Job Server is configured by default to use 1 GB of memory. This will be enough in most of the cases. In some special cases (multiple big scanner/importer batches) it can be extended to 1.5 GB (with 32-bit Java) or higher values (with 64-bit Java) through the configuration. |
HDD: | For installation 300 MB of disk space is required. An additional disk space is required for the logs. In most of the productive migration when debugging is not activated, about 50 MB should be reserved for every 100,000 migrated objects. |
Content storage: | The scanners running in Job Server extract the content of documents from the DMS and store them in a file system folder. That folder might be located on the Job Server machine (in case of small and medium migration projects) or on dedicated storage machine, NAS or fileserver in case of large and very large projects. How much disk space is needed to be reserved for exporting the content depends on the number and the size of documents that have to be migrated. |
Database
The migration-center database stores all information generated during a migration project. This includes job configurations, object metadata, transformation rules, status and history information, etc.
Operating system: | See Oracle system requirements |
Required software: | Oracle 11g R2, or Oracle 12c R1 and R2, Oracle 18c, Oracle 19c |
CPU: | Depends on the migration size (see next chapters) |
RAM: | Depends on the migration size (see next chapters). |
Data storage: | Generally sizing of the data files is determined based on the number of objects expected to be migrated using the respective database instance. By default, the tablespaces created by MC installer totalize 60 MB. They are set to auto extend automatically when more storage space is required by MC components. There is no upper limit set. The required storage is dependent by the number of objects that will be migrated and the amount of metadata that will be scanned and imported. Roughly 75% of necessary storage is occupied by the objects metadata (source and target metadata). The rest of 25% are required by the configurations and other internal data. To estimate the necessary storage for a migration set the following formula might be used: Storage size = 5 * (NR_OF_OBJECTS) * (AVG_METADATA_SIZE) NR_OF_OBJECTS – the number of objects and relations to be migrated AVG_METADATA_SIZE – the average size in bytes of an object metadata (attribute names size + attribute values size) Example: Migration of 800,000 documents and folders from Documentum into Documentum. The medium number of attributes was 70 and the estimated AVG_METADATA_SIZE was 5,000 bytes. Storage size = 5 * 800,000 * 5,000 = ~20 GB. Note: In order to be accessible from any MC client, the scanner/importer report log files are stored in MC database as CLOB fields. If big scanners or importer are run with “debugLevel=4” the size of report logs gets increased significantly and therefore additional tablespace should be taken in consideration. |
Character set | The Unicode (AL32UTF8) character set is recommended to be used in order to handle correctly any kind of character that may appear in the metadata. |
Other recommendations | MC inserts and updates rows intensively and therefore fast disks help the performance. Even though the rows are inserted and updated intensively the MC transactions are generally small. There are no situations when more than 5,000 table rows are inserted, updated or deleted in a single transaction. |
Small projects (up to 500,000 objects)
Sizing and configuration tips
No special hardware or software requirements
All MC components can be run on the same machine
Deployment overview
For such a small migration projects, where up to 500,000 objects need to be migrated, there are no special hardware or software requirements. Basically, all three MC components can be run on a single desktop computer or on an adequately sized virtual machine. In this case the machine should have enough physical memory for OS, Oracle server, MC components and other applications that may run on that machine. The recommended processor is a dual or quad core having a clock rate of minimum 2.2 GHz.
Oracle instance
Standard database installation can be followed (as it is described in “migration-center installation guide”)
RAM: 4 GB of total memory for the instance.
Use “Automatic memory management”. This can be chosen when creating the Oracle instance.
Medium migration projects (500,000 – 5 million objects)
Sizing and configuration tips
Dedicated Oracle Server machine
Two MC Job Servers for a better scalability
Deployment overview
It is recommended to use a dedicated machine for the Oracle instance needed by MC.
For a better scalability of scanning and importing jobs or when migration timeframe is short, it’s recommended to deploy the Server Components (Job Server) on two machines. In this way you may run multiple scanner and importers in parallel speeding up the entire migration process. The performance of scanners and importers is dependent by the Source and Target System so if one of those systems performs slowly, the migration process can be speeded up by running multiple scanners and importers in parallel. If necessary, the number of deployed Jobservers can be extended.
Oracle instance
The host should not necessarily be a server machine. A well sized desktop machine would be enough.
CPU: Quad Core, min. 2.5 GHz
8 GB of memory allocated for the instance
Use “Automatic memory management” in order for the instance to tune its target memory size, redistributing memory as needed between the system global area (SGA) and the instance program global area (instance PGA). If that is not possible, the recommendation is to allocate as much as possible for SGA memory (especially for buffer cache) and keep PGA memory in the range of 200 MB.
It is very recommended the instance has at least 3 Redo Log files having in total a minimum size of 1 GB. This is important when multiple big transformation and validation jobs are run by MC because those jobs update a big number of database rows that require enough redo log space. You may get the information about existing redo log files by running the query: SELECT * FROM v$log;
Large migration projects (5 million – 15 million objects)
Sizing and configuration tips
Dedicated Oracle Instance running on a Server machine.
Three or more MC Job Servers for a better scalability
Deployment overview
It is recommended to use a dedicated Oracle Instance running on server hardware.
For a better scalability of scanning and importing jobs three or more instances of Server Components (Job Server) need to be deployed. The performance of scanners and importers is dependent by the Source and Target System so if one of those systems performs slowly, the migration process can be speeded up by running multiple scanners and importers in parallel. If necessary, the number of deployed Job Servers can be extended.
Oracle instance
A dedicated server machine is required. It is recommended to use a dedicated Oracle instance for running only Migration Center but no other applications.
CPU: 4-8 Cores, min. 2.5 GHz
16-32 GB of memory allocated for the instance
Use “Automatic memory management” in order for the instance to tune its target memory size, redistributing memory as needed between the system global area (SGA) and the instance program global area (instance PGA). If that is not possible the recommendation is to allocate as much as possible for SGA memory (especially for buffer cache) and keep PGA memory in the range of 400 MB.
Make sure that instance has at least 4 Redo Log files having in total a minimum size of 1.5 GB. This is important when multiple big transformation and validation jobs are run by MC because those jobs update a big number of database rows that require enough redo log space. You may get the information about existing redo log files by running the query: SELECT * FROM v$log;
Very large migration projects (more than 15 million objects)
Sizing and configuration tips
Multiple Dedicated Oracle Instances running on a Server machine or use an instance on a high-performance Oracle cluster
Four or more Job Servers.
A file server or a storage server used for temporary storing the content.
Deployment overview
The deployment for the migration of a very large number of objects should be planned carefully. In this case it is recommended to use multiple MC databases. In most cases it is not advised to migrate more than 10 million objects with a single Oracle instance even if the hardware is sized accordingly. There are several reasons for using multiple database instances:
The data from different sources is not mixed up in a single instance helping the user to handle easier the errors that appear during migration
The transformations and validations will be better scaled on multiple database instances
Facilitate the work of multiple migration teams
A dedicated file/storage server should be shared by all Job Servers for storing and accessing the content during migration. This will help the migration in the way that set of objects scanned with any scanner can be imported with any importer.
Oracle instance
Several dedicated server machines are required. For each machine it is recommended to use a dedicated Oracle instance for running only Migration Center but no other applications.
CPU: 4-8 Cores, min. 3.0 GHz
Minimum 16 GB of memory allocated for each instance.
Use “Automatic memory management” in order for the instance to tune its target memory size, redistributing memory as needed between the system global area (SGA) and the instance program global area (instance PGA). If that is not possible the recommendation is to allocate as much as possible for SGA memory (especially for buffer cache) and keep PGA memory in the range of 600 MB.
Is recommended that FMEMC_DATA tablespace to be split on multiple physical data files stored on different physical disks in order to maximize the performance for a specific hardware configuration.
Make sure that instance has at least 5 Redo Log files having in total a minimum size of 2 GB. This is important when multiple big batch jobs (transformations, validations, scanners, importers) jobs are run by MC because those jobs update a big number of database rows in quite a short time and therefore redo log space should be sized accordingly in order to prevent redo log wait events. You may get the information about existing redo log files by running the query: SELECT * FROM v$log;
Last updated