Data Migration Workflow
A practical migration workflow that keeps each stage visible, controlled and verifiable from assessment through to final cutover.
Overview
Essential stages for a successful migration
Here we outline the essential stages of a data migration process tailored to the unique requirements of our business clients. While the resources needed for each step may vary depending on the project, these key stages remain constant. Our DM REVOLVE toolset navigates the migration through each stage, while Workflow-O-Matic (WOM) records, schedules and verifies progress along the way.
Use the workflow below as the delivery spine for scope, mapping, extraction, validation, loading and cutover.
Workflow stages
- Project scoping and assessment
- Mapping and transformation design
- Extraction and cleansing
- Validation and testing
- Loading and integration
- Final cutover and support
Why it works
The workflow keeps the migration visible at every step, allowing the team to verify progress and keep the project on schedule.
Process
The Data Migration Workflow
Project Scoping and Assessment
Define the migration scope, constraints, risks and business objectives before delivery begins.
Data Mapping and Transformation Design
Document source-to-target mappings and design the transformation rules needed for accurate movement.
Data Extraction and Cleansing
Extract the required data, improve quality and remove issues that could affect downstream migration.
Data Validation and Testing
Validate the transformed data and test the migration flow before any live loading occurs.
Data Loading and Integration
Load the prepared data into the target environment and ensure it integrates correctly with the new system.
Final Cutover and Post-Migration Support
Complete the cutover, support the go-live process and resolve any issues that appear after migration.
By Paul Bimrose on July 27, 2021
DM Planning and Scoping
DM Strategies
Mapping
Extract
Transform
Load
Check and cleanse
Validate
The following slides break down each process
Time spent on planning early on in the project can be invaluable. Discovery and scoping in the planning stage helps to spot potential problems down the road. It includes, but is not limited to
Investigating legacy systems
Defining the data migration scope ie objects to be included
Hardware and security requirements
Data quality concerns
Data volumes
DM strategies are used to define
DM scope within individual data objects eg open transactions only
The DM requirements of each individual data object
Cutover strategies for particular data objects
Specific data quality concerns and how to address them eg duplicates
Expected data volumes
Mapping the data from the legacy system to the new system can present many challenges.
Missing values
Incorrectly formatted data
Invalid values
SME’s (Subject Matter Experts) from the clients side will be essential in working through the legacy data sources for the relevant records and fields.
Once the data for each object is identified we can extract it from the legacy system. The relevant information can be obtained with the help of
Communications between ourselves and the SME’s
Select statements
DM REVOLVE
Hard work!
Once extracted the data objects are checked for completeness and correctness
Once the extraction has been completed any transformations that the data may need should be done. This will usually be changing legacy fields, for example
Months of the year, from January, February etc to Jan, Feb
Splitting a single name field out into first, middle, surname
A set value
Conversion from old to new reference mapping
Once the transformations have finished, each object is checked by the data owner for correctness
All of the data has now been
Extracted from the legacy system
Cleansed
Transformed
It’s time to load it into the new target system. This is done in a simulated testing environment, it will be done several times as the project progresses and new data objects are completed.
If a data object doesn’t pass validation it will need to be checked over again for further refinement. It will be progressed back to whichever stage is needed to fix the problem and stepped through the processes again. This will continue until it is validated and signed off. A data object that needs refinement can go through this process several times before passing.
With the help of our nightly refreshed ROM reports a focused data cleansing effort will be used.
As the various data objects are loaded into the target system they will need to be validated. This is done by the data owner and needs to be a thorough process, catching any data that is
Missing
Incorrect
Excess records eg duplicates
Financially verified
Functionally correct in the new target system
Once the data objects have been validated the migration procedures are locked down and ready for the deployment cutover.
Next step
Use the workflow to keep your migration on track
By leveraging the power of the DM REVOLVE toolset and WOM system, we provide a seamless and efficient migration experience that addresses the specific needs and challenges faced by business clients.
