Magento 2 Migration Strategy

May 08, 2019
By Bounteous

As we get closer to the Magento 1 end of life date of June 2020 it's important to start making a plan of action now for your migration strategy. Data migration is never a trivial task, even when you're just upgrading between versions - which is why it's important to consider all of the factors before choosing how to upgrade your eCommerce solution.

In the best case, a migration like Magento 1 to Magento 2 wouldn't even be detectable. In the worst case, you could trigger a catastrophic event such as permanent loss of critical data. As we've helped our clients through this process, we've compiled a few tips and best practices that will help make the process go smoothly. Fortunately, the Magento 2 Data Migration Tool gives you a great deal of control over how well your migration will go!

Know Your Data!

In order to have the best result, you need to have an intimate knowledge of your Magento 1 data. This includes both the core data structure and any customizations that have been made to the database. The structure of the Magento 2 schema resembles that of Magento 1, so luckily much of that knowledge will be transferable.

Familiarize Yourself with the Migration Tool

The migration tool is driven by a series of XML mapping files that list source and destination tables and columns. Any custom tables, columns, or column type changes will have to be accounted for in these mappings — otherwise, your migration will fail. These mapping XML files are what give you the power over the migration tool. Here are a few examples of some of the control nodes you’ll see in these map files:

<ignore> is used to skip the specified table or column during migration.
<rename> is used to change column or table names during the migration.
<move> can relocate columns from one table to another.
<transform> is used on table row values to manipulate the raw data during migration.

Once you're very comfortable with the structure of your Magento 1 database, have read all the official migration documentation, and have a grasp of how the migration tool is going to operate, it’s time to start getting your hands dirty. The best way to learn about the migration tool is to start using it. So, in a non-production environment, take a MySQL backup of your Magento 2 database, set up your migration’s config.xml file, trigger a migration, and see what happens!

[ERROR]: Source documents are not mapped: custom_table

Don’t be surprised if the data integrity check fails. In the migration’s config.xml file, the steps XML node contains a list of migration “steps” each responsible for different portions of the migration. They are executed in the order listed here, the first of which is the Data Integrity Step. This step compares your source and destination schemas to make sure the mappings are correct. The migration tool is pretty good at telling you why the integrity check failed, stating specifically which table or column, referred to as documents and fields respectively, failed to pass the check and why. For example, the above error message indicates that there is a table named “custom_table” that exists in the Magento 1 database that is not accounted for in the mapping file.

You’ll need to customize your mappings in order to pass the data integrity check before it will start attempting to move any data.

Iterate and Overcome

All you can do next is run the migration, expose the mapping shortfalls through integrity check failures, adjust your mappings, and try again. There may be a need to clean your Magento 1 data during this process. The Data Integrity Step verifies foreign key constraints, so orphan records in your Magento 1 database may need to be dealt with.

Once you’ve gotten past the Data Integrity Step, any migration failures require you to re-import a backup of your Magento 2 database. Presently, there is no way to revert or “undo” a migration. Errors past the Data Integrity Step may be less specific. Try increasing the verbosity by adding a -v flag to the migration command. I also find it helpful to output the results into a log for posterity.

bin/magento migrate:data --reset /{path_to_config}/config.xml -v > migration_output.log

Full Throttle Testing

Once you’ve managed to wrangle the data migration tool and have successfully completed your first test migration — it’s time to start testing. Test everything! I’m not exaggerating. Test literally every data-driven feature that exists in your build. You won’t get to come back from this once you pull the trigger in a production scenario so it's of utmost important to test everything before and make sure it functions properly. Taking the time now to do a full regression test, and finalize your mappings is time well spent. You’ll likely have to do this multiple times over as you adjust and refine the mappings.

Production Migration - Brace for Impact (Fingers Crossed)

I’m going to end by sharing a few things that helped us with our successful production migrations.

If your Magento 2 codebase and migration tool were installed via composer, you need your custom config and mapping files to exist in the magento/vendor/magento/data-migration-tool/etc/ directory. We got around this by committing our custom files in a separate directory, then symlinking those files into their respective places in vendor once they were deployed to the production server.

If you are dealing with a large Magento 1 database, it might be valuable to export the database and import it as a separate schema into your Magento 2 MySQL server. This might speed up your migration by reducing network latency.

Make sure you do some test runs with production data before launch day. Not only will you get a grasp of how long the production migration might take, but it will expose any data integrity failures of your production data. One unserializable value or orphan record that didn’t exist in your test database could throw a wrench in your production migration.

The process can be long and iterative, so it pays to start early and plan accordingly. When our team finally got to the point of doing our first production data migration, it had been weeks of customizing mapping files, writing custom migration steps, and seemingly endless test cycles. While many companies may have the skills in-house to perform the migration, it may be necessary to seek out a partner with Magento 2 upgrade experience who can help carefully navigate the intricacies of the process. Ultimately, the time you put into the process will pay off dividends, so plan carefully, test continuously, and migrate successfully!