How to migrate Magento 2 from one server to another

#1

Chapter 1: Introduction

As you can see, I have split my screen into 4 sections. We’ve got my web browser of choice (in this case Google Chrome), my SSH Client of choice (which is Putty for Windows - Available at Putty.org), a breakdown of Chapters to help you quickly navigate through the video. And finally any notes that will assist me.

There is a slight twist though… Because I’ll be accessing both my Original Server and my New Server during this video, I will be changing the appearance of the bottom-left window to represent each server to help distinguish them.

This is how it will appear when I’m accessing my New Server… And then I’ll add this red border around the window when I’m accessing my Original Server.

So, whether you’re moving Web Hosts or creating a Development Environment to safely work on your store, all of the steps in this tutorial will be the same apart from that last.

Also, this tutorial will work whether you’re Original Server is in either Developer or Production Mode. Again, the only step that will vary is at the end.

As I alluded to at the beginning of this video, I’ve already gone ahead and prepared my New Server to run a copy of Magento 2.3 before creating this tutorial. If you’d like to know exactly how I setup this New Server then you should watch the first 40 minutes of my “How to install Magento 2.3 and build a web server” tutorial from last year - Up to and including the “Folder permissions” section. I’ll add a link to that video in the description below.

Chapter 2: Original Server Preparation

Database Backup

There’s essentially 2 parts to Magento when it comes to data. There’s the media and system files which are all located in the root directory of Magento. And there’s also the database that’s stored elsewhere on the web server. So, when I come to copy the files over I’ll need to ensure I also grab a fresh copy of the database. And the easiest way to grab a copy of the database is to run a database backup.

This can easily be achieved from the backend of Magento under System > Backups and following the steps for a database backup. However, as this is a tutorial I’ll show you how this can also be achieved from the Command Line Interface.

Right now, I’m logged into my Original Server as the magento user. As usual, I’ll navigate to the Magento root directory by entering:

cd /var/www/html

I know I’m in the right place, because when I list all of the files in the current directly using ls -la it’ll look something like this.

Now, I’ll use a tool called mysqldump that will allow me to create a full backup of my Magento Database. And that command looks like this:

mysqldump -p -u magento-master magento-master > var/backups/migrate.sql
  • mysqldump is the tool
  • -p enabled the password prompt required to execute the command
  • -u magento-master sets the mysql username I want to perform the action as
  • magento-master sets the mysql database I want to perform the action to
  • > sets the action to export
  • var/backups/migrate.sql sets the location and filename I want to export to

After pressing Enter, I will be prompted for the password of the magento-master user. After entering the password and pressing Enter once again, the screen will appear frozen - Just displaying the Enter Password prompt. In actual fact, the database is being backed up but there is no status or output whilst this is happening. The process can take anything from a few seconds to a few minutes depending on the size of your database. In my experience I’ve waited up for 4 minutes for a 5 Gigabyte database, so don’t panic if you’re waiting a while.

Once the backup is complete, you won’t see a confirmation message. Instead, the normal prompt will just reappear. But for piece-of-mind I’ll verify that the database has been backed up by typing:

ls -la var/backups

And in the output I can see the new migrate.sql file.

Firewall Whitelisting (Digitalocean)

Next, I need to allow my New Server to access my Original Server via SSH. This is where things start to get a little grey because each of your setups will vary depending on your Web Host Services. So, I’m going to give you 2 potential scenarios for doing this.

Essentially your web server can have 2 Firewalls. The firewall that’s built into your Web Servers Operating System. And a Firewall that may or may not be provided by your Web Host. My Web Server is hosted by Digitalocean who do in fact provide a Firewall Service that sits in front of my Web Server. This service easily allows me to modify my firewall from the comfort of my Digitalocean account…

By the way, if you’re thinking about using Digitalocean then my referral link in the video description below currently entitles you to $100 of free credit.

[Walkthrough in Video]

If you’re using another Web Host that provides a similar service then you’ll need to refer to their own documentation because every Web Host Service is different.

Firewall Whitelisting (In-built)

In the second scenario, the in-built firewall that’s ran on the Web Servers Operating System is controlled via the Command Line Interface.

But before I can execute the following command, I need to switch over to my superuser - Because my magento user doesn’t have the correct permissions to run it. I can switch to my superuser by typing:

su craig

I’ll then be prompted for a password. This is the password of my superuser that I’m trying to log in as, not the password for my magento user.

Now that I’m logged in, I can simply update my Original Servers whitelist by entering the following command:

sudo ufw allow in from 178.62.22.25 to any port 22
  • sudo executes the command as a superuser
  • ufw is the tool
  • allow in from 178.62.22.25 sets the IP address of the server I want to allow
  • to any port 22 sets the port number that I want to allow the IP to connect to

Now I just need to run the following command for the changes to take effect:

sudo ufw reload

At this point both preparations of the Magento Database Backup and the Firewall Whitelist have been made. So I can completely log out of my Original Server now.

Remember, that only you and your Web Host will ultimately know where your Web Servers Firewall is controlled from. So, you’ll need to refer to their documentation if you’re stuck.

Chapter 3: New Server Preparation

Preparing the magento root directory

Now I’m at the point where I’m logged into my New Server as the magento user via the Command Line Interface. Thankfully, there’s only one thing I need to do in order to prepare my New Server.

Remember, I already went ahead and setup my New Server. To see exactly how I setup this New Server you should watch the first 40 minutes of my “How to install Magento 2.3 and build a web server” tutorial from last year - Up to and including the section entitled “Folder permissions”. There’s a link to that video in the description below.

So, now I’m going to navigate to the magento root folder by typing:

cd /var/www/html/

When I originally installed Apache on my New Server, it created a test file in this directory called index.html. I can verify that by either typing in the IP address of my new server in a browser and viewing this page. Or by typing ls -la in the Command Line Interface. So, before I start copying any files I’m going to delete that file with the following command:

rm index.html

Now I have a nice clean folder to work with.

Chapter 4: Data Migration

rsync

The tool of choice that I’ll be using to do the heavy lifting today is called “rsync”. This is a great tool that allows the secure transfer of files over SSH. Thus not requiring the need to download files from one server and upload them onto another (which can take a long time).

You can use rsync on either server, because you can either tell it to:

  • Copy local files to a remote server
  • Copy files from a remote server to a local directory

Normally, it doesn’t matter which way around you do this but the preparations I’ve made so far assume that I’ll be copying files from a remote server to a local directory

Copying the files

So, now going to execute the rsync command that will allow me to copy the contents of the magento root folder from my Original Server onto my New Server in the location from where I’m executing to the commands. And this is the command I’ll be using:

rsync -avzhe ssh magento@209.97.134.30:/var/www/html/ .

This can be broken up into 7 sections:

  • Rysnc initiates the utility
  • -avzhe represents a number of options I need to enable when running rsync
  • ssh specifies that I want to use the ssh protocol
  • magento is the username of the user on my Original Server who I want to login as
  • @209.97.134.30 is the IP address of the server I want to connect to
  • :/var/www/html/ is the directory I want to copy. It’s important to note the “/” at the end of “/var/www/html/”. If you forget to add this slash, then the rsync tool will copy both the “html” folder and the contents - Instead of just the contents. So, without the trailing “/” you end up with a directory called /var/www/html/html/".
  • Finally the “.” is the location I want to paste the files on my New Server. The “.” simply means paste it from the location where I’m running the command from. As a reminder I’ll be running the command from /var/www/html/ which is where I navigated to back in the last Chapter. Therefore, the files will be copied there.

When I press enter, I’ll be asked about a fingerprint. Typing yes and hitting Enter will then prompt me for the password of the user that I’m trying to log in as (which is the magento user).

As soon as I enter the password and hit enter, the screen will display a list of files currently being copied. This can take a couple of minutes, so I’ll skip to the end.

Verifying the copied files

Now that the rsync tool has finished, I can type ls -la to verify that all of my files have been copied across correctly.

As you can see, the /var/www/html/ folder now contains all of my Magento 2 files from my Original Server.

Chapter 5: Post Migration Setup

Ownership, Groups and Permissions

When I ran rsync, it copied across all of the Magento Files including the backup of the database. But before I move onto the database restoration, I want to do some housekeeping with the Ownership and Permissions of the files to ensure everything is correct.

When I last ran the ls -la command to list all of the copied files, you may have noticed that both the Ownership AND Group were carried over from my Original Server - Which is exactly what I wanted to happen.

However, if you’re using different credentials on your new server such as username “bob” instead of “magento” then you’ll need to amend this before you continue further. Checkout the article on further instructions (see bottom of this article). As I don’t, I’ll continue to the next step.

Next I want to verify that all of the Folder and Files have the correct permissions for Magento to run correctly. In theory the rsync command should have preserved all of this data, but it doesn’t hurt to check using the following command:

find var generated vendor pub/static pub/media app/etc -type f -exec chmod g+w {} + && find var generated vendor pub/static pub/media app/etc -type d -exec chmod g+ws {} + && chmod u+x bin/magento

Don’t forget that you can copy and paste this command from the article linked in the description.

Troubleshooting

Sometimes, after I migrate servers I run into an issue where the static content doesn’t clear properly. To remedy this issue, I now always run the following command at this point:

rm -rf pub/static/adminhtml pub/static/frontend

This simply deletes the temporary static content files for the frontend and backend, which forces them to be regenerated next time you load the them.

Database Preparation

Now it’s time to create a MySQL User and Database for Magento. Once these have been created, I can simply import or restore my backed up database.

  1. So I’ll visit my phpMyAdmin page at http://178.62.22.25/phpmyadmin/
  2. Then log as my mysql root user
  3. Once logged in, click User Accounts
  4. Then click Add user account
  5. For the username I’ll have to enter the same details as what I used on my Original Server, which is magento-master
  6. Change Host name to Local
  7. And for the password, I’ll also need to enter the same one that I used on my Original Server
  8. Now click the box below labelled “Create database with same name and grant all privileges.”, which will conveniently create an empty database for me at the same time as creating the user.
  9. And now hit Go at the bottom-right of the page

When following these steps, it’s possible that you will see an error stating that “Your password does not satisfy the current policy requirements”. This is simply a side-effect from having a weak password for your Magento Database to begin with. If you do run into this issue, the article will go over how to overcome this issue (see bottom of this article).

But as I’ve successfully prepared everything I need to restore my database backup. I’m going to leave phpMyAdmin open and continue to the next step.

Database Restoration

In order to restore my Magento Database, I need to import my database backup into the empty database I just created. To do that, I need to enter the following command

mysql -p -u magento-master magento-master < var/backups/migrate.sql
  • mysql is the tool
  • -p enabled the password prompt required to execute the command
  • -u magento-master sets the mysql username I want to perform the action as
  • magento-master sets the mysql database I want to perform the action to
  • < sets the action to import
  • var/backups/migrate.sql sets the location and filename I want to import from

Just like when I ran the mysqldump command, the screen will appear frozen showing the Enter password prompt. But after a few seconds or minutes the standard prompt will reappear once again. The easiest way to verify that the database has been imported is by clicking on the “magento-master” database in phpMyAdmin, which will refresh the page and display all of the imported tables.

Chapter 6: URL Setup

Database Amendments

Now that the database restoration is complete, I’ll need to manually amend a few records in the database in order for it to be accessible via my web browser. That’s because the database is currently configured to work as if I’m accessing this copy of Magento via my Original URL (digitalstartuptutorials.co.uk). But in actual fact, I’ll be using this new server for development purposes - Therefore I have no need to setup a domain. I just want to access my New Server by entering its IP address in my web browser.

To do this, I’ll refresh phpMyAdmin by clicking on the magento-master database - Which will allow me to see all of the tables I just imported. Then I need to open the core_config_data table by clicking on it.

Now, there are 4 values that require my attention. The first 2 are fairly obvious. The path names are web/unsecure/base_url and web/secure/base_url. As you can see the values for these are the domain names I was originally using. One with a prefix of HTTP for unsecure and the other with a prefix of HTTPS for secure.

By double clicking on the values (like so), I can delete the domain name and replace it with the IP address of my New Server. It’s really important that I leave the prefix and suffix or this won’t work. So, you can see how I’ve left the HTTP:// at the beginning and the / at the end.

The next 2 path names are called web/secure/use_in_frontend and web/secure/use_in_adminhtml. If your Original Server has an SSL Certificate and forces HTTPS then these values will be set to “1”. When you leave these as “1” on a server that doesn’t have an SSL Certificate installed or or HTTPS setup (such as my New Server I’m working on here) then the website won’t work when you try to load it in your browser. In a situation like this, both values would need changing from a “1” to a “0”. In this demonstration, my Original Server never had this enabled to begin with - So, I can just leave it as it is.

If you’re simply moving your website to another Web Server and plan to keep the same URL then it’s unlikely you’ll want to make these Database Amendments. Instead the only change you’ll need to make is updating the Server IP with your Domain Provider.

Results

Finally, with all of the steps now complete it’s time to clear the Magento cache. These steps will vary depending on whether you were in Developer Mode or Production Mode at the time of the migration.

(Developer Mode)
So, assuming Magento was already in Developer Mode when you migrated it over then you’ll want to enter the following command and you’re done:

bin/magento cache:clear

(Production Mode)
But if your Original Server was already in Production Mode (just like mine), then you’ll want to enter the following 3 commands:

  1. bin/magento setup:di:compile
    
  2. bin/magento setup:static-content:deploy en_GB en_US
    
  3. bin/magento cache:flush
    

Now, if I’ve done each step correctly and I type in the IP address of my New Server into the browser, I should see the copy of my site. And just to check, I’ll load the backend and login…

…And there you have it.

I almost forget!! Cron Tasks

You’ll need to set these back up again on your new server. It’s super-simple. Just run the following command as your magento user from within the Magento directory:

php bin/magento cron:install

Summary

Well, I hope this video taught you something new and gave you the confidence to try this yourself. If you do follow along, always remember to backup your Production Server before doing anything. And always pay particular attention to each step and command as I find that most people who reach out for help get stuck because of something they missed or typo’d.

However, if you do get legitimately stuck or you have any additional questions that you’d like answering then please feel free to join the Digital Startup Community Forum where either myself or another member will try to help you out.

Thank you for watching and until next time, take care.


Appendix

How to Set Folder and File Ownership and Group

If at any point you need to change the Ownership or/and Group information of your Magento Files and Folders then you need to run the following command as a superuser:

  • sudo chown -R [owner]:[group] [directory] (Structure)
  • sudo chown -R bob:www-data /var/www/html/ (Example)

When you run into “Your password does not satisfy the current policy requirements”

At the point of seeing this error, simply start again by clicking on User Accounts. This time, create a new strong password by following these rules:

  1. Longer Than 16 Characters
  2. Contains Numbers
  3. Contains Lower & Uppercase Characters
  4. Contains Special Characters

Once you have successfully completed the steps for creating the MySQL Magento User and Database, you will now need to update the Magento file that contains the old password. You can do this by using the nano text editor to edit the env.php file using this command as magento user:

nano app/etc/env.php

In the first few lines, you will see a line containing your old password like this 'oldpassword'. Just replace this with your new password. When finished press Ctrl+X to exit then “Y” and Enter to save your changes.

Other Useful Links

0 Likes

Tweaking my YT video format (I'd love your feedback)
Creating a Copy of my Website
Magento sends me to original IP after server migration