Users accounts Server/Work-Area Migration - Expediting the Migration Process
NOTE: This tutorial is provided for systems administrators, it requires strong knowledge of linux command-line environment. One must proceed with caution.
Contents
Summary
This procedure helps expediting the migration process of users accounts from one server/workarea to another server/workarea. This is valid for migrating from one server to another (MASTER to SLAVE, or from SLAVE_A to SLAVE_B, or from SLAVE to MASTER).
A sample scenario is that of one standard/basic migration that lasted for several hours with direct impact on users access for all these several hours, can now be accomplished with less than 5 (five) minutes real impact on user.
Standard Migration
The basic/standard IPBRICK (web interface) procedure consists in:
- PHASE 1 - change location information in database (IPBRICK.OS MASTER database, and system/LDAP database)
- PHASE 2 - move user's data files (personal files and maildir) from source to destination
Sometime this procedure may take a long time in "PHASE 2" (moving the data files from source to destination). In some reported scenarios this resulted in several hours per user. This depends on several factors, such as:
- size of user's data files
- size in bytes
- size in number of individual files
- performance of system infrastructure:
- overall performance of source server, storage/disk, network bandwidth
- overall performance of destination server, storage/disk, network bandwidth
Meanwhile during the migration process the user may found himself locked out - with no access to his/hers data files (personal files nor maildir).
This is the problem: impact on user's access to its own data files.
NEW - Assisted Migration
With this procedure we reduce the impact to the minimum time required to perform a simple file sync - less than 5 minutes per user.
Here we will be using a cheat procedure - user's data files migration will be done entirely in a manual procedure, we will ensure that the standard IPBRICK (web interface) will be in charge of migrating an empty folder (instantaneous migration).
Step by step description:
- ON-LINE - no impact on users access
- initial copy/synchronization of user's data files from real source user folder (source server / work-area / user folder) to temporary destination folder located at destination filesystem (destination server / work-area) - this may be a long process, may last for several hours but with no user is locked off
- intermediate synchronization - after finishing the first initial copy you may perform one or more regular intermediate synchronizations, you may register how long this intermediate synchronizations take, this is a very real measure of how long it will take to perform the last synchronization, this is the time the user will be kept off-line (impact on user)
- OFF-LINE - now it is time to perform the real migration, user will be locked off access to its data files
- last synchronization - rename real source folder to a temporary folder in the same source filesystem, perform last synchronization from this temporary source folder to the temporary destination folder - synchronization is now complete
- create an empty source folder for user
- go to IPBRICK web interface and instruct to move user to the select destination server and destination work area, Apply Configurations - Updated Successfuly
- go to destination server, destination work area, confirm that there is an empty folder for the selected user, remove that folder, rename previous temporary destination folder to the correct username
- ON-LINE - the procedure is now complete
Requirements
CAUTION / ATTENTION
This tutorial is provided "as is" for systems administrators, it requires strong knowledge of linux command-line environment. You must proceed with caution and at your own risk.
We recommend you to read this procedure in advance before you start any intervention in your server. If you don't feel comfortable with the procedures stated here, we recommend you to halt your intervention and opt to request official support from our technicians at support@ipbrick.com .
NOTE: You have been warned!
Backup
Before you start this process you must confirm that your backup is up to date. You will be messing around with your users data files, if something brokes or fails, the backup is your only best friend.
IPBRICK versions
This procedure is valid for IPBRICK.OS up to the current IPBRICK.OS version - at this time v6.3u1.
File System space availability
Please ensure that the destination filesystem has enough available space to accommodate all of the users data files.
Sample Procedure
NOTE: This tutorial is provided for systems administrators, it requires strong knowledge of linux command-line environment. One must proceed with caution.
Source / Destination
In this example for instruction demonstration we will assume the following:
- username: jack
- source
- server IP: 192.168.10.1
- work area: Work Area 1
- destination
- server IP: 192.168.10.2
- work area: Work Area 2
Folders
The folders that we will be working with are:
- REAL source user's account folder - located in source server
192.168.10.1:/home1/_accounts/jack - TEMPORARY destination folder - to be created in destination server
192.168.10.2:/home2/_accounts/_TEMP_jack - TEMPORARY source foldername - to be used only in the last synchronization in source server
192.168.10.1:/home1/_accounts/_DELETEME_jack - BLANK destination folder - to be used in destination server, only after last apply configurations :
192.168.10.2:/home2/_accounts/_BLANK_jack - TEMPORARY destination mount folder - to be used in destination server, only for temporary mounts for copy/synchronization :
192.168.10.2:/mnt/REAL_jack
Sync Mechanism
In this procedure, for matter of simplicity, we will be using NFS and RSYNC.
NOTE: You may use other advanced options such as RSYNC over SSH.
Detailed technical procedure
ON-LINE
The procedures in this section allows the user to continue normal access to its own files and maildir.
Initial/First Copy
Initial/first copy - only at DESTINATION SERVER 192.168.10.2
- create destination folders
~# mkdir /mnt/REAL_jack ~# mkdir /home2/_accounts/_TEMP_jack
- mount source folder at destination server mount point
~# mount 192.168.10.1:/home1/_accounts/jack /mnt/REAL_jack
- start first copy/synchronization
~# rsync -avu --delete /mnt/REAL_jack/ /home2/_accounts/_TEMP_jack/ > /root/jack_rsync.log 2> /root/jack_rsync.err
- clean up after first copy/synchronization
~# umount /mnt/REAL_jack/ ~# rmdir /mnt/REAL_jack/
Intermediate Synchronization(s)
After finishing the first initial copy you may perform one or more regular intermediate synchronizations, you may register how long this intermediate synchronizations take, this is a very real measure of how long it will take to perform the last synchronization, this is the time the user will be kept off-line (impact on user)
We continue to work only at DESTINATION SERVER 192.168.10.2
- preparing mount point
~# mkdir /mnt/REAL_jack
- mount source folder at destination server mount point
~# mount 192.168.10.1:/home1/_accounts/jack /mnt/REAL_jack
- start intermediate synchronization
~# date ; rsync -avu --delete /mnt/REAL_jack/ /home2/_accounts/_TEMP_jack/ > /root/jack_rsync.log 2> /root/jack_rsync.err ; date
- clean up after synchronization
~# umount /mnt/REAL_jack/ ~# rmdir /mnt/REAL_jack/
Please take notice of the time lapse in the rsync command. This is the duration of the intermediate synchronization. This is the estimate time for the final synchronization.
NOTE: You may perform this step multiple times before the final/last synchronization. We recommend you to perform at least one intermediate synchronization immediately before going to the next OFF-LINE step.
OFF-LINE
The procedures in this section require the user to be off-line, this is required to ensure that there is no data lost.
ATTENTION: this section includes procedures to be executed in the source and in the destination server.
User is put OFF-LINE
To execute at SOURCE SERVER : 192.168.10.1
- rename real source folder to a temporary folder
~# mv /home1/_accounts/jack /home1/_accounts/_DELETEME_jack
- create temporary source empty user folder
~# mkdir /home1/_accounts/jack
- list/confirm the empty folder (empty listing)
~# ls /home1/_accounts/jack
LAST Synchronization
To execute at DESTINATION SERVER : 192.168.10.2
- preparing mount point
~# mkdir /mnt/REAL_jack
- mount source folder at destination server mount point
~# mount 192.168.10.1:/home1/_accounts/_DELETEME_jack /mnt/REAL_jack
- start last synchronization
~# rsync -avu --delete /mnt/REAL_jack/ /home2/_accounts/_TEMP_jack/ > /root/jack_rsync.log 2> /root/jack_rsync.err
- clean up after synchronization
~# umount /mnt/REAL_jack/ ~# rmdir /mnt/REAL_jack/
Efective IPBRICK/System Migration
Go to IPBRICK web interface and instruct to move user (jack) to the select destination server (192.168.10.2) and destination work area (Work Area 2), Apply Configurations.
Updated Successfuly.
To execute at DESTINATION SERVER, 192.168.10.2
- confirm empty user folder successful migration (in the listing bellow you should get an empty folder)
~# ls /home2/_accounts/jack
- rename temporary empty folder to final blank folder
~# mv /home2/_accounts/jack /home2/_accounts/_BLANK_jack
- rename REAL destination user folder to the final name
~# mv /home2/_accounts/_TEMP_jack /home2/_accounts/jack
ON-LINE - Finish
The procedure is now complete. The user may now access to its own personal files and maildir.
Last Clean-up
After all validations its now time to perform global cleanup of temporary files/folders, and free space on the source file system/server.
- To execute at DESTINATION SERVER :
192.168.10.2- remove temporary empty folder
~# rmdir /home2/_accounts/_BLANK_jack
- To execute at SOURCE SERVER :
192.168.10.1- remove source copy of users folders/files- ATTENTION: this is a permanent deletion of users files, please proceed with extreme caution
~# rm -rf /home1/_accounts/_DELETEME_jack
