Announcing multiple migration machine support
Over the course of its development, the Distribution List Migration v2 has continued to add support for more than one distribution list operation. In version 2.4, these capabilities were extended from running a bulk migration on a single machine to running a bulk migration across multiple machines.
The purpose of migrating distribution lists across multiple machines allows us to:
- Use more than one authentication account for Office 365 operations (eliminates throttling concerns);
- Use more than one authentication account for on-premises operations including Active Directory and Exchange;
- Have up to 25 simultaneous migrations running at a single time; and
- Centrally track migrations happening across multiple migration endpoints.
The module accomplishes multiple machine migrations by leveraging the single host bulk migration process previously released. The administrator designates a single machine to execute the migration. If this machine is also included in the list of migration endpoints, multiple migrations will be performed on that host. If the server executing the migrations is not included on the list of machines to execute migrations, the machine acts as a controller and remotely creates the multiple migration jobs on the remote hosts. The script supports up to 5 hosts performing the migrations (in the event that a controller is used this does not count as a migration host).
To support these types of migrations, the script makes several changes to the commands used to invoke the script. In this section I’ll review the changes and then look at a sample command.
-
Array of group is supplied to the groups rather than a single email address.
When invoking a multi-machine migration, the assumption is that multiple groups will be migrated. The group SMTP addresses allows the administrator to provide an array of SMTP addresses for migration. The list of addresses is divided evenly between each host with any extra being added to the last host used for migrations. In this example 202 smtp addresses are provided in the array of addresses. If the administrator specifies 5 hosts then 40 migrations are scheduled on hosts 1-4 with host 5 having 42 migrations scheduled.
To create the list of groups I recommend using a text file and listing the groups of addresses to be migrated. This text file can then be imported into a variable for use in the command.
Sample Group Text File saved as c:\info\groups.txt
DistributionList1@contoso.com
DistributionList2@contoso.com
DistributionList3@contoso.com
…
DistributionList105@contoso.com
$groups = get-content c:\info\groups.txt
-
Array of migration hosts is supplied. This list represents the machines that will perform the migration tasks. The list has the fully qualified domain name of each host. Each host is required to have the same pre-requisites as a single machine. Please review the single machine migration prerequisites. To create the list of migration hosts I recommend using a text file and listing the fully qualified domain names on each line. This test file can then be imported into a variable for use in the command.
Sample Group Text File save as c:\info\machines.txt
Azure-Mig-0.domain.com
Azure-Mig-1.domain.com
Azure-Mig-2.domain.com
Azure-Mig-3.domain.com
Azure-Mig-4.domain.com
$machines = get-content c:\info\machines.txt
-
Each set of credentials is now an array of credentials. The administrator may specify one or more credentials to be used during the migration. It is a requirement that a credential be supplied for each migration host even if the same credential is used across all hosts. If using more than 3 migration machines multiple Exchange Online credentials will be required in order to avoid PowerShell throttling. In testing multiple credentials were not required for Active Directory, Azure AD Connect, or Exchange on-premises but the script does allow for these. Specific testing should occur to ensure that Active Directory Web Services which is used for the module to perform remote Active Directory calls is not overloaded when using a single account. If Active Directory timeouts or errors occur in the logs consider using more than one administrator.
The following example shows how multiple credentials may be captured.
$creds=@() #Create empty array of credentials
$creds+=get-credential #Request a set of credentials interactively. Repeat once for each host. Specify the same or different credentials
The following example shows how multiple credentials may be captured from credentials stored within XML files on the migration controller.
$creds=@() #Create empty array of credentials
$cred1 = import-clixml c:\info\cred1.xml
$cred2 = import-clixml c:\info\cred2.xml
$cred3 = import-clixml c:\info\cred3.xml
$creds+=$cred1
$creds+=$cred2
$creds+=$cred3
As a reminder it is required to specify a credentials array that contains one set of credentials for each migration host even if the credentials are the same.
-
The administrator specifies a remote drive letter to use for centralized log file storage.
On the migration controller the log directory is automatically shared out. Permissions are restricted to each account specified in the credentials array for Active Directory and each hosts machine account performing migrations. The shared directory serves as the centralized logging location for all of the hosts performing migrations. Each host creates an individual folder in this directory. On the migration hosts themselves the specified drive letter is automatically mapped on the host to the centralized share. The log directory is updated to force logging operations to occur on the network share. This allows information for multiple migrations to be aggregated in a single location for success and error analysis. The mapped drive is removed at the conclusion of the migration operation. The share on the migration controller is maintained.
-
If the administrator has selected to pre-collect data such as folder permissions or send as rights, the pre-collection files must be in the AuditData folder in the logging directory of the migration controller. For example, if c:\temp is used as the log folder patch on the migration controller the AuditData folder must be at c:\temp\AuditData. The script automatically creates a sub directory for each migration host and copies the AuditData folder to it to avoid file contention when attempting to import the data for analysis.
An administrator may invoke a migration by using the sample command:
Start-MultipleMachineDistributionListMigration -groupSMTPAddresses $groups -globalCatalogServer GC.domain.com -activeDirectoryCredential $creds -aadconnectServer adconnect.domain.com -aadConnectCredential $creds -exchangeServer webmail.domain.com -exchangeCredential $creds -exchangeOnlineCredential $creds -logFolderPath c:\temp -remoteDriveLetter S -servers $machines -enableHybridMailFlow:$TRUE
The following shows a sample flow diagram of multiple machine migrations.
The logs for the controller are stored on the migration controller in the log file directory. The controller log keeps the summary of the PowerShell jobs dispatched and at the end of the migration will scan all individual migration folders for success or failure. The administrator may then review the individual migration logs contained in each servers directory to troubleshoot any issues.
Happy migrating!
TIMMCMIC 