Office 365 – Distribution List Migration Version 2.0 – Part 8

Over the last several weeks, I have had the pleasure of speaking with multiple customers that have tested and deployed the Distribution List Migration Module V2. The feedback I received has been very helpful and based on it, I’ve made several changes and improvements to the module. I want to take an opportunity to explore some of the changes I made.

Improved migration experience for versions of Exchange older than Exchange 2016

The Exchange 2016 schema added some attributes to Active Directory that may be stamped on distribution lists. These attributes do not exist in the Exchange 2010 or Exchange 2013 schema. My original code defined the group of Active Directory attributes to be cleared based on Exchange 2016. For older versions of Exchange (or when an Exchange server was specified for use during the migration) distribution lists would be disabled using Disable-DistributionGroup. This proved on several occasions to be unreliable and resulted in a failure to disable the distribution group. New code was introduced that scans Active Directory for the schema version of Exchange in use – and adjusts the Active Directory attributes to be cleared accordingly. To ensure a more reliable experience disabling the distribution group, Active Directory PowerShell commands using the appropriate Exchange version attributes are used to disable the group.

Elimination of pro-active sleeps in response to occasional timing failures

During testing, it was noted that attempts to gather information for newly created objects would often fail when the object was created and then immediately queried. For example, when the new distribution list is created in Office 365, the code would immediately search for it, and in almost all circumstances, it would not be found, resulting in an error. Earlier versions of the module handled this by implementing proactive sleeps in between the creation and recording functions. In some cases, the sleep was not long enough and in other cases it was unnecessary, but either way they increased the overall migration time. The sleeps have been eliminated and replaced with a retry logic that allows the migration to immediately proceed if the sleep is not required.

Enhanced retry logic to handle expected failures and retry operations before hard failing the migration

New logic was introduced surrounding main functions within the script to allow the script to retry operations in the event a single function call fails. For example, if the script tries to locate an object in the directory immediately after creation and fails, a sleep is invoked. The script then tries for a pre-designated number of times to retry the operation before considering the operation as failed. This has improved the migration efficiency and success in larger environments where there may be additional factors outside the migration that caused commands to fail.

Improved sleep status

When a sleep was invoked a single line was written to the screen. Depending on the duration this could give the appearance that the script was hung or not proceeding. A new sleep function was introduced to provide a count down of the sleep operation so that visual feedback is available to the administrator.

 

Improved handling of multi-valued attributes

There are several multi-valued attributes that exist on a distribution group. For example, membership, accept messages from, and reject messages from. In older versions of the module, logic was used to step through each individual member and add them to their respective multi-valued attribute. For example, if there were 100 members, each member was evaluated individually and added to the group. New logic was introduced in newer versions that pre-creates the arrays of users to add and then adds them to their respective multi-valued attributes in a single operation. This improves the efficiency of the script. It also acts as a reset operation for the value, so when the group is created and the administrator creating it is automatically added as a member or manager, these are overwritten with only the values that existed on the on-premises group.

Improved memory utilization

In order to perform evaluations and recording of object permissions such as Send As and Full Mailbox Access the script has to discover and record all recipients either on-premises or in Office 365. Older versions of the module used large get commands to record the set of objects for evaluation by other functions. PowerShell by default returns all attributes with a given object which resulted in a significant amount of overhead for recording recipients, even though all that was required for future evaluation as the recipients identity and primary SMTP address. The module continues to utilize broad get commands but scopes the objects return values to just those necessary for further evaluation. By comparison, in older versions of the module for 10,000 Exchange Online recipients, a single migration took roughly 1.5 GB of memory. With the scoped changes, the same 10,000 recipients now consume only around 300 MB of memory for a single migration.

Improved status for collection and audit commands

The PowerShell module contains several functions to allow administrators to pre-audit data that is expensive to get during an interactive migration. For example, collecting all Send As permissions for all objects in Office 365 and storing them for evaluation during each distribution list migration. The collection commands use the PowerShell status function to keep the administrator notified of progress. The status function now includes the recipient number currently being processed, as well as the total number of expected recipients to process in the notification (e.g., “processing mailbox 100 of 499“). This small improvement allows for a better understanding of collection status.

Improved performance for collection and audit commands

The collection and audit commands are documented and expected to be long-running commands due to the nature of evaluating each object. Performance improvements were introduced to the Exchange Online Management Shell v2 command that’s used to do work that takes advantage of root hints. This prevents the command endpoint from having to proxy the command to another location. This has produced almost a fourfold improvement in data collection and resulted in no retry operations for the duration of the collection. Processing around 13,000 Exchange Online recipients for mailbox folder permissions now takes approximately 19 hours; in previous releases, it took over two days.

New handling for dynamic distribution groups

As with standard distribution groups, dynamic distribution groups in both Exchange Online and Exchange on-premises can have restrictions set on them. For example, accept messages from, reject messages from, and managed by. Updated versions of the module now evaluate dynamic distribution groups in both environments and track any dependencies that need to be replaced by the migrated distribution group.