Office 365 – Distribution List Migrations Version 2.0 – Part 26

New handling of nested distribution groups in multiple distribution group migrations…

A question that is often asked is how to handle nested distribution list when considering a migration. When performing a single distribution list migration a recipient normalization process scans each member for attributes and object classes. If a group object class is found and is mail enabled the single group migration fails with an exception. The administrator must either first migrate the child distribution group or remove the mail enabled group. Here is a sample exception:

 

ParentGroupSMTPAddress : atestgroup27@domain.com

Alias : aTestGroup270

RecipientOrUser : Recipient

isError : True

isErrorMessage : NestedGroupException – A mail enabled group is a child member of the migrated list. The

child group must be migrated first or removed from group membership.

Name : aTestGroup270

GUID :

OnPremADAttribute : member

GroupType : -2147483640

isAlreadyMigrated : False

ExternalDirectoryObjectID :

DN : CN=aTestGroup270,OU=MigrationTest,OU=DLConversion,DC=home,DC=domain,DC=com

RecipientType :

OnPremADAttributeCommonName : Member

PrimarySMTPAddressOrUPN : aTestGroup270@domain.com

 

For customers that were evaluating options of migration distribution lists in bulk my original advice was to generate the list of primary SMTP addresses for migration. This list would then be run through the multiple migration process. In the end any group that could be migrated was migrated and any group left had an exception. The same set of addresses would be processed multiple times until no more migrations completed. The lists that were left required administrative intervention. The process overall was highly inefficient and sorting through the errors could be complicated. For some customers it was easier to iterate through the list rather than try to outline the dependencies and perform the migrations in order.

For customers that were able to map nested group membership the addresses would be sorted in correct order. The multiple migration process was called on the list and as long as there were no overlap in how the addresses were batched into groups of 5 the migrations would proceed successfully. Nested DLs were migrated first and the parent DLs later in the batch. If there was any overlap, for example a parent and child included in the same batch of simultaneous migrations, the parent would fail with the exception above.

For a review of how the module tracks nested membership and groups that have already been migrated see the following post. Office 365 – Distribution List Migrations Version 2.0 – Part 6 | TIMMCMIC (wordpress.com)

In version 2.9.5 new code is implemented in start-multipleDistributionListMigration that no longer requires the administrator to iterate through the list multiple times or pre-sort the list to account for nesting. The new code implements the following process:

• Iterate through the list and attempt to migrate each distribution list specified.
• If the start-DistributionListMigration encounters a nested group exception add the groups normalized information to a CSV file.
• At the end of the first pass start-MultipleDistributionListMigration tests for the csv file presence.
  • If the CSV file is present:
    • The recipient objects are imported.
    • The child distribution list primary SMTP address is searched in the array of recipients initially migrated.
    • If the child primary SMTP address is found the parent list is added again for migration.
    • If the child primary SMTP address is not found the parent is ignore with a permanent exception. The child was not originally included in the migration set.
    • Scan all groups to ensure there are no cross dependencies. If a cross dependency is found fail the group migration.
  • If the CSV file is not present the migration is complete.
• If the recipient array is populated with parent group SMTP addresses the multiple list migration is restarted using the concatenated list.
• The process continues to repeat until there are no more groups to retry.

   

Let examine a practical example of this using the following structure.

 

The array of SMTP addresses that the administrator specified for migration:

GroupA

GroupZ

GroupB

GroupC

GroupB1

GroupC1

GroupC2

GroupC3

GroupC4

GroupX

GroupY

 

The start-MultipleDistributionListMigration code breaks the groups down into the maximum number of threads a single host can migrate. The batch size is 5 groups per batch. The first batch of groups for migration includes:

 

GroupA

GroupZ

GroupB

GroupC

GroupB1

 

At the end of the first batch of migrations GroupB1 migrates successfully as it has no child group dependencies. GroupA, GroupZ, GroupB, and GroupC all fail with nested child groups. These groups are added to the retry list.

 

The next batch of groups includes:

 

GroupC1

GroupC2

GroupC3

GroupC4

GroupX

 

At the end of the second batch of migrations GroupC1, GroupC2, GroupC3, and GroupC4 all migrate successfully as they have no child group dependencies. GroupX fails with a nested child group exception and is added to the retry list.

 

The final batch is a single group – GroupY. GroupY will fail to migrate with a nested child group exception and will be added to the retry list.

 

At the conclusion of all first attempts the retry list contains the following groups:

 

GroupA

GroupZ

GroupB

GroupC

GroupX

GroupY

 

The groups are now scanned to determine if they are eligible for retry.

 

The first can searches the list for circular dependencies. A circular dependency is when two groups are members of each other. In this instance GroupX is a member of GroupY and GroupY is a member of GroupX. This is a cross dependency that cannot be reconciled. These groups hard fail and are added to the failure list for manual administrator intervention.

 

The second scan searches the list of exceptions. Each exception contains the ChildDL primary SMTP address and the ParentDL primary SMTP address. The list of original SMTP addresses is scanned and if the child groups primary SMTP address was present in the list to be migrated then the group is added for retry. Here is an example using our calculated retry list.

 

GroupA@domain.com -> GroupB@domain.com (GroupB@domain.com was originally included add GroupA to the retry list).

GroupA@domain.com -> GroupC@domain.com (GroupC@domain.com was originally included add GroupA to the retry list).

GroupZ@domain.com -> GroupZ1@domain.com (GroupZ1@domain.com was NOT originally included. GroupZ will fail and be added to the failure list for manual administrator intervention).

GroupB@domain.com -> GroupB1@domain.com (GroupB1@domain.com was originally included add GroupB to the retry list).

GroupC@domain.com -> GroupC1@domain.com (GroupC1@domain.com was originally included add GroupC to the retry list).

GroupC@domain.com -> GroupC2@domain.com (GroupC2@domain.com was originally included add GroupC to the retry list).

GroupC@domain.com -> GroupC3@domain.com (GroupC3@domain.com was originally included add GroupC to the retry list).

GroupC@domain.com -> GroupC4@domain.com (GroupC4@domain.com was originally included add GroupC to the retry list).

 

At the end of child validation scanning the list is as follows:

 

GroupA

GroupA

GroupB

GroupC

GroupC

GroupC

GroupC

 

The results are trimmed into a unique list to retry:

 

GroupA

GroupB

GroupC

 

The code then retries the migration using the updated list of addresses to retry. The list contains less than 5 members only a single batch is created. At the conclusion of this second attempt GroupB and GroupC migrate successfully as there are no nested child lists in the group any longer. GroupA fails since at the time of the retry GroupB and GroupC were both members. The process now repeats itself using the new nested exception list.

 

GroupA@domain.com -> GroupB@domain.com (GroupB@domain.com was originally included add GroupA to the retry list).

GroupA@domain.com -> GroupC@domain.com (GroupC@domain.com was originally included add GroupA to the retry list).

 

This results in the following retry list:

 

GroupA

GroupA

 

The results are then trimmed into a unique list to retry.

 

The code then retries the migration using the updated addresses to retry. The list in this case contains only one address so a single batch is created. At the conclusion GroupA now migrates successfully as there are no nested child group dependencies.

 

The migration is concluded at this time.

 

GroupZ will not migrate due to a child group not included in the migration set. Here is an example of an exception for this reason:

 

[1/6/2023 12:20:56 AM] – =====

[1/6/2023 12:20:56 AM] – Alias: a300

[1/6/2023 12:20:56 AM] – Name: a300

[1/6/2023 12:20:56 AM] – PrimarySMTPAddressOrUPN: a300@domain.com

[1/6/2023 12:20:56 AM] – RecipientType: group

[1/6/2023 12:20:56 AM] – GroupType: -2147483640

[1/6/2023 12:20:56 AM] – RecipientOrUser: Recipient

[1/6/2023 12:20:56 AM] – ExternalDirectoryObjectID:

[1/6/2023 12:20:56 AM] – OnPremADAttribute: member

[1/6/2023 12:20:56 AM] – DN: CN=a300,OU=DoNotSync,OU=MigrationTest,OU=DLConversion,DC=home,DC=domain,DC=com

[1/6/2023 12:20:56 AM] – ParentGroupSMTPAddress: aTopZ@domain.com

[1/6/2023 12:20:56 AM] – isAlreadyMigrated: False

[1/6/2023 12:20:56 AM] – isError: True

[1/6/2023 12:20:56 AM] – isErrorMessage: ChildGroupMirationException: The groupt to be migrated has a child group not included in the migration set.

[1/6/2023 12:20:56 AM] – =====

 

GroupX and GroupY will fail to migrate due to a circular dependency. Here is an example of an exception for this reason:

 

[1/6/2023 12:20:56 AM] – =====

[1/6/2023 12:20:56 AM] – Alias: aTopY

[1/6/2023 12:20:56 AM] – Name: aTopY

[1/6/2023 12:20:56 AM] – PrimarySMTPAddressOrUPN: aTopY@domain.com

[1/6/2023 12:20:56 AM] – RecipientType: group

[1/6/2023 12:20:56 AM] – GroupType: -2147483640

[1/6/2023 12:20:56 AM] – RecipientOrUser: Recipient

[1/6/2023 12:20:56 AM] – ExternalDirectoryObjectID:

[1/6/2023 12:20:56 AM] – OnPremADAttribute: member

[1/6/2023 12:20:56 AM] – DN: CN=aTopY,OU=RecursionTest,OU=MigrationTest,OU=DLConversion,DC=home,DC=domain,DC=com

[1/6/2023 12:20:56 AM] – ParentGroupSMTPAddress: aTopX@domain.com

[1/6/2023 12:20:56 AM] – isAlreadyMigrated: False

[1/6/2023 12:20:56 AM] – isError: True

[1/6/2023 12:20:56 AM] – isErrorMessage: CircularReferenceException: This group has a child distribution list that also has this group as a member. This creates a circular dependency which cannot be handeled automatically.

[1/6/2023 12:20:56 AM] – =====

 

What happens if a group fails to migrate for another reason? If a child group fails to migrate due to another error besides a nested group exception the group is not written to the CSV file. This means that when the pass is completed and the CSV file is analyzed – the child DL will no longer be found as included in the migration set. This yields an error that the child group was not present in the migration set. It is quite confusing because it was included in the migration set but unfortunately errored for another reason.

I hope this change allows mass migrations to proceed with greater success and less administrator intervention. Happy migrating!