Problem: In our hybrid Exchange 2010 / Exchange Online environment, a user somehow ends up with a mailbox in both Exchange orgs. After some background on our environment, I'll explain how this problem occurred, and the solution I developed to resolve it.
Background: DOMAIN1 is the domestic (U.S.) domain, and has both users and the on-premises Exchange 2010 servers. DOMAIN2 exists due to a merger, is in a separate forest, but there's a two-way trust between the forests ('cause we're all one big happy family). Azure AD Connect imports accounts from both forests, then exports accounts to a single O365 E5 tenant. Licensing for the O365 features is group-based, using global security groups that are mapped to license options in Azure AD. Not all employees are provisioned with a mailbox, so the Exchange feature license is controlled by its own AD security group.
When all mailboxes were in Exchange 2010, each DOMAIN1 user was provisioned with a regular User Mailbox. Users in DOMAIN2 had to be provisioned with a Linked Mailbox. For those in the audience that haven't experienced Exchange in a multi-forest environment before, it's important to understand that only objects in the same forest as the Exchange org can be mail-enabled. In order to provision a mailbox for a user in another forest, you have to create a Linked Mailbox, which involves creating a new, disabled user account in DOMAIN1, populate all the necessary Exchange attributes on that account, and then give the user in DOMAIN2 permission to use the DOMAIN1 user mailbox as if it were his/her own. The New-Mailbox command, available in both the Exchange Admin Shell and in the Exchange Admin Console, makes creating linked mailboxes a breeze. But it's very important to understand that, in order to have a mailbox in Exchange, every user in DOMAIN2 will end up having TWO accounts - their normal login account in DOMAIN2 and the disabled mailbox account in DOMAIN1. No Exchange attributes get written to the account in DOMAIN2. In fact, provisioning a linked mailbox makes no changes to the account in DOMAIN2 at all. If you review every attribute of the DOMAIN2 account, there is absolutely no indication that this account has a mailbox.
As we began our journey to O365 and Exchange Online, it was nice to discover that, for DOMAIN2 users with linked mailboxes, as Azure AD Connect imports their accounts from both forests, it will recognize and intelligently merge their primary user account data from DOMAIN2 with the Exchange attributes from their account in DOMAIN1, and then export that combined information to create a single account in Azure/O365 containing all the necessary attributes.
Those last two paragraphs may seem to have gone a bit off-topic, but I described all of that just to help you understand that this is still how we're provisioning mailboxes today, even though everyone has been migrated to Exchange Online. So, this is how we currently provision mailboxes for new hires:
- For new users in DOMAIN1, we simply run the Enable-RemoteMailbox command in the on-premises Exchange org, which stamps the account with all the necessary attributes. The account is also added to the AD security group that is used to assign the Exchange Online feature license, so once the account syncs up into Azure and Exchange, a mailbox will be created and the feature license assigned, automatically. Simple and easy, works every time.
- For users in DOMAIN2, we can't just enable a remote mailbox since Exchange can't see the user accounts in DOMAIN2, so as before, we have to create a linked mailbox in the on-premises Exchange org, wait for Azure AD Connect to merge those accounts and export the changes to Azure, and then migrate the linked mailbox to Exchange Online. The account in DOMAIN2 also needs to be added to the AD security group that is used to assign the Exchange Online feature license, and it is the timing of this license assignment for linked mailbox users that can cause the issue that is the subject of this article.
Scenario: A help desk technician creates a new user account in
DOMAIN2 and immediately adds that account to the O365 Exchange Online security group in Active Directory. Within 30 minutes, that account is imported and synced up into Azure AD by Azure AD Connect, and no
further action is taken for several hours or longer. Since the account
has no Exchange attributes but has been assigned an Exchange feature license,
Exchange Online just goes right ahead and creates a mailbox for the user. Later (sometimes much later), the help desk receives a request to
create a mailbox for this user, and so they proceed (correctly) to create a new linked
mailbox in the on-premises Exchange org. The on-premises Exchange server allows
the mailbox to be created, since it has no knowledge of the mailbox that Exchange Online had already created.
Side effects: As described previously, in creating the linked mailbox, a
disabled DOMAIN1 account was created and populated with all the usual Exchange attributes. Azure AD Connect correctly sees this
account, sees that it is linked to the DOMAIN2 account, correctly merges the
two accounts together and exports the now-modified account data up into Azure
AD. The user's Exchange Online mailbox is updated with all the Exchange attributes that
were stamped on the user's DOMAIN1 account during the linked mailbox creation,
so the Exchange Online mailbox now has all the proper details of a full-blown company mailbox. But...
What's the problem? Well, we still have that linked
mailbox that was created back in the on-premises Exchange environment. And
that's a big problem because both the linked mailbox and the cloud mailbox have
the same SMTP addresses on them (with a few exceptions), and if we leave both of them as-is, any mail
received by the on-premises Exchange org will be delivered to the on-premises mailbox
(because that's the only one that the on-premises Exchange server knows about), and any
mail received by Exchange Online will be delivered to the Exchange Online mailbox (ditto)
- and the user, who is probably only aware of the Exchange Online mailbox, will have no idea why
they're getting some emails but not others.
So, how do we fix this? In the recent past, I would simply delete the mistakenly-created Exchange Online mailbox and then migrate the linked mailbox, which didn't always go smoothly. Plus, that's a lot of work, so I started looking for a simpler, more reliable alternative. Since we ultimately want to
keep the Exchange Online mailbox (which may have been receiving important mail by now) and it's already configured correctly, thanks to Azure AD
Connect, I realized that all I really needed to do was to get rid of the linked mailbox and make the on-premises Exchange org aware of the Exchange Online mailbox's presence. The solution I came up with is to modify the Exchange properties on the disabled DOMAIN1 account, by either adding, modifying, or deleting the attributes that control how the on-premises Exchange org "sees" that user object.
This solution requires quite a few attribute changes, and after modifying five or six accounts by bringing up another account and comparing attribute values, it quickly became clear that the manual process was far too labor-intensive and prone to mistakes. It's too easy to forget which attributes need to be modified, and copying values from one account to another is just so boring and slow. So now I've collected all of those attributes into an easily repeatable powershell script. We'll use the Set-ADUser command to make our changes all at once, but we need to do a bit of set up before
we can run it.
Caveat: Please consider the attributes and values you see below as specific to my company's Exchange environment. While it is quite likely that your org uses many of the same attributes and possibly even the same attribute values, I strongly urge you to compare these with those on your user accounts and make adjustments as needed before you attempt to run this in your environment. I can only confirm that what follows below seems to work well for our environment.
There's only one attribute that is user-specific, and that's
the targetAddress, which is what the on-premises Exchange org uses to route the user's mail
up to their Exchange Online mailbox. So our first step is to set a variable to the samaccountname of the user's disabled DOMAIN1 account. We'll use that to retrieve this account's mailNickname
(which was set when the linked mailbox was created), then build the targetAddress.
$aduser = ""
$target
= "SMTP:" + (get-aduser $aduser -Properties
mailnickname).mailnickname + "@COMPANY.mail.onmicrosoft.com"
Some attributes don't exist on this user's account yet, so
we'll have to add them. These attributes have the same values for all remote mailboxes, so we'll create a hash table with all the proper values hard-coded
right in. Note that the $target variable is used in this hash table, to set the targetAddress.
$addhash
= @{ `
msExchAddressBookFlags=1; `
msExchArchiveQuota=52428800; `
msExchArchiveWarnQuota=47185920; `
msExchBypassAudit=$false; `
msExchMailboxAuditEnable=$false; `
msExchMailboxAuditLogAgeLimit=7776000; `
msExchMDBRulesQuota=64; `
msExchModerationFlags=6; `
msExchProvisioningFlags=0; `
msExchRemoteRecipientType=4; `
msExchTransportRecipientSettingsFlags=0; `
targetAddress=$target
}
A couple more attributes already exist but need to be changed, again
with values that are common to all remote user mailboxes, so we'll create another
hash table for those.
$replhash
= @{ `
msExchRecipientDisplayType=-2147483642; `
msExchRecipientTypeDetails=2147483648
}
Finally, there are half a dozen attributes that just need to disappear (be cleared), so we'll create a variable containing an array of those
attribute names.
$clearargs
= `
'homeMDB', `
'homeMTA', `
'mDBUseDefaults', `
'msExchMailboxSecurityDescriptor', `
'msExchHomeServerName', `
'msExchRBACPolicyLink'
That's it for the prep work, so now we can run the
command.
Set-ADUser
$aduser `
-Add $addhash `
-Replace $replhash `
-Clear $clearargs
If your account does not have rights to make these changes
in Active Directory, you can specify alternate credentials:
Set-ADUser
$aduser `
-Add $addhash `
-Replace $replhash `
-Clear $clearargs `
-Credential
(Get-Credential)
Assuming the changes apply without error, there will be no
output. And if you go back into the on-premises Exchange console, you will
see that the linked mailbox object has disappeared, replaced with the familiar Remote
User Mailbox object. And once all those changes have synced up into Azure and Exchange Online, everything will work as expected.
Troubleshooting: If something goes wrong and you
don't see the Remote User Mailbox object in the Exchange console… first,
try refreshing the listing again (remember, it'll be in the Mail Contact node).
Exchange could be using a different domain controller than the one that
processed your Set-ADUser command, so allow a few minutes for AD replication then refresh the list again. If all else fails, pull
up the user's DOMAIN1 account side-by-side with another, working DOMAIN1 account,
go to the Attribute Editor tab of both accounts and compare the settings. They
won't all match, so just concentrate on the attributes listed in the above hashtables.
Ciao!