Cannot open ECP via On-Prem Exchange

Situation:

We had migrated all our employees to exchange online and decommissioned our on-prem exchange servers which was hosting user databases. Retained on-prem exchange server 2013 CU22 just for SMTP application relay purpose. Also have created a new on-prem database with 2 user mailbox. However, with that user we were not able to login to the ECP on-prem. 

No issues in login to the online Exchange Admin Centre. 

What errors do you see? 

:-( Something went wrong We can't get that information right now. Please try again later 

What's the environment and are there recent changes? 

Exchange server 2013 Cu22 on Windows 2012R2. 

Our emails (...domain) have been migrated to exchange online. 

We are using exchange on-prem for the application servers hosted on Azure to relay to onprem-exchange. Noticed that on-prem ECP wasn't accessible. Single database and two mailbox accounts on on-prem exchange. 

What have you tried to troubleshoot this? 

Verified on-prem database and 2 users accounts already available on on-prem exchange. but cannot login via https://localhost/ecp

Resolution:

You cannot access ECP, it is getting redirected to office 365 while accessing local host/ecp.

We checked the HTTP redirect on Default Frontend, there are no settings found.

We checked the HTTP redirect on ECP, we found no settings

We found HTTP redirect configured for OWA, redirected to office 365 portal

We unchecked the settings and were able to access ECP successfully.

How do I check my mailbox database mount status?

Quick Exchange (on-prem)  Tips:

How do I check my mailbox database mount status?

Get-MailboxDatabase with the -status

Shell to create a user mailbox

New-Mailbox -Alias onPrem1 -Name "Test onprem1" -FirstName onPrem -LastName onprem1 -DisplayName "Test onprem1" -UserPrincipalName onprem1@yourdomain.com -Password (ConvertTo-SecureString -String 'P@ssw0rd' -AsPlainText -Force)

Shell to create a existing user mailbox

Enable-Mailbox estherv@yourdomain.com -Database UsersMailboxDatabase

Enable-Mailbox ktadmin@yourdomain.com -Database MDB1

check results:

Get-Mailbox -Database MDB1

copy on-prem exchange mail flow receive connectors

how to copy on-prem exchange mail flow receive connectors to a new exchange server via powershell command

New-ReceiveConnector  -TransportRole FrontEnd "AppsDB Relay Connector" -Server **MEX01P -Bindings 0.0.0.0:25 -RemoteIPRanges ( Get-ReceiveConnector "**MV1MEX001\AppsDB Relay Connector" ).remoteIPRanges

Connectivity analyser

https://testconnectivity.microsoft.com/

Exchange Online: How to enable your tenant for modern authentication

The Office 365 tenant/resource host (Exchange Online, SharePoint Online and Skype for Business Online) will need to be configured to accept a modern authentication connection. Here is the per service state of modern authentication by default :

• Exchange Online - ON by default.
• SharePoint Online - ON by default.
• Skype for Business Online - ON by default.

Steps

This article explains how to enable your Exchange Online tenant to support modern authentication.

1. Connect to Exchange Online using remote PowerShell: refer here ventej.blogspot.com
2. Run the following command:
• Set-OrganizationConfig -OAuth2ClientProfileEnabled:$true
3. Verify that the change was successful by running the following:    
• Get-OrganizationConfig | ft name, *OAuth*

Ref: Microsoft article

Multiple credential prompts in the Outlook client

User Impact

Users may have received repeated credential prompts within the Outlook desktop client.

This issue only impacted customers using basic authentication. Customers were able to use Modern Authentication to mitigate impact for affected users; however, this process may have required several hours to take effect for some customers.

For some customers who have disabled Modern Authentication, there was a secondary issue that was causing the client to attempt to use Modern Authentication regardless of the setting.

Scope of Impact

This issue affected a subset of customers and users who were connecting to the service using basic authentication and utilized service-based search or a Focused inbox.

Incident Start Date and Time

Monday, May 4, 2020, at 7:00 AM UTC

Incident End Date and Time

Monday, May 11, 2020, at 5:05 AM UTC

Root Cause

A recent update to the Exchange Online service contained a code issue that caused repeated credential prompts for basic authentication users.

Glossary

Representation State Transfer (REST) – These APIs are service endpoints that support sets of HTTP operations, which provide, create, retrieve, update, or delete access to the service’s resources. More information can be found here - https://docs.microsoft.com/en-us/rest/api/azure/.

OAuth – Modern Authentication, an authentication protocol. More information can be found here - https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-v2-protocols.

Actions Taken (All times UTC)

Tuesday, May 5

4:00 AM – Telemetry indicates that this was when the deployment which triggered the regression was deployed. Support cases indicate credential prompts earlier than that, however, they may be isolated and not related to the regression exposed by this update.

1:43 PM – We received reports that customers were seeing credential prompts when trying to access Exchange Online using the Outlook client.  We started an investigation, but were unable to gather meaningful reproduction data.

Wednesday, May 6

3:28 PM – We determined that requests using OAuth 2.0 (Modern Authentication) were returning 401 errors, but that OAuth 2.0 wasn’t enabled.

6:50 PM – We concluded that the problem manifested in Representational State Transfer (REST). This service doesn’t support OAuth 2.0; with the Outlook client trying to use OAuth, credential prompts will appear.

7:20 PM – We reviewed recent changes to see if this may have disabled OAuth. We also discovered that this affects any user who used a service-based search or a Focused Inbox. Users with modern authentication wouldn’t have been affected.

11:38 PM – We found that the regression was exposed by a service update for Exchange Online. When a user makes an authentication request, the service is generating a value which is not correctly parsed, therefore not correctly validating OAuth 2.0 as expected. At this stage, the problematic change had reached 45% of customers worldwide. We stopped the change from reaching any more of the cloud, and started developing a fix.

Thursday, May 7

12:57 AM – We made sure that the fix was enabled in all build versions and updates, so that subsequent service updates would be streamlined with the fix, and prevent all future credential prompts. To make sure that the fix doesn’t impact the rest of the service, engineers estimated this would take approximately three days to complete development, testing and rollout.

2:21 AM – We finished developing the fix and started testing.

3:07 PM – We completed testing and prepared the fix for deployment.

7:39 PM – We started deploying the fix.

Friday, May 8

4:02 PM – The fix reached 33% completion.

Saturday, May 9

6:01 PM – The fix reached 86% completion.

Sunday, May 10

2:41 AM – The fix reached 92% completion.

Monday, May 11

5:05 AM – We confirmed that the fix had completed and declared the incident resolved.

Microsoft Next Steps

Findings	Action	Completion Date
A service update caused a regression which didn’t correctly validate OAuth 2.0 and caused users to experience repeated credential prompts under specific scenarios.	We're reviewing our update and validation procedures to prevent similar issues from reoccurring and to identify issues before initiating deployment.	June 2020
	We're reviewing our deployment and patching procedures to more quickly identify the source of impact and mitigate impact in a more timely fashion.
	We’re adding extra validation steps within our code to prevent this issue from happening again

Connect to Exchange Online - PowerShell Mode

Powershell Command to check whether Modern Authentication is enabled or not?

Connect to Exchange Online - using elevated PowerShell command run the following:

$UserCredential = Get-Credential

In the Windows PowerShell Credential Request dialog box, type your work or school account and password, and then click OK.

$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $UserCredential -Authentication Basic -AllowRedirection

Import-PSSession $Session -DisableNameChecking

Get-OrganizationConfig | ft name, *OAuth*

Name                     OAuth2ClientProfileEnabled

----                     --------------------------

****365.onmicrosoft.com                      False

Important Note

Be sure to disconnect the remote PowerShell session when you're finished. If you close the Windows PowerShell window without disconnecting the session, you could use up all the remote PowerShell sessions available to you, and you'll need to wait for the sessions to expire. To disconnect the remote PowerShell session, run the following command.

Remove-PSSession $Session

Connect to Exchange Online to check whether Modern Authentication is enabled or not?

Connect to Exchange Online to check whether Modern Authentication is enabled or not?

Microsoft is going to disable basic/legacy authentication for Exchange Online. What does that actually mean and does that impact me?

Update: On April 3rd, the Exchange Team announced that due to the COVID019 crisis, they will postpone disabling legacy authentication until the second half of 2021.

Update: On April 30, the Exchange Team announced that OAuth 2.0 authentication for IAMP and SMTP AUTH protocols is now available. In order to leverage this functionality mail clients need to start using it (so they need an update)

On March 7, 2018 the Microsoft Exchange Team announced that on October 13, 2020 it would stop the support for Basic Authentication (also called Legacy authentication) for Exchange Web Services (EWS) in Exchange Online (EXO), the version of Exchange offered as a service part of Office 365. EWS is a web service which can be used by client applications to access the EXO environment. The team also announced that EWS would not receive any feature updates anymore, and suggests customers to transition towards using Microsoft Graph to access EXO.

One and a half year later, on November 20, 2019 the Exchange Team also announced to stop supporting Basic Authentication for Exchange ActiveSync (EAS), Post Office Protocol (POP), Internet Message Access Protocol (IMAP) and Remote PowerShell on October 13 2020 as well. Authenticated Simple Mail Transfer Protocol (SMTP) will stay supported when used with Basic Authentication.

Instead of supporting Basic/Legacy authentication Microsoft will move towards only supporting Modern Authentication for most of the methods used to connect to Exchange Online.

So what is this Legacy/Basic and Modern Authentication exactly?

When using Basic/Legacy Authentication application sends a username and password with every request to Exchange Online which either forwards the credentials towards Azure AD or a federated authentication provider like Active Directory Federation Services (ADFS). The problem with Basic/Legacy authentication is that it’s vulnerable to brute force or password spray attacks.

Modern Authentication is based on OAuth 2.0 and the Active Directory Authentication Library (ADAL) providing token based authentication. OAuth 2.0 in this case is the protocol being used, and ADAL is used to authenticate against Azure AD.

Token based authentication, as described by the World Wide Web Consortium (W3C):

“The general concept behind a token-based authentication system is simple. Allow users to enter their username and password in order to obtain a token which allows them to fetch a specific resource – without using their username and password. Once their token has been obtained, the user can offer the token – which offers access to a specific resource for a time period – to the remote site. Using some form of authentication: a header, GET or POST request, or a cookie of some kind, the site can then determine what level of access the request in question should be afforded.“

So Microsoft wants you to move towards Modern Authentication since it’s a more secure solution. Another big advantage of using Modern Authentication is that it can leverage Azure AD Conditional Access, giving you the option to force MFA for users among other options. See my blogpost series on Azure AD Conditional Access for more information about implementing Conditional Access..

How can I check whether Modern Authentication is enabled or not?

If your tenant was created after August 1, 2017 Modern Authentication is turned on by default, that does not mean that in the meantime someone disabled Modern Authentication of course.

The easiest way to check if Modern Authentication is enabled is by looking in the Microsoft 365 Admin Center. Once in the  Admin Center go to Settings -> Settings – > Modern Authentication. Here you can check if Modern authentication is enabled.

Powershell Command to check whether Modern Authentication is enabled or not?

Connect to Exchange Online 

using elevated PowerShell command run the following:

$UserCredential = Get-Credential

In the Windows PowerShell Credential Request dialog box, type your work or school account and password, and then click OK.

$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $UserCredential -Authentication Basic -AllowRedirection

Import-PSSession $Session -DisableNameChecking

Get-OrganizationConfig | ft name, *OAuth*

Name                     OAuth2ClientProfileEnabled

----                     --------------------------

****365.onmicrosoft.com                      False

Important Note

Be sure to disconnect the remote PowerShell session when you're finished. If you close the Windows PowerShell window without disconnecting the session, you could use up all the remote PowerShell sessions available to you, and you'll need to wait for the sessions to expire. To disconnect the remote PowerShell session, run the following command.

Remove-PSSession $Session

Enable Automatic Replies for another user

Setting up Out-of-Office on behalf of another user

Via Exchange PowerShell

If you are an Exchange administrator, then using the Set-MailboxAutoReplyConfiguration Exchange PowerShell command is the supported and native way to go to enable Automatic Replies without logging on to the mailbox itself.

Syntax:

Set-MailboxAutoReplyConfiguration -Identity <username> -AutoReplyState Enabled -InternalMessage "Internal auto-reply message." -ExternalMessage "External auto-reply message."

tested in windows 10 client w office 2016, Exchange 2013 running on windows server 2012 R2 environment

Exchange 2013 Maintenance: Logs

Take a full backup. Once you take full backup all your logs will vanish. Else what?

 

Circular logging should not be enabled in a production environment. Circular logging are manually enabled only if you have noticed that the Exchange logs are enormously growing due to the following: either the nodes are unhealthy or there wasn't any successful exchange full backup.

 

To clear unwanted or orphaned logs, apart from enabling circular logging, it is also safe to dismount the Exchange DBs and mount back again. However, before doing it make sure to have a successful Exchange backup.

 

Exchange 2013:

 

Step1: ECP>>Servers>>Databases>>select required DB>>Check Active and Passive server's DB health. should be healthy. Double Click DB>>Maintenance>>Enable/Disable Circular logging. (Should not be enabled unless and otherwise required)

 

Step2: ECP>>Servers>>Databases>>select required DB>>Check Active and Passive server's DB health. should be healthy>>click more (3 dots...)>>Dismount. make sure the process completes successfully>> Mount the DB back again. And make sure the DB status shows mounted and healthy.

 

What if the log drive is full and exchange is down?

 

It is not a good idea to enable circular logging on a mailbox sever. As you may be aware that log drive should not be on a system drive, recommended to have a dedicated drive. Capacity planning is required for at least a period of 2-3 years both for SBM and large enterprise level org.

 

Consider moving the log files from the log drive to a temporary storage until the backup issue gets resolved. Once you have enough space, enable circular logging. Uncheck this feature after making sure there is enough log drive space.

 

In a nutshell circular logging recycles the logs.  Exchange relies on transaction or write-ahead logs to store events before they are committed to the database.  When the defined logs have been filled up, circular logging assumes that the first log must have been committed and recycles the logs to save disk space.

 

Granting Calendar Access via Exchange Control Panel (ECP)

Ex: Add-MailboxFolderPermission -identity "Managingdirector:\Calendar" -user "personalassistant" -AccessRights Reviewer

 

To change permissions on a Calendar item, type the following command:

 

This command will grant Calendar access, with level reviewer to his Personal Assitant. You can use Editor instead of Reviewer for more access.

 

If Managing director had PA in the past, you may want to remove that access. To that you can use the following command:

 

Ex: Remove-MailboxFolderPermission -identity "ManagingDirector:\Calendar" -user "oldPA"

Experiencing difficulty in adding profile photo of employees

Basically, In active Directory we can use images up to 96×96 pixels in resolution. But in exchange 2013 you can use high resolution photo for the users which will be stored in user mailbox. This will allow you to upload photos up to 648x648 pixels.

If you dont have Photoshop or alternate software to edit pictures, don't worry this can be done online as well via http://resizeimage.net/

Requirements:

- Photo should be in format *.jpg.

- Photo should be in square shape. simply, Hight=Width.

Setting User Photos:

Two ways:

Users can upload their own photos through OWA >> Click on Display name (up right) >> Change Photo
Administrator can import photots in Exchange by using command below:
Single line command:

Set-UserPhoto “a.hany” -PictureData ([System.IO.File]::ReadAllBytes(“C:\UsersPhotos\a.hany96x96.jpg”)) –Confirm:$False

You may also set photo or remove profile photo by the following methods:

Set-UserPhoto -Identity "Andrew Hany" -PictureData ([System.IO.File]::ReadAllBytes("C:\Photo\Photo_Outlook\a.hany_96x96.jpg"))

This example uploads and saves a photo to Andrew Hany's user account using a single command.

Set-UserPhoto -Identity "Andrew Hany" -PictureData ([System.IO.File]::ReadAllBytes("C:\Photo\Photo_Outlook\a.hany_96x96.jpg")) -Preview; 

Set-UserPhoto "Andrew Hany" -Save

This example shows how to use two commands to upload and save a preview photo to Andrew Hany's user account. The first command uploads a preview photo to Andrew Hany's user account, and the second command saves the uploaded photo as the preview photo.

Set-UserPhoto -Identity "Andrew Hany" -Cancel

This example deletes the preview photo that was uploaded in the previous example.

Reviewing User Photos:

access the following URL to check

https://mail.contoso.com/ews/Exchange.asmx/s/GetUserPhoto?email=a.hany@contoso.com&size=HR648x648

Note: You need to change highlighted parts according to your environment.

Outlook crashes with error message "Only one version of Outlook can run at a time"

Check if multiple process of Outlook.exe is running in the task manager.
· Press Ctrl+Alt+Del, and then chose Task Manager.
· Under processes tab look for Outlook.
· If you find the Outlook process running, right click on it and end the process.
If you do not see multiple process running in the background, then try opening Outlook in safe mode and check the result:
·         Hold Windows key + R.
·         Copy and paste, or type the following command in the Open box, and then press Enter:
Outlook /safe
Note: There is space between Outlook and /.