cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Fille Access Manager Exchange Server Connections

Fille Access Manager Exchange Server Connections

    Exchange Server Management - Concepts

    Windows Servers have a remote management interface called the WinRM service that listens for connections.  Remote PowerShell connections to servers connect to the WinRM service on the target server.  The default WinRM port is not always available from all networks and is probably not optimal for some use cases on Exchange Server.  Instead, you can connect to the Exchange Server via web services that proxy commands to the WinRM service.  This web service is a virtual application named PowerShell that is hosted alongside Outlook Web Access (OWA) and other Exchange web applications in the same Internet Information Server website (IIS).  When troubleshooting connections to Exchange Server there are several things to consider beyond the pre-requisites in the File Access Manager documentation:

    • http/https and the port
    • IIS authentication
    • Certificates
    • Exchange Server architecture
    • Exchange Server problems

     

    How does File Access Manager connect to Exchange?

    The entitlement collector (EC) and activity monitor get information from Exchange by sending web service requests to 2 separate virtual applications on the Exchange web site.

     

    /PowerShell Virtual Application - This virtual application is setup as part of the Exchange Server installation.  The Crawler and EC connect to this.

    /WBXPowerShell Virtual Application - This virtual application is a copy of the /PowerShell application made by the  WBXPowerShell_Installer_x64.msi.  At this writing the BAM connects to this virtual application, though this may change in future versions.

     

    Screenshot - IIS Manager on an Exchange Server

     

     

    Exchange Architecture

    Load Balancer

    In some cases the Exchange architecture has 2 or more Exchange servers behind a load balancer.  File Access Manager does not support connections through a load balancer, so you will have to connect directly to one of the Exchange Servers.  In this architecture users connect to mail.abc.com and each Exchange server has a certificate with mail.abc.com for the common name rather than hostname.abc.com, this may cause trust issues when making SSL connections.  The work around will be to edit the host file on the File Access Manager server that is making the connection to the Exchange Server.

     

    Exchange Front-End Servers

    File Access Manager must connect to a  Client Access Servers (CAS) server.  In some cases the CAS server is front-end servers only that has no mailboxes and only proxy traffic to backend Exchange servers.  A back-end server has mailboxes and is configured to communicate with a front-end server with HTTP, POP3 or IMAP.  If File Access Manager is connection to a front-end CAS server then the pre-requisites must apply to the back-end server(s) as well.

     

     

    HTTP vs. HTTPS Connections

    When File Access Manager connects to Exchange server using HTTP it also uses Kerberos authentication.  You can rest assured that the credentials are protected so you can probably avoid the extra configuration and troubleshooting effort required for HTTPS connections.  If the Exchange administrators are using HTTP for their PowerShell connections then use HTTP for File Access Manager also.

     

     

    Using PowerShell to Test Connectivity to Exchange Server

    If you are unable to connect to an Exchange server you can troubleshoot by trying to connect using PowerShell.  Remember that the EC connects to the /PowerShell virtual application and the BAM connects to /WBXPowerShell so be sure to update the examples below as needed.

     

    Example 1

    protocol: http

    port: 80

    authentication: Kerberos

    $session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri http://ex01.dom1.loc/PowerShell -Authentication Kerberos -Credential $credsPS

    Import-PSSession $session

     

    Example 2

    protocol: https

    port: 443

    authentication: Basic

    If Kerberos authentication does not work over SSL, see the troubleshooting section of this document.

    $session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ex01.dom1.loc/PowerShell -Authentication Basic -Credential $credsPS

    Import-PSSession $session

     

    To workaround certificate trust issues you can use the -SessionOptions switch with the connection:

    $SessionOptions = New-PSSessionOption –SkipCACheck –SkipCNCheck –SkipRevocationCheck

     

    error: WARNING: Proxy creation has been skipped

    You may get this error when running the above commands: "WARNING: Proxy creation has been skipped for the following command: 'Add-DistributionGroupMember…"  You can remedy this by running these commands:

    Enter-PSSession $SessionPS

    Import-PSSession $session

     

    Close the PowerShell Connection

    Exchange Online allows a small number of concurrent PowerShell connections so be sure to close you test connections.  If they are left open and the maximum sessions are reached then you must wait for a session to expire before you can connect again.  Close the PowerShell session close it with this command:

    Remove-PSSession $session

    or this:

    Exit-PSSession

    Remove-PSSession -ID $Session.ID

    Screenshot – Opening a PowerShell session

     

     

    Exchange Online

    PowerShell connections to the O365 Exchange Online service are throttled on the server side after a short amount of time.  Expect that when a crawl and EC task begins it will make steady progress for about 5 minutes, then may slow down considerably.

     

     

    Connecting to Exchange Server - Common Errors

     

    Untrusted Certificate Error

    Self-signed certificates in Exchange test environments prevent a successful connection to Exchange Servers.  Export the certificate from the Exchange server website and import it into Trusted Root Certification Authority (Local Computer) on the server where the EC is installed. Check the IIS bindings to get the display name of the certificate that should be exported for the EC.

     

    In this case the Exchange application has been added to File Access Manager and is configured to connect port 443 using SSL and the EC and BAM work when using non-SSL connections to Exchange on port 80.

    entitlement collector log sample

    The EC log will show this error when there is a certificate trust issue.

    2016-05-04 14:16:08,764,40,ERROR,WBX.whiteOPS.Server.RoleAnalytics.ConcurrentPowerShellComponent,openRunspacePool,Could not open PowerShell session using Authentication: 'Default' User name: 'DOM1\SIQ_EX' Server: 'https://ex01/Powershell' Schema: 'http://schemas.microsoft.com/powershell/Microsoft.Exchange'

    System.Management.Automation.Remoting.PSRemotingTransportException: Connecting to remote server ex01 failed with the following error message : The WinRM client received an HTTP bad request status (400), but the remote service did not include any other information about the cause of the failure. For more information, see the about_Remote_Troubleshooting Help topic.

       at System.Management.Automation.Runspaces.Internal.RunspacePoolInternal.EndOpen(IAsyncResult asyncResult)

       at WBX.whiteOPS.Server.RoleAnalytics.ConcurrentPowerShellComponent.openRunspacePool(RunspacePool runspacePool)

    2016-05-04 14:16:08,764,40,DEBUG,WBX.whiteOPS.Server.RoleAnalytics.ConcurrentPowerShellComponent,initRunspacePool,Falling back to Basic authentication

    2016-05-04 14:16:08,764,40,DEBUG,WBX.whiteOPS.Server.RoleAnalytics.ConcurrentPowerShellComponent,openRunspacePool,Opening runspace pool

    2016-05-04 14:16:08,826,40,ERROR,WBX.whiteOPS.Server.RoleAnalytics.ConcurrentPowerShellComponent,openRunspacePool,Could not open PowerShell session using Authentication: 'Basic' User name: 'DOM1\SIQ_EX' Server: 'https://ex01/PowerShell-LiveID?PSVersion=4.0 ' Schema: 'http://schemas.microsoft.com/powershell/Microsoft.Exchange'

    System.Management.Automation.Remoting.PSRemotingTransportException: Connecting to remote server ex01 failed with the following error message : The WinRM client sent a request to an HTTP server and got a response saying the requested HTTP URL was not available. This is usually returned by a HTTP server that does not support the WS-Management protocol. For more information, see the about_Remote_Troubleshooting Help topic.

       at System.Management.Automation.Runspaces.Internal.RunspacePoolInternal.EndOpen(IAsyncResult asyncResult)

       at WBX.whiteOPS.Server.RoleAnalytics.ConcurrentPowerShellComponent.openRunspacePool(RunspacePool runspacePool)

    2016-05-04 14:16:08,826,40,ERROR,WBX.whiteOPS.Server.RoleAnalytics.ExchangeCrawlerEngine,getBamConfiguration,Could not connect to Exchange server https://EX01:443/Powershell with DOM1\SIQ_EX via PowerShell

     

    PowerShell session sample

    Similarly, if you try to connect with PowerShell using this command...

    $session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ex01.dom1.loc/PowerShell -Authentication Kerberos -Credential $creds

     

    ...you may see this error:

    New-PSSession : [ex01.dom1.loc] Connecting to remote server ex01.dom1.loc failed with the following error message :

    The server certificate on the destination computer (ex01.dom1.loc:443) has the following errors:

    The SSL certificate is signed by an unknown certificate authority. For more information, see the

    about_Remote_Troubleshooting Help topic.

    At line:1 char:12

    + $session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri ht ...

    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : OpenError: (System.Manageme....RemoteRunspace:RemoteRunspace) [New-PSSession], PSRemotin

    gTransportException

    + FullyQualifiedErrorId : 12175,PSSessionOpenFailed

     

    Solution to the untrusted certificate problem

    Exchange servers using self-signed certificates can cause this problem, which is very common in test environments.  Export the certificate from the Exchange website and import it into Trusted Root Certification Authority (Local Computer) on the server where the EC is installed. Check the IIS bindings to get the display name of the certificate that should be exported for the EC.

     

     

    Could not open PowerShell session using Authentication: 'Default'

    In some configurations an Exchange Server might require basic authentication for connections.

    • Exchange application has been added to File Access Manager and is configured to connect port 443 using SSL.
    • The EC and BAM work when using non-SSL connections to Exchange on port 80.
    • The Exchange server certificate is in the Trusted Root certification Authority, but now a different error is occurring.

     

    Sample error from EC log

    2016-05-06 09:13:38,654,7,ERROR,WBX.whiteOPS.Server.RoleAnalytics.ConcurrentPowerShellComponent,openRunspacePool,Could not open PowerShell session using Authentication: 'Default' User name: 'DOM1\SIQ_EX' Server: 'https://ex01/Powershell' Schema: 'http://schemas.microsoft.com/powershell/Microsoft.Exchange'

    System.Management.Automation.Remoting.PSRemotingTransportException: Connecting to remote server ex01 failed with the following error message : The WinRM client received an HTTP bad request status (400), but the remote service did not include any other information about the cause of the failure. For more information, see the about_Remote_Troubleshooting Help topic.

       at System.Management.Automation.Runspaces.Internal.RunspacePoolInternal.EndOpen(IAsyncResult asyncResult)

       at WBX.whiteOPS.Server.RoleAnalytics.ConcurrentPowerShellComponent.openRunspacePool(RunspacePool runspacePool)

    2016-05-06 09:13:38,680,7,DEBUG,WBX.whiteOPS.Server.RoleAnalytics.ConcurrentPowerShellComponent,initRunspacePool,Falling back to Basic authentication

    2016-05-06 09:13:38,681,7,DEBUG,WBX.whiteOPS.Server.RoleAnalytics.ConcurrentPowerShellComponent,openRunspacePool,Opening runspace pool

    2016-05-06 09:13:38,789,7,ERROR,WBX.whiteOPS.Server.RoleAnalytics.ConcurrentPowerShellComponent,openRunspacePool,Could not open PowerShell session using Authentication: 'Basic' User name: 'DOM1\SIQ_EX' Server: 'https://ex01/PowerShell-LiveID?PSVersion=4.0 ' Schema: 'http://schemas.microsoft.com/powershell/Microsoft.Exchange'

    System.Management.Automation.Remoting.PSRemotingTransportException: Connecting to remote server ex01 failed with the following error message : The WinRM client sent a request to an HTTP server and got a response saying the requested HTTP URL was not available. This is usually returned by a HTTP server that does not support the WS-Management protocol. For more information, see the about_Remote_Troubleshooting Help topic.

       at System.Management.Automation.Runspaces.Internal.RunspacePoolInternal.EndOpen(IAsyncResult asyncResult)

       at WBX.whiteOPS.Server.RoleAnalytics.ConcurrentPowerShellComponent.openRunspacePool(RunspacePool runspacePool)

    2016-05-06 09:13:38,792,7,ERROR,WBX.whiteOPS.Server.RoleAnalytics.ExchangeCrawlerEngine,getBamConfiguration,Could not connect to Exchange server https://EX01:443/Powershell with DOM1\SIQ_EX via PowerShell

     

     

    Sample Error from PowerShell

    Likewise, attempts to Exchnage using PowerShell with SSL and Kerberos authentication result in this output.  It is misleading because the username and password in use is correct as it works on port 80/non-SSL connections:

    New-PSSession : [ex01.dom1.loc] Connecting to remote server ex01.dom1.loc failed with the following error message :

    The user name or password is incorrect. For more information, see the about_Remote_Troubleshooting Help topic.

    At line:1 char:12

    + $session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri ht ...

    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : OpenError: (System.Manageme....RemoteRunspace:RemoteRunspace) [New-PSSession], PSRemotingTransportException

    + FullyQualifiedErrorId

     

    Solution

    Turn on Basic authentication on the /PowerShell website.  If you do not do this the PowerShell session will run using the either the Windows service account or the application pool account and will fail

     

    /WBXPowerShell web application will not start

    The EC and BAM connection to /PowerShell or /WBXPowerShell which are IIS application on the Exchange server.  These applications start when the IIS gets the first request for a resource in the application.  If for some reason the IIS application cannot start the EC or BAM will not get the type of message it expects.

    • There is not a straightforward message in the EC or BAM log files to tell you this.

    • Also the PowerShell session will fail and the messages it returns will not help you identify the problem.

    • This can happen no matter what port, protocol, or credentials are in use.

     

    EC log file or PowerShell session error message

    System.Management.Automation.Remoting.PSRemotingTransportException: Connecting to remote server ex01 failed with the following error message : The WinRM client cannot process the request. It cannot determine the content type of the HTTP response from the destination computer. The content type is absent or invalid. For more information, see the about_Remote_Troubleshooting Help topic.

     

    Windows Log File

    You might see these events in the Windows event log file

     

    ERROR 3008

    Event code: 3008

    Event message: A configuration error has occurred.

    Event time: 3/24/2016 5:47:02 PM

    Event time (UTC): 3/24/2016 9:47:02 PM

    Event ID: 6ee0ff193e454be39c65230e4adf562a

    Event sequence: 1

    Event occurrence: 1

    Event detail code: 0

     

    Application information:

        Application domain: /LM/W3SVC/1/ROOT/WBXPowerShell-4-131033296219817913

        Trust level: Full

        Application Virtual Path: /WBXPowerShell

        Application Path: E:\Program Files\Microsoft\Exchange Server\V14\ClientAccess\WBXPowerShell\

        Machine name: MYSERVERNAME

     

    Process information:

        Process ID: 7484

        Process name: w3wp.exe

        Account name: NT AUTHORITY\SYSTEM

     

    Exception information:

        Exception type: ConfigurationErrorsException

        Exception message: Could not load file or assembly 'Hitech.SomeProduct.Exchange.SomeLibrary' or one of its dependencies. The system cannot find the file specified.

     

    Request information:

        Request URL: https://MyServerName.is.customer.com:443/wbxPowerShell

        Request path: /wbxPowerShell

        User host address: 10.122.0.136

        User:

        Is authenticated: False

        Authentication Type:

        Thread account name: NT AUTHORITY\SYSTEM

     

    Thread information:

        Thread ID: 1

        Thread account name: NT AUTHORITY\SYSTEM

        Is impersonating: False

        Stack trace:    at System.Web.Configuration.ConfigUtil.GetType(String typeName, String propertyName, ConfigurationElement configElement, XmlNode node, Boolean checkAptcaBit, Boolean ignoreCase)

       at System.Web.Configuration.Common.ModulesEntry..ctor(String name, String typeName, String propertyName, ConfigurationElement configElement)

       at System.Web.HttpApplication.BuildIntegratedModuleCollection(List`1 moduleList)

       at System.Web.HttpApplication.GetModuleCollection(IntPtr appContext)

       at System.Web.HttpApplication.RegisterEventSubscriptionsWithIIS(IntPtr appContext, HttpContext context, MethodInfo[] handlers)

       at System.Web.HttpApplication.InitSpecial(HttpApplicationState state, MethodInfo[] handlers, IntPtr appContext, HttpContext context)

       at System.Web.HttpApplicationFactory.GetSpecialApplicationInstance(IntPtr appContext, HttpContext context)

       at System.Web.Hosting.PipelineRuntime.InitializeApplication(IntPtr appContext)

     

    Solution

    One reason an IIS application fails to start is that it is missing a dependency.  In the case above, the web.config file of the /WBXPowerShell application has a reference to Hitech.SomeProduct.Exchange.SomeLibrary that it cannot resolve.   This is not needed by WBXPowerShell and can be commented in the web.config file in the root folder of the virtual application.  The path to the web.config file and the line that needed to be commented out is shown below.

     

    Program Files\Microsoft\ExchangeServer\V14\ClientAccess\WBXPowerShell\web.config

     

    <add name="PowerShellMonitor" type="Hitech.SomeProduct.Exchange.PowerShell.HttpHandler, Hitech.SomeProduct.Exchange.SomeLibrary"/>

     

    Turn on Basic authentication on the /PowerShell website.  If you do not do this the PowerShell session will run using the either the Windows service account or the application pool account and will fail

     

    Error: O365 / Too many sessions

    O365 allows a limited number of connections and if they are exceeded the error below is raised.

     

    2016-05-06 10:42:26,624,23,ERROR,WBX.Common.Utilities.PowerShellUtilities,runCommands,Error: IncorrectProtocolVersion,PSSessionOpenFailed

    System.Management.Automation.Remoting.PSRemotingTransportException: Processing data from remote server ps.outlook.com failed with the following error message: [ClientAccessServer=SN1PR0501CA0001,BackEndServer=dm2pr0401mb0960.namprd04.prod.outlook.com,RequestId=39a8e990-0f0d-4260-8e3c-8b20ec93d790,TimeStamp=5/6/2016 3:42:26 PM] [AuthZRequestId=f32902d1-42ed-4946-8fac-13e17f4aa5cb][FailureCategory=AuthZ-AuthorizationException] Fail to create a runspace because you have exceeded the maximum number of connections allowed : 3 for the policy party : MaxConcurrency. Please close existing runspace and try again.

    Policy: CN=GlobalThrottlingPolicy_16dcaf50-eef2-42a9-a8e2-4255a75e3095,CN=Global Settings,CN=ExchangeLabs,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=namprd04,DC=prod,DC=outlook,DC=com;

    Snapshot: Owner:        Sid~S-1-5-21-3835325838-1622933477-1555390567-34503953~WSMan~false

    BudgetType:        WSMan

    ActiveRunspaces:        3/3

    Balance:        600000/1800000/-3000000

    PowerShellCmdletsLeft:        200/200

    ExchangeCmdletsLeft:        9223372036854775807/Unlimited

    CmdletTimePeriod:        5

    DestructiveCmdletsLeft:        9223372036854775807/Unlimited

    DestructiveCmdletTimePeriod:        Unlimited

    QueueDepth:        Unlimited

    MaxRunspacesTimePeriod:        60

    RunSpacesRemaining:        2/5

    LastTimeFrameUpdate:        5/6/2016 3:42:26 PM

    LastTimeFrameUpdateDestructiveCmdlets:        5/6/2016 3:42:17 PM

    LastTimeFrameUpdateMaxRunspaces:        5/6/2016 3:42:17 PM

    Locked:        False

    LockRemaining:        00:00:00

     

    For more information, see the about_Remote_Troubleshooting Help topic.

     

    Unable to retrieve mailbox folder statistics for mailbox

    The permission collector is working, but is not able to collect the permissions on come mailboxes due to this error, and PowerShell commands executed from the command line give a similar result.

    System.Management.Automation.RemoteException: Unable to retrieve mailbox folder statistics for mailbox SOME-MAILBOX@somedomain.com. Failure: Error code -2146233088 occurred with message The mailbox of user SOME-MAILBOX@somedomain.com that is located on a server that is running version 14 can't be opened on a server that is running version 15..

     

    Solution

    This is a problem with Exchange mailboxes that have not been properly migrated to from one version of Exchange to another.  The Exchange server administrator should use the New-MoveRequest command or other method to resolve this problem.

     

    No Exchange Server Activities in the File Access Manager Client

    In this scenario there are no activities for Exchange Server showing in the File Access Manager Client and:

    • You have installed the Exchange Server activity monitor and verified the configuration is correct.
    • There are no errors in the File Access Manager log files.
    • The debug log shows that File Access Manager is successfully running PowerShell commands to collect the audit data.
    • Manually running PS commands also does not return any audit results.

     

    This can occur if the Exchange Server indexes are corrupt and Exchange Server does not return audit results.  This problem can occur in new or existing installations of Exchange server and different versions.  Run the Get-MailboxDatabaseCopyStatus command and check the ContentIndexState.

    screenshot - output of Get-MailboxDatabaseCopyStatus with an index problem

     

    Follow these instructions to fix the indexes.

    screenshot - output of Get-MailboxDatabaseCopyStatus with a healthy index

    Version history
    Revision #:
    2 of 2
    Last update:
    ‎Feb 16, 2021 11:05 AM
    Updated by:
     
    Contributors