Wednesday, 21 October 2015

Exchange 2013 Managed Availability free disk space thresholds

On a fresh installation of Exchange 2013 in my lab I ran the following:

Get-HealthReport -Server EX2K13-2

I got an output of all the Managed Availability Health Sets and one of them was unhealthy:

Server              State               HealthSet           AlertValue          LastTransitionTime  MonitorCount
------              -----               ---------           ----------          ------------------  ------------

....
EX2K13-2            NotApplicable       Inference           Healthy             30.9.2015. 19:43:33 2
EX2K13-2            NotApplicable       OWA                 Healthy             14.10.2015. 11:1... 19
EX2K13-2            NotApplicable       FfoMonitoring       Healthy             30.9.2015. 19:43:45 25
EX2K13-2            NotApplicable       MailboxSpace        Unhealthy           14.10.2015. 11:3... 7
EX2K13-2            NotApplicable       MRS                 Healthy             30.9.2015. 19:43:58 2
EX2K13-2            NotApplicable       MessageTracing      Healthy             30.9.2015. 19:43:47 7
EX2K13-2            NotApplicable       Network             Healthy             30.9.2015. 19:58:23 5

.....

I wanted to figure out why is this health set in unhealthy state.

Get-ServerHealth -Server EX2K13-2 -HealthSet MailboxSpace | select name,alertvalue

I can see that the two monitors where unhealthy, the ones monitoring free logical drive space where databases are located:

Name                                                                 AlertValue
----                                                                 ----------
MaintenanceFailureMonitor.MailboxSpace                                  Healthy
MaintenanceTimeoutMonitor.MailboxSpace                                  Healthy
DatabaseSizeMonitor                                                     Healthy
StorageLogicalDriveSpaceMonitor                                       Unhealthy
DatabaseSizeEscalationProcessingMonitor                                 Healthy
DatabaseSizeMonitor                                                     Healthy
StorageLogicalDriveSpaceMonitor                                       Unhealthy


The next thing to do was to collect all of the defined monitors from Event Log crimson channel:

$DefinedMonitors = (Get-WinEvent -ComputerName EX2K13-2 -LogName Microsoft-Exchange-ActiveMonitoring/MonitorDefinition | % {[xml]$_.toXml()}).event.userData.eventXml

Then we look for the SampleMask value for StorageLogicalDriveSpaceMonitor which returns all the probes that are configured for this monitor:

($DefinedMonitors | ? {$_.Name -eq 'StorageLogicalDriveSpaceMonitor'}).SampleMask

There are two probes defined:

MSExchangeDagMgmt/EdbAndLogVolSpace/MAILBOX DATABASE 1369619328
MSExchangeDagMgmt/EdbAndLogVolSpace/EX2K13DB1


Next, we gather the probe results from the crimson channel. We can do this for both probes, but I will do this only for the latter:

$ProbeResults = (Get-WinEvent -ComputerName EX2K13-2 -LogName Microsoft-Exchange-ActiveMonitoring/ProbeResult -
FilterXPath "*[UserData[EventXML[ResultName='MSExchangeDagMgmt/EdbAndLogVolSpace/EX2K13DB1']]]" | % {[XML]$_.toXml()}).e
vent.userData.eventXml


This is one example event of the output:

ExecutionStartTime : 2015-10-21T15:32:57.1805887Z
ExecutionEndTime   : 2015-10-21T15:32:57.1805887Z
ResultId           : 2878967
ResultName         : MSExchangeDagMgmt/EdbAndLogVolSpace/EX2K13DB1
ResultType         : 4
Error              : 'EX2K13DB1' is low on log volume space [E:\]. Current=19,38 GB, Threshold=175,78 GB
ExecutionContext   : [null]
FailureContext     : [null]
StateAttribute1    : 'EX2K13DB1' is low on log volume space [E:\]. Current=19,38 GB, Threshold=175,78 GB
StateAttribute2    : [null]
StateAttribute3    : [null]
StateAttribute4    : [null]
StateAttribute5    : [null]
StateAttribute6    : 0
StateAttribute7    : 0
StateAttribute8    : 0
StateAttribute9    : 0
StateAttribute10   : 0
StateAttribute11   : [null]
StateAttribute12   : [null]
StateAttribute13   : [null]
StateAttribute14   : [null]
StateAttribute15   : [null]
StateAttribute16   : 0
StateAttribute17   : 0
StateAttribute18   : 0
StateAttribute19   : 0
StateAttribute20   : 0
StateAttribute21   : [null]
StateAttribute22   : [null]
StateAttribute23   : [null]
StateAttribute24   : [null]
StateAttribute25   : [null]


We see that free space on the disk is 19,38 GB, but probe threshold is 175,78 GB and that's why we are seeing this monitor putting the health set in unhealthy state.

We can also search for a responder that is defined for this monitor by again querying the crimson channel. We get the responders first:

$DefinedResponders = (Get-WinEvent -ComputerName EX2K13-2 -LogName Microsoft-Exchange-ActiveMonitoring/ResponderDefinition | % {[xml]$_.toXml()}).event.userData.eventXml

 Then we search for the monitor in the AlertMask attribute:

$DefinedResponders | ?{$_.alertmask -like "StorageLogicalDriveSpaceMonitor*"} | select name,alertmask

Name                                                        AlertMask
----                                                        ---------
StorageLogicalDriveSpaceEscalate                            StorageLogicalDriveSpaceMonitor/MAILBOX DATABASE 1369619328
StorageLogicalDriveSpaceEscalate                            StorageLogicalDriveSpaceMonitor/EX2K13DB1


Here are the full details for one of the responders:

auto-ns2                       : http://schemas.microsoft.com/win/2004/08/events
xmlns                          : myNs
Id                             : 1504
AssemblyPath                   : C:\Program Files\Microsoft\Exchange Server\V15\Bin\Microsoft.Exchange.Monitoring.Activ
                                 eMonitoring.Local.Components.dll
TypeName                       : Microsoft.Exchange.Monitoring.ActiveMonitoring.ActiveMonitoring.Responders.EscalateRes
                                 ponder
Name                           : StorageLogicalDriveSpaceEscalate
WorkItemVersion                : [null]
ServiceName                    : MailboxSpace
DeploymentId                   : 0
ExecutionLocation              : [null]
CreatedTime                    : 2015-10-14T09:19:43.4130982Z
Enabled                        : 1
TargetPartition                : [null]
TargetGroup                    : [null]
TargetResource                 : EX2K13DB1
TargetExtension                : [null]
TargetVersion                  : [null]
RecurrenceIntervalSeconds      : 0
TimeoutSeconds                 : 300
StartTime                      : 2015-10-14T09:19:43.4130982Z
UpdateTime                     : 2015-10-14T09:15:30.3926229Z
MaxRetryAttempts               : 3
ExtensionAttributes            : <ExtensionAttributes LoadFromResourceAttributeValue="False" />
AlertMask                      : StorageLogicalDriveSpaceMonitor/EX2K13DB1
WaitIntervalSeconds            : 14400
MinimumSecondsBetweenEscalates : 14400
EscalationSubject              : [DataProtection Alert] (EX2K13-2) Database 'EX2K13DB1' is low on log volume space
EscalationMessage              : Database 'EX2K13DB1' is low on log volume space. {Probe.StateAttribute1}

EscalationService              : [null]
EscalationTeam                 : High Availability
NotificationServiceClass       : 2
DailySchedulePattern           : Pacific Standard Time/Monday,Tuesday,Wednesday,Thursday,Friday,Saturday,Sunday/00:00/2
                                 3:59
AlwaysEscalateOnMonitorChanges : 0
Endpoint                       : [null]
CreatedById                    : 67
Account                        : [null]
AlertTypeId                    : StorageLogicalDriveSpaceMonitor
TargetHealthState              : 3
CorrelatedMonitorsXml          : [null]
ActionOnCorrelatedMonitors     : 0
ResponderCategory              : [null]
ThrottleGroupName              : [null]
ThrottlePolicyXml              : [null]
UploadScopeNotification        : 0
SuppressEscalation             : 0
Version                        : 65536


EscalationMessage attribute is what is actually logged in the following event log: Microsoft-Exchange-ManagedAvailability/Monitoring



2 comments:

  1. This was very helpfull. Thank you for sharing this information

    ReplyDelete
  2. what is the solution for this how i can modify threshold size

    ReplyDelete