WMI Unhealthy on 2008R2 Domain Controllers - WBEM_E_QUOTA_VIOLATION

Windows Management Instrumentation (WMI) is a key core windows management technology. It provides a consistent approach to carry day to day management operations with programming or scripting languages.

I recently started getting WMI failures on daily basis on my 2008R2 domain controllers accompanied by several scripts failure and DNS performance degradation.

Also I noticed that the Configuration Manager SCCM evaluation rules on this domain controller failed and SCCM is reporting errors. The policy Request date on the SCCM is few hours back and it will never report back to SCCM till the DC/Server is rebooted.

Troubleshooting Steps:

1. I started by running the WMI diagnosis tool from http://www.microsoft.com/en-us/download/details.aspx?id=7684
2. The WMI diag log file reported WBEM_E_QUOTA_VIOLATION as follows:

.5265 16:34:02 (0) ** 981 error(s) 0x8004106C - (WBEM_E_QUOTA_VIOLATION) WMI is taking up too much memory

.5266 16:34:02 (0) ** => This error is typically due to the following major reasons:

.5267 16:34:02 (0) **    - The requested WMI operation is extremely costly in terms of resources and

.5268 16:34:02 (0) **      the WMI provider handling this operation has exceeded the authorized limits.

 3.  tried later to check whether the basic WMI function is working by running the below test:

1.     From Elevated Command Prompt type Run wbemtest, connect the namespace root\cimv2

2.     Click Query… and enter the following query “Select * from Win32_ComputerSystem”

3.     This test failed and the following error was reported.

0x80041017 Facility: WMI  Description: Invalid Query

1   4. I tried fixing and rebuilding the WMI Repository as follows:

• Disable and stop the WMI service. sc config winmgmt start= disabled and net stop winmgmt
• At a command prompt (cmd), change to the WBEM folder. cd %windir%\system32\wbem
• Rename the repository folder. rename repository repository.old
• Re-enable the WMI service. sc config winmgmt start= auto
• Run the following command to manually recompile all of the default WMI .mof files and .mfl files
• cd %windir%\system32\wbem
• for /f %s in ('dir /b *.mof *.mfl') do mofcomp %s

The only way to get around this issue was to manually reboot the server. After Rebooting the server, it works for  few hours without a problem then the failures start again. One thing else to be noticed is that the WMIPRVSE.exe process is consuming huge amount of memory during this problem.

Resolution Steps:

1. Increased the "MemoryPerHost” value to 1 GB (1073741824), by default it should be 536870912 which means 512 MB as per attached article

Memory and Handle Quotas in the WMI Provider Service

http://blogs.technet.com/b/askperf/archive/2008/09/16/memory-and-handle-quotas-in-the-wmi-provider-service.aspx

2. Install the following WMI fixes

KB Article Number (s) : 2705357  

Language: All (Global)  

Platform: x64  

Location: ( http://hotfixv4.microsoft.com/Windows%207/Windows%20Server2008%20R2%20SP1/sp2/Fix399300/7600/free/447586_intl_x64_zip.exe)

KB Article Number (s) : 2692929  

Language: All (Global)  

Platform: x64  

Location: ( http://hotfixv4.microsoft.com/Windows%207/Windows%20Server2008%20R2%20SP1/sp2/Fix395847/7600/free/446374_intl_x64_zip.exe)

KB Article Number (s) : 2617858  

Language: All (Global)  

Platform: x64  

Location: ( http://hotfixv4.microsoft.com/Windows%207/Windows%20Server2008%20R2%20SP1/sp2/Fix384504/7600/free/437954_intl_x64_zip.exe)

KB Article Number (s) : 2465990  

Language: All (Global)  

Platform: x64  

Location: ( http://hotfixv4.microsoft.com/Windows%207/Windows%20Server2008%20R2%20SP1/sp2/Fix354372/7600/free/425609_intl_x64_zip.exe)

KB Article Number (s) : 2492536  

Language: All (Global)  

Platform: x64  

Location: ( http://hotfixv4.microsoft.com/Windows%207/Windows%20Server2008%20R2%20SP1/sp2/Fix360823/7600/free/429002_intl_x64_zip.exe)

For a list of suggested WMI hotfixes on different windows platform, please check this blog which is maintained and updated regularly.

http://blogs.technet.com/b/askperf/archive/2011/08/05/suggested-hotfixes-for-wmi-related-issue-on-windows-platforms.aspx

Read More

AD CS not configured for Revocation checking of all certificates

Recently the SCOM server (One of your best friends on the network) started reporting the error "AD CS not configured for Revocation checking of all certificates", SCOM reported Event 128 as a warning

Event 128 is normally reported when someone try to use Certificate Request with all time-valid CA certificates to request a certificate. However y default CA doesn't support such request and event 128 gets reported.

After checking this issue and consulting Microsoft tech support, this issue will normally occur only after renewal of CA certificate. When the CA certificate is renewed, the OCSP Response signing certificate used for validation of existing certificates must still be signed by the CA certificate that was used to issue the existing certificates and new CA certificate. However by default CA doesn't support the renewal of OCSP Response signing certificate by using a previous CA certificate.

This issue/behavior can be fixed as follows:

1. From elevated CMD run the certutil -setreg ca\UseDefinedCACertInRequest 1
2. Restart the CA services

This command will enable the CA support for certificate request signed by old certificate. If the OCSP is not renewed, you need to go ahead and renew it as per the following articles

http://blogs.technet.com/b/xdot509/archive/2013/06/06/operating-a-pki-ca-certificate-renewals-and-ocsp.aspx

http://technet.microsoft.com/en-us/library/cc770413(WS.10).aspx

Read More

Error 0x803100B7 Group Policy settings require the creation of a startup PIN, but a pre-boot keyboard is not available on this device

I Purchased few weeks ago the Microsoft Surface Pro tablet, its a very nice production tablet that really enables remote users to run their production applications and workloads. There are still some room of improvement to get promoted as the number one choice of tablets for business users. From my point of view the three main things that need improvement are the Battery Life, 3G/4G connectivity option and better Camera.

Surface Pro comes with windows 8 Professional which is very nice and allows you to join your corporate network however it lacks a great feature which is Direct Access ! So I decided to turn it to fully productive device and install windows Enterprise on it. Its very simple as if you are building a new normal fresh computer.

I formatted the Surface drive however I kept the recovery image (for any future need), after finishing Windows Enterprise I installed the latest Surface Pro Firmware and Driver Pack http://www.microsoft.com/en-eg/download/details.aspx?id=38826

Finally I got my DirectAccess working on my Surface. That was really an exciting moment. Th next challenge was joining my domain MBAM/Bitlocker policy. Our MBAM / Bitlocker policy requires the use of a PIN while booting the computer. When the MBAM encryption wizard started I got the error 0x803100B7 Group Policy settings require the creation of a startup PIN, but a pre-boot keyboard is not available on this device and by checking the event viewer the following details were provided as per attached image

To Fix this issue we need to change/enable few settings in the Surface Local Policy.

Note: In order to use the Pre-authentication you need to have a Keyboard attached to the surface during the boot, You may use the Surface Touch/Type Keyboard or any external Keyboard connected to the USB port.

1. Type GPEDIT.MSC in the Run bar to access the local Group Policy Editor
2. Drill down to Computer Configuration - Administrative Templates - Windows Components - BitLocker Drive Encryption - Operating System Drives
3. Enable both "Require Additional Authentication at Startup" and "Enable use of BitLocker authentication requiring preboot keyboard input" - Check below image.

After that restart the Bitlocker Management Client Service to kick in back the MBAM wizard which should complete normally without any problem. 







Read More

Enable Auto Enrollment to Avoid Expiring Certificates

Its common that sometimes few admins miss the renewal of some key certificates in their Microsoft internal PKI (Public Key Infrastructure), this is due to the fact that its a bit of manual task and you need to set manually some Outlook reminders (My favorite method) or run schedules tasks to remind you before the Certificate expiration date.

However if you a user that logs frequently on this CA (Certificate Authority) server we can enable Auto Enrollment for this user. After configuring it, we don’t need to worry about the expiring certificates as long as the specific user still logs onto the CA.

To Enable Auto Enrollment you need to do the following:

1. Right click on the Certificate Template where you need to enable the Auto Enrollment feature
2. On the Security Tab (Check below image), add a specific user or grant an existing user the Auto Enroll permission (In my case i picked a normal low privileged service account that connects periodically on the server at least each month for maintenance and installing latest windows updates.)                                                                                                                                                                                                                                                                      
   
                                                                                                                                                            
3. Publish the Template and issue the needed certificate.
4. Open the Group Policy Management (On your Domain Controller) and either create a new Group policy or simply edit the Default Domain Policy
5. Navigate to User Configuration - Policies - Windows Settings - Security Settings - Public Key Policy and enable Autoenrollment as shown below. 

This user with the Autoenroll feature enabled when logged in on the CA server will get notified and the certificate will get enrolled and the Certificate won't get expired.

Read More

The Validity Period of an Issued Certificate is Shorter than Configured

I recently passed with couple of scenarios where one of the issued Certificates in Microsoft PKI infrastructure solution has validity period shorter than the period already configured on the template of this certificate. The main reason of changing and increasing the validity period/years for several specific certificates is to avoid frequent renewal process. 

The scenario i passed by recently was when a user duplicated one of the templates and changed the Validity from the default 2 Years to 4 Years and issued the new Certificate however the issued certificate still reads 2 Years. This can be due to one of two reasons

1. The CA certificate period /Remaining Period (CA cannot issue a certificate that is longer than its own CA certificate) is less than the user certificate period. You cannot issue a user certificate which is 10 Years valid if your Root CA is 5 years only.
2. The default Validity Period that is allowed by CA (defined in CA reg)

To check for the CA Certificate period/Duration, you need to do the following

1. Open the CA Console
2. Right Click on the CA - Properties
3. From the General TAB click View Certificate and check the duration.

If the CA Remaining duration is less than the required user certificate duration then you need to increase the CA value and renew the CA certificate as follows:

1. Configure CAPolicy.inf that directly controls CA certificate.
2. Go to C:\Windows Folder, find the file CAPolicy.inf
3. Change the "RenewalValidityPeriodUnits" value to the appropriate period (10 or 15 Years)
4. Restart the CA Service
5. Renew the CA Certificate (Right Click on the CA - All Tasks - Renew CA Certificate)

If the CA Period/Duration is fine and longer than the user certificate need then we need to check the default Validity Period in the CA Registry by doing the following:

1. Open Admin CMD on the CA server and type certutil -getreg ca                                                                                                                                                                                    
2. Check the ValidityPeriodUnits which refers to the maximum period that this CA can issue. You can define this value according to your own requirements, but it won’t exceed the lifetime of CA.
3. From the Same CMD run certutil -setreg ca\ValidityPeriodUnits 5 (This will increase the validity to 5 years)
4. Stop and restart CA service.

Now try again to Enroll certificate again from client to check the validity period.

Read More

How to Publish New Certificate Revocation List (CRL) from Offline Root CA to Active Directory and Inetpub

Its highly recommended when building your Microsoft PKI (Public Key Infrastructure) to have your Root CA offline after issuing the Enterprise Sub CA certificates. Its recommended to minimize the access to the Offline Root CA as possible. The Root CA is not a domain joined machine and can be turned off without any problem.

One of the Key issue is the CRL generated from the Root CA, you need to set the CRL interval for a large value so that we don’t need to copy the CRL to an online location frequently and do not implement delta CRLs, because the publication of each delta CRL would require access to the offline root CA in order to copy the delta CRL to an online publication location. In order to change the CRL interval you need to:

1. Turn on the Offline Root CA and login with Admin account
2. Open the Certification Authority Console
3. Right Click on the "Revoked Certificates" and click Properties.
4. Set “CRL Publish interval” to a large value (Default is 26 Weeks) and  uncheck “Publish Delta CRL” check-box.

In order to Publish a new CRL from the offline Root CA to the Enterprise Sub CA you need to do the following:

1. Publish a new CRL on the Root CA, this can be done by Right Click the "Revoked Certificates" - All Tasks - Publish                                                                                                                                                                                                                                                          
   
                                                                                                                                                                    
2. Copy the CRL file from the Root CA located under %systemroot%\system32\certsrv\certenroll to the Sub CA Server
3. Turn off the Root CA
4. Copy the above file to the InetPub folder (HTTP Path) in the Sub CA server which is by default located under the C:\inetpub\wwwroot\Certdata
5. Open an Admin Command Prompt and run the following command to publish it to the Active Directory (LDAP Path).                                                                                           certutil -f -dspublish " C:\Inetpub\wwwroot\certdata\RootCA.crl

This process of renewing the CRL and publishing a new one is manually done since the Root CA is offline and thats why its better to make the CRL publish interval more than the default value so you won't do it frequently. You may also want to set an automated reminder before the next renewal date.

Read More

How to Manually Delete Old/Empty WSUS computer Group from Database

Recently i was trying to delete/Remove one of the old computer groups under WSUS Console - Computers - All Computers. This Group was an old group with no members/Clients or any pending approvals any more. I tried removing it from the GUI by Right licking the object and Delete but the server hanged and i got connection error as shown below.

This problem might occur in WSUS servers utilizing the internal DB which has several limits and with the huge number of updates and many groups you can face such issue.

To solve this problem and manually remove this Group you will need to work it from the database and edit couple of tables as follows:

1. Make Sure to take a backup from your WSUS server and WSUS DB.
2. Use SQL Management Studio to connect to Windows internal database \\.\pipe\MSSQL$Microsoft##SSEE\sql\query
3. We will mainly remove the old records from tbtargetgroupand tbflattenedtargetgroup tables
4. Run “select * from tbtargetgroup” to get the list of groups.
5. Identify the TargetGroupID of one of the groups that you want to delete
6. Run delete from tbflattenedtargetgroup where TargetGroupID = ‘<TargetGroupID for the group which is to be removed>’
7. Run delete from tbdeployment where targetgroupid = ‘<Previous step ID>’
8. Run delete from tbtargetgroup where TargetGroupID = ‘<Previous same ID>’

This should take care of the deletion of these groups. Again its always required to ensure you have full backup from your DB.

Read More