All posts by Kirill Nikolaev

Security, Windows Server 2012

«0x80070721 A security package specific error occurred» while using WMI between domains

September 8, 2014 Kirill Nikolaev Leave a comment

Symptoms:

You have two IBM System x servers with Windows Server 2012 or newer and with «IBM USB Remote NDIS Network Device» network interface enabled. Both of these servers reside in the same AD DS forest but in different domains. You try to setup a WMI session from one to another (using wbemtest, for example). In that case WMI connection fails and you receive an error:

Number: 0x80070721 Facility: Win32 Description: A security package specific error occurred.» In System log of the server which initiates the connection we have: Log Name: System Source: Microsoft-Windows-Security-Kerberos Date: 10/6/2013 9:13:09 PM Event ID: 4 Task Category: None Level: Error Keywords: Classic User: N/A Computer: SRV1.alpha.example.com Description: The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server srv1$. The target name used was host/srv2.beta.example.com. This indicates that the target server failed to decrypt the ticket provided by the client. This can occur when the target server principal name (SPN) is registered on an account other than the account the target service is using. Ensure that the target SPN is only registered on the account used by the server. This error can also happen if the target service account password is different than what is configured on the Kerberos Key Distribution Center for that target service. Ensure that the service on the server and the KDC are both configured to use the same password. If the server name is not fully qualified, and the target domain (ALPHA.EXAMPLE.COM) is different from the client domain (BETA.EXAMPLE.COM), check if there are identically named server accounts in these two domains, or use the fully-qualified name to identify the server.

Why does this happen?

Starting from Windows Server 2012, when one machine connects to another computer’s WMI, Windows asks remote system for IP-addresses of its network interfaces. Then operating system chooses which one of them serves its needs to connect best.
In case of IBM System x, both servers have network interface with the same IP-address – 169.254.95.120. Windows chooses this IP-address as the best and tries to connect to it. But instead of remote system, it connects to itself and you see the error.

When you try to connect with wbemtest, it calls a WMI API. In the background WMI uses DCOM to communicate with other servers. When DCOM establishes a session it uses «ServerAlive2» query to check the other server.
Here is what network capture looks like:
14 14:38:55 10.01.2014 0.0111483 srv2.beta.example.com 135 (0x87) 91.103.70.14 55535 (0xD8EF) DCOM DCOM:IObjectExporter:ServerAlive2 Response {MSRPC:10, TCP:9, IPv4:8} NetworkAddress: srv1 NetworkAddress: 169.254.95.120 NetworkAddress: 192.0.2.1

Same traffic when we access from another end:
38 14:37:12 10.01.2014 37.1871388 srv1.alpha.example.com 135 (0x87) 10.253.12.2 59278 (0xE78E) DCOM DCOM:IObjectExporter:ServerAlive2 Response {MSRPC:14, TCP:13, IPv4:8} NetworkAddress: srv2 NetworkAddress: 169.254.95.120 NetworkAddress: 203.0.113.2

The application layer chooses the common IP-address and loops back the real connection. The DCOM gets a Kerberos ticket, and tries to authenticate with it, so this is why we see AP_ERR_MODIFIED errors from Kerberos. So a DCOM based communication (for example WMI) won’t work if the participants has a common IP address.

This problem is known by Microsoft and will not be fixed since it is by design.

How to mitigate it?

Just disable IMM USB Network Interface at one or at both servers. But beware: updating of IMM firmware from Windows or using ASU64 will enable this interface again. If you choose this option, I suggest you to setup monitoring to alert you when that interface leave enabled.
Change IP-address at one if the interfaces to something else. You can use almost any private address but here are recommendations from IBM. You even can use DHCP-server to achieve it (I hope you are using separate VLAN for management interfaces, aren’t you?).

Exchange Server 2010, SCDPM

SCDPM: Fail to Modify Disk Allocation after Exchange DAG Switched

August 18, 2014 Kirill Nikolaev Leave a comment

Symptoms:

Suppose, you have an Exchange 2010 installation with one or more Database Availability Groups with 2 or more servers in each DAG. You setup backup for one of these DAGs using DPM 2010 UR? or newer (incl. 2012 R2 UR2). Later you change active status for protected copy of mailbox database (for example, you switch active copy to another mailbox server). After that, for database copies which status has changed, you’ll receive following error at Review Disk Allocation page at New/Modify protection group master at DPM console:
"The operation failed because the data source VSS component {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7} is missing. Check to see that the protected data source is installed properly and the VSS writer service is running.

ID: 915
Details: The operation completed successfully (0x0)”

If you’ll try to add such DB to secondary DPM server, you’ll receive same error at a disk size calculation step.

This problem is known by Microsoft and will not be fixed.

Why does this happen?

DPM stores information about protected resources into tbl_IM_DataSource and tbl_IM_ProtectedObject tables in DPMDB. If you look into ApplicationPath, LogicalPath or PhysicalPath cells, you find an XML-document describing protected resource. Here is one for Exchange mailbox database in DAG:

<?xml version="1.0" encoding="utf-16"?>

</InquiryPathEntryType>

</InquiryPathEntryType>

</ArrayOfInquiryPathEntryType>

DAGNODE2.example.com – DAG-node from which database is protected.
MAILDB01 – name of protected DB
Microsoft Exchange Server\Microsoft Information Store\Replica\DAGNODE2 – path to a copy of protected DB at a DAG node which we are protect. Mind “Replica” element of the path – it means we protect passive (not active) copy of DB. In case of active copy, this part of path won’t exist.

When you change status of mailbox database in DAG, the actual LogicalPath changes, but DPM knows nothing about it and keeps an inconsistent data in DPMDB.

Resolution:

There are two workarounds (choose which suits you best):

At DPM side:

Stop protection of problematic DB with retaining its data.
Add the DB back into an appropriate protection group. DPM will update tbl_IM_DataSource and tbl_IM_ProtectedObject tables.
When consistency check completes, you’ll be able to manage allocated disk space for this DB and setup secondary protection for it.

At Exchange side:

Restore active state for problematic DB as it was when you added it into DPM:
1. If the DB was in an active state – make it active again.
2. If the DB was in a passive state – make it passive.

If you require so, you can reinstate DB’s state after modifying disk allocation / setup secondary protection – it doesn’t interfere with synchronization / recovery points creation – it just makes impossible to calculate size of a DB.

ADDS, PowerShell, SCCM

SCCM: Device Collection Based On a Local Group Membership

August 18, 2014 Kirill Nikolaev 2 Comments

New task came up recently – I need to separate in SCCM self-managed workstations from IT-managed ones. We define following criteria for IT-managed workstations: no other accounts are in local Administrators group except for built-in Administrator, Domain Admins group and a group for Service Desk administrators. All workstations are located in the same OU, so I cannot use OU-based collections.

As you may know, SCCM 2012 doesn’t have built-in tools to get local groups membership. Thanks to Sherry Kissinger who solved this problem for us using Compliance Settings. After you install her package, you’ll get a new Configuration Baseline and Configuration Item in SCCM console named as “WMI Framework For Local Groups with Logging” and “Local Group Members into WMI with Logging”. This package also creates 2 new tables and 1 view into SCCM database: LocalGroupMembers_DATA, LocalGroupMembers_HIST and v_GS_LocalGroupMembers0.

After creating and deploying baseline, you can use v_GS_LocalGroupMembers0 view to create reports based on local groups membership.
Don’t forget: you must not deploy that baseline to domain controllers! For example, you can create a collection which includes all your systems except domain controllers: create new device collection using All Systems as limiting collection and add it with include rule, then add All Domain Controllers collection with exclude rule. You can download MOF-file for such collection here.

Unfortunately, neither LocalGroupMembers_DATA, nor v_GS_LocalGroupMembers0 can be used in WQL-queries when you create a collection.
Am I stuck? Let’s review what do I had for now:

I have all data about local groups membership in custom table.
I can create any reports using that data.
I can create collections using data from standard tables in SCCM DB.
But I cannot create collections based on a data from custom SQL-tables.

I need a way to put data from table LocalGroupMembers_DATA into standard SCCM tables and PowerShell is here to save the day.
There are at least two ways to get data from SQL with PowerShell:

Connect to DB directly and use T-SQL queries with SQL cmdlets.
Connect to SQL Server Reporting Services using New-WebServiceProxy cmdlet. Stefan Stranger and Jin Chen wrote an example script to achieve it.

With PowerShell we can do anything with that SQL-data. Our goal is to populate device collections with workstations and here we go again with two different options:

We can add computers into group in AD DS and then create a device collection using this group. For this method to work you need to activate Active Directory Group Discovery discovery method for site and domain where AD group will reside.
Add computers into collection directly using Add-CMDeviceCollectionDirectMembershipRule cmdlet.

Since both group and a report will be useful for me in the future, I’m stick with them.

Now our scenario looks like this:

Activate Active Directory Group Discovery.
Collect local group membership using Compliance Settings.
Create a report with gathered data an any SSRS.
Get names of computers from this report with New-WebServiceProxy cmdlet.
Add these computers into an AD group.
Create a device collection by that AD group.

I build a report where I list all computers don’t comply with conditions discussed earlier.
Here is what first DataSet query looks like:

SELECT distinct

Computer.Name0 AS [Computer Name]

FROM

v_GS_LocalGroupMembers0 AS LGMs

INNER JOIN v_R_System AS Computer

ON LGMs.ResourceID = Computer.ResourceID

INNER JOIN v_RA_System_SystemOUName AS OUs ON Computer.ResourceID = OUs.ResourceID

WHERE (

(OUs.System_OU_Name0 IN (@CompOU))

AND (LGMs.Name0 = N'Administrators')

AND NOT (

LGMs.Account0 = N'Domain Admins' AND LGMs.Category0 = N'Group' AND LGMs.Type0 = N'Domain'

)

AND NOT (

LGMs.Account0 = N'Administrator' AND LGMs.Category0 = N'UserAccount' AND LGMs.Type0 = N'Local' AND LGMs.Name0 = N'Administrators'

)

AND NOT (

LGMs.Account0 = N'ServiceDesk Workstations Administrators' AND LGMs.Category0 = N'Group' AND LGMs.Type0 = N'Domain' AND LGMs.Domain0 = N'EXAMPLE'

)

It can be easily expanded to include another set of groups to ignore.
Mind CompOU parameter: in web-interface you can select multiple OUs where to search computers.
To get a full list of OUs from a forest, you can use another query:

SELECT DISTINCT

v_RA_System_SystemOUName.System_OU_Name0

FROM

v_RA_System_SystemOUName

ORDER BY v_RA_System_SystemOUName.System_OU_Name0 ASC

I modified RenderSQLReportFromPosh.v1.000.ps1 so it could populate AD DS group in addition to get data from reports. Here’s its code:

#requires -version 2.0
###############################################################################
# Render SQL Reports using PowerShell
# This script is using the new Posh v 2.0 cmdlet New-WebServiceProxy
# FileName: RenderSQLReportFromPosh.v1.000.ps1
# Authors: Stefan Stranger (Microsoft)
# Help from: Jin Chen (Microsoft)
# Example of Rendering the OpsMgr Licenses Report from the Generic Report Library
#
# v1.000 – 15/05/2010 - stefstr - initial sstranger's release
# 10/29/2013 - Modified by Kirill 'kf' Nikolaev to satisfy SCCM needs
###############################################################################

#Log file path
$ScriptPath = Split-Path $MyInvocation.MyCommand.Path
$ScriptName = ($MyInvocation.MyCommand.Name).Substring(0,($MyInvocation.MyCommand.Name).Length-4)
$Log = Join-Path $ScriptPath "$ScriptName.txt"

Add-Content $Log (Get-Date)

#Clear-Content $Log -ErrorAction SilentlyContinue
$Error.Clear()
			
#Define Variables            
#Enter URI to asmx file on Report Server            
$URI  = 'https://reportserver.example.com//ReportServer//ReportExecution2005.asmx?wsdl'
#Enter Report Path
$ReportPath = '/ConfigMgr_CAS/Compliance and Settings Management/Workstations with non-default local administrators'
#Enter group name to fill with workstations
$GroupName = 'Self-Managed Workstations'

$format = 'csv'
$deviceinfo = ''
$extention = ''
$mimeType = ''
$Result = ''
$render = ''
$encoding = 'UTF-8'
$warnings = $null            
$streamIDs = $null            
$Reports = New-WebServiceProxy -Uri $URI -UseDefaultCredential -namespace 'ReportExecution2005'
            
            
$rsExec = new-object ReportExecution2005.ReportExecutionService            
$rsExec.Credentials = [System.Net.CredentialCache]::DefaultCredentials

$execInfo = @($ReportPath, $null)

#Load the selected report.            
$rsExec.GetType().GetMethod('LoadReport').Invoke($rsExec, $execInfo) | out-null

#Report Parameters            
#Depending on the number of Parameters being used in the Report you need to add more Parameters.            
#Search the rdl file for the correct parameter names.
$param1 = new-object ReportExecution2005.ParameterValue
$param1.Name = 'CompOU'
$param1.Value = 'EXAMPLE.COM/WORKSTATIONS'

$param2 = new-object ReportExecution2005.ParameterValue
$param2.Name = 'CompOU'
$param2.Value = 'EXAMPLE.COM/WORKSTATIONS-OLD'

$parameters = [ReportExecution2005.ParameterValue[]] ($param1, $param2)             
$ExecParams = $rsExec.SetExecutionParameters($parameters, 'en-us');
$render = $rsExec.Render($format, $deviceInfo,[ref] $extention, [ref] $mimeType,[ref] $encoding, [ref] $warnings, [ref] $streamIDs)             
$Result = [text.encoding]::ascii.getString($render)

$ComputerNames = @()
$SplittedResult = $Result.Split("`n")
for ($i = 1; $i -le $SplittedResult.Count-3) { #For unknown reason, last 3 rows of SSRS answer are blank, so we have to cut them and a title too.
	$ComputerNames += ($SplittedResult[$i]).Substring(0,($SplittedResult[$i]).Length-1) #Again, for unknown reason, there is an invisible line-feed symbol, we remove it.
	$i++
}
$ComputerObjects = @()
foreach ($Computer in $ComputerNames) {
	$ComputerObjects += Get-ADComputer $Computer
}
$CurrentMembers = @()
$CurrentMembers = Get-ADGroupMember -Identity $GroupName

$ToAdd = @()
$ToRemove = @()
if ($CurrentMembers) {
	if ($ComputerObjects) {
		$CompareResult = Compare-Object -ReferenceObject $ComputerObjects -DifferenceObject $CurrentMembers
		foreach ($Item in $CompareResult) {
			$DN = $Item.InputObject.DistinguishedName
			if ($Item.SideIndicator -eq '<=') {
				$ToAdd += $Item.InputObject
				Add-Content $Log "$DN - ToAdd"
			}
			elseif ($Item.SideIndicator -eq '=>') {
				$ToRemove += $Item.InputObject
				Add-Content $Log "$DN - ToRemove"
			}
		}
	}
	else {
		foreach ($Item in $CompareResult) {
			$DN = $Item.DistinguishedName
			$ToRemove += $Item.DistinguishedName
			Add-Content $Log "$DN - ToRemove"
		}
	}
}
else {
	foreach ($Item in $ComputerObjects) {
		$DN = $Item.DistinguishedName
		$ToAdd += $Item.DistinguishedName
		Add-Content $Log "$DN - ToAdd"
	}
}
if ($ToAdd) {
	try {
		Add-ADGroupMember -Identity $GroupName -Members $ToAdd -ErrorAction SilentlyContinue
	}
	catch {
		Add-Content $Log 'Cannot add'
		Add-Content $Log $Error[0]
	}
}
if ($ToRemove) {
	try {
		Remove-ADGroupMember -Identity $GroupName -Members $ToRemove -ErrorAction SilentlyContinue -Confirm:$false
		}
	catch {
		Add-Content $Log 'Cannot remove'
		Add-Content $Log $Error[0]
	}
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

#requires -version 2.0

###############################################################################

# Render SQL Reports using PowerShell

# This script is using the new Posh v 2.0 cmdlet New-WebServiceProxy

# FileName: RenderSQLReportFromPosh.v1.000.ps1

# Authors: Stefan Stranger (Microsoft)

# Help from: Jin Chen (Microsoft)

# Example of Rendering the OpsMgr Licenses Report from the Generic Report Library

# v1.000 – 15/05/2010 - stefstr - initial sstranger's release

# 10/29/2013 - Modified by Kirill 'kf' Nikolaev to satisfy SCCM needs

###############################################################################

#Log file path

$ScriptPath = Split-Path $MyInvocation.MyCommand.Path

$ScriptName = ($MyInvocation.MyCommand.Name).Substring(0,($MyInvocation.MyCommand.Name).Length-4)

$Log = Join-Path $ScriptPath "$ScriptName.txt"

Add-Content $Log (Get-Date)

#Clear-Content $Log -ErrorAction SilentlyContinue

$Error.Clear()

#Define Variables

#Enter URI to asmx file on Report Server

$URI = 'https://reportserver.example.com//ReportServer//ReportExecution2005.asmx?wsdl'

#Enter Report Path

$ReportPath = '/ConfigMgr_CAS/Compliance and Settings Management/Workstations with non-default local administrators'

#Enter group name to fill with workstations

$GroupName = 'Self-Managed Workstations'

$format = 'csv'

$deviceinfo = ''

$extention = ''

$mimeType = ''

$Result = ''

$render = ''

$encoding = 'UTF-8'

$warnings = $null

$streamIDs = $null

$Reports = New-WebServiceProxy -Uri $URI -UseDefaultCredential -namespace 'ReportExecution2005'

$rsExec = new-object ReportExecution2005.ReportExecutionService

$rsExec.Credentials = [System.Net.CredentialCache]::DefaultCredentials

$execInfo = @($ReportPath, $null)

#Load the selected report.

$rsExec.GetType().GetMethod('LoadReport').Invoke($rsExec, $execInfo) | out-null

#Report Parameters

#Depending on the number of Parameters being used in the Report you need to add more Parameters.

#Search the rdl file for the correct parameter names.

$param1 = new-object ReportExecution2005.ParameterValue

$param1.Name = 'CompOU'

$param1.Value = 'EXAMPLE.COM/WORKSTATIONS'

$param2 = new-object ReportExecution2005.ParameterValue

$param2.Name = 'CompOU'

$param2.Value = 'EXAMPLE.COM/WORKSTATIONS-OLD'

$parameters = [ReportExecution2005.ParameterValue[]] ($param1, $param2)

$ExecParams = $rsExec.SetExecutionParameters($parameters, 'en-us');

$render = $rsExec.Render($format, $deviceInfo,[ref] $extention, [ref] $mimeType,[ref] $encoding, [ref] $warnings, [ref] $streamIDs)

$Result = [text.encoding]::ascii.getString($render)

$ComputerNames = @()

$SplittedResult = $Result.Split("`n")

for ($i = 1; $i -le $SplittedResult.Count-3) { #For unknown reason, last 3 rows of SSRS answer are blank, so we have to cut them and a title too.

$ComputerNames += ($SplittedResult[$i]).Substring(0,($SplittedResult[$i]).Length-1) #Again, for unknown reason, there is an invisible line-feed symbol, we remove it.

$i++

}

$ComputerObjects = @()

foreach ($Computer in $ComputerNames) {

$ComputerObjects += Get-ADComputer $Computer

}

$CurrentMembers = @()

$CurrentMembers = Get-ADGroupMember -Identity $GroupName

$ToAdd = @()

$ToRemove = @()

if ($CurrentMembers) {

if ($ComputerObjects) {

$CompareResult = Compare-Object -ReferenceObject $ComputerObjects -DifferenceObject $CurrentMembers

foreach ($Item in $CompareResult) {

$DN = $Item.InputObject.DistinguishedName

if ($Item.SideIndicator -eq '<=') {

$ToAdd += $Item.InputObject

Add-Content $Log "$DN - ToAdd"

}

elseif ($Item.SideIndicator -eq '=>') {

$ToRemove += $Item.InputObject

Add-Content $Log "$DN - ToRemove"

}

else {

foreach ($Item in $CompareResult) {

$DN = $Item.DistinguishedName

$ToRemove += $Item.DistinguishedName

Add-Content $Log "$DN - ToRemove"

}

else {

foreach ($Item in $ComputerObjects) {

$DN = $Item.DistinguishedName

$ToAdd += $Item.DistinguishedName

Add-Content $Log "$DN - ToAdd"

}

if ($ToAdd) {

try {

Add-ADGroupMember -Identity $GroupName -Members $ToAdd -ErrorAction SilentlyContinue

}

catch {

Add-Content $Log 'Cannot add'

Add-Content $Log $Error[0]

}

if ($ToRemove) {

try {

Remove-ADGroupMember -Identity $GroupName -Members $ToRemove -ErrorAction SilentlyContinue -Confirm:$false

}

catch {

Add-Content $Log 'Cannot remove'

Add-Content $Log $Error[0]

}

My modified script receives a report from $URL and $ReportPath locations, compares a list from it with members of $GroupName AD DS group and adds/removes computers from that group until it and the report would be the same.
You can find a path for log of actions in $Log variable. Here, script records all computers which were added or removed from the group.
OUs to search are defined into $param1 and $param2 variables. If you need more OUs, create a new parameter variables and do not forget to add them into $parameters.

As last, I created standard device collection based on AD group $GroupName.

You can download report as an RDL-file and a script here. Do not forget to create DataSource in the report to connect to your SSRS instance.

ADDS, SCCM

SCCM: All Domain Controllers Collection

August 18, 2014 Kirill Nikolaev 4 Comments

There are many ways to create a collection containing all domain controllers. Here are some examples:

By a role of a computer:

SELECT SMS_R_SYSTEM.ResourceID,SMS_R_SYSTEM.ResourceType,SMS_R_SYSTEM.Name,SMS_R_SYSTEM.SMSUniqueIdentifier,SMS_R_SYSTEM.ResourceDomainORWorkgroup,SMS_R_SYSTEM.Client

FROM SMS_R_System

INNER JOIN SMS_G_System_COMPUTER_SYSTEM ON

SMS_G_System_COMPUTER_SYSTEM.ResourceId = SMS_R_System.ResourceId

WHERE SMS_G_System_COMPUTER_SYSTEM.Roles LIKE "%Domain_Controller%"

By the primary AD DS group:

SELECT SMS_R_SYSTEM.ResourceID,SMS_R_SYSTEM.ResourceType,SMS_R_SYSTEM.Name,SMS_R_SYSTEM.SMSUniqueIdentifier,SMS_R_SYSTEM.ResourceDomainORWorkgroup,SMS_R_SYSTEM.Client

FROM SMS_R_System

WHERE SMS_R_System.PrimaryGroupID = "516"

By AD DS group name:

SELECT SMS_R_SYSTEM.ResourceID,SMS_R_SYSTEM.ResourceType,SMS_R_SYSTEM.Name,SMS_R_SYSTEM.SMSUniqueIdentifier,SMS_R_SYSTEM.ResourceDomainORWorkgroup,SMS_R_SYSTEM.Client

FROM SMS_R_System

WHERE SMS_R_System.SystemGroupName = "EXAMPLE\Domain Controllers"

I personally prefer the first version, by a role of a computer. You can download a MOF-file for this collection here. Just import it as described in How to Create Collections in Configuration Manager article and new “All Domain Controllers” collection will appear in your SCCM console.

Sysinternals, Windows

Process Monitor 3.10

March 10, 2014 Kirill Nikolaev Leave a comment

An important update of Process Monitor was released couple of a days ago:

As you may know, we can use two functions with completely opposite names to open registry key: RegCreateKeyEx and RegOpenKeyEx. When you use RegCreateKeyEx, it creates registry key if it’s non-existed, but just opens it if key exists. RegCreateKeyEx writes which operation (create or open) it’s performed into a separate variable.
RegOpenKeyEx cannot create registry keys and returns error if key doesn’t exist.

Before this release there were no way to determine what operation exactly RegCreateKeyEx perform. “Granted Access” property for execution of that function always contained “Read/Write” value.

From this last update, Process Monitor finally can show you what RegCreateKeyEx does. There is no “Granted Access” property for RegCreateKey operation anymore, it was replaced with new “Disposition” property. “Disposition” may contains following strings:
REG_CREATE_NEW_KEY – if new registry key was created.

REG_OPENED_EXISTING_KEY – if RegCreateKeyEx just opened previously existed key.

“Desired Access” property still contains “Read/Write” value, because we cannot predict which action RegCreateKeyEx will do.

Using this new feature, you can separate RegCreateKeyEx calls: just add new condition into Process Monitor’s filter with following parameters:
Column – “Detail”
Relation – “contains”
Value – “REG_CREATE_NEW_KEY” or “REG_OPENED_EXISTING_KEY”
Action – “Include”

SCDPM

Moved protected server between primary DPM servers – cannot add secondary protection

March 2, 2014 Kirill Nikolaev Leave a comment

SYMPTOMS:

You have one secondary DPM-server (DPM3) and 2 primary connected to it (DPM1 and DPM2). You have a server (PS1) protected by DPM1 and DPM3. You decide to move protection of PS1 from DPM1 to DPM2. You stop protection of PS1 at DPM1 and DPM3. You connect PS1 to DPM2. You wait until first replica is created, then you try to add PS1 back to DPM3 but from DPM2 this time.

In that case you cannot see PS1 at DPM3 in the list of data sources protected by DPM2.

CAUSE:

This happens because each protected server has a single record in a table of protected servers at DPM configuration database. One of the attributes of this record is an ID of DPM server protecting that data source. Since secondary DPM knows nothing about you moving data sources between primary servers, it doesn’t update that record with an ID of a new primary DPM server.

RESOLUTION:

You can fix this by manually updating DPMServerId attribute of migrated protected server with an ID of new primary DPM server.

WARNING! This scenario IS NOT supported by Microsoft. Use instructions below at one’s own risk, except and only when you are specifically instructed by Microsoft technical support specialist to do this.

Be sure to backup your DPMDB database before run any T-SQL command!

Connect to DPMDB of your secondary server (You can find more about it here).
Get a list of DPM-servers and their IDs:
SELECT ServerId, ServerName, DPMServerId FROM [dbo].[tbl_AM_Server] WHERE IsDPM = 1
Write down IDs of DPM1 (<OLD-ID>) and DPM2 (<NEW-ID>).
Determine which primary DPM-server protects PS1 according to the knowledge of DPM3:
SELECT DPMServerId FROM [dbo].[tbl_AM_Server] WHERE ServerName = '<Protected Server's FQDN>'
Make sure it equals <OLD-ID>, otherwise this instruction is not for you — contact Microsoft Support instead.
Update protected server record with ID of new primary server:
UPDATE [dbo].[tbl_AM_Server] SET DPMServerId = '<NEW-ID>' WHERE ServerName = '<Protected Server's FQDN>'
Run new/modify protection group master and press “Clear cache” button.
You are now able to protect moved resource at secondary DPM again.
Never ever move protected servers between primary DPMs connected to the same secondary server, since it is NOT supported by Microsoft.

SIMILAR ISSUES:

SCDPM

How to connect to DPM SQL database (DPMDB)

February 26, 2014 Kirill Nikolaev 1 Comment

Connection parameters for System Center Data Protection Manager configuration database are located into registry key HKLM\SOFTWARE\Microsoft\Microsoft Data Protection Manager\DB on the computer where DPM server is installed. You can quickly determine essential parameters by running following PowerShell commands:

SQL server name:
(Get-ItemProperty 'HKLM:SOFTWARE\Microsoft\Microsoft Data Protection Manager\DB').SqlServer
>DPMSRV

Database name:
(Get-ItemProperty 'HKLM:SOFTWARE\Microsoft\Microsoft Data Protection Manager\DB').DatabaseName
>DPMDB_DPMSRV

SQL server instance:
(Get-ItemProperty 'HKLM:SOFTWARE\Microsoft\Microsoft Data Protection Manager\DB').InstanceName
>MSSQLSERVER

Use SQL Server Management Studio to connect to DPM database using values from above. If SQL server instance name is “MSSQLSERVER”, you should omit it while connecting to SQL server.
Only local administrators group is alowed to access DPMDB by default. If you run SSMS on a DPM server locally, don’t forget to run it as administrator.

HP, RDS

HP Lights-Out Passthrough Service cannot be installed at a terminal server

September 15, 2011 Kirill Nikolaev Leave a comment

SYMPTOMS:

When you install HP Lights-Out Passthrough Service at a Terminal Server (which means you have RDP Service enabled for multiple users) you receive following errors:

In installer: Service ‘HP Lights-Out Passthrough Service’ (hplopts) failed to start. Verify that you have sufficient privileges to start system services.
In Application Log: The Passthrough Service could not be started. You must install Terminal Services or enable Remote Desktop.

SOLUTION:

This happens because of weird prerequisites check.

Set registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\fDenyTSConnections = “0”. Installation should succeed.

Exchange Server 2003, Windows Server 2008 R2

Unable to find the callback library jcb.dll (or one of its dependencies)

August 31, 2011 Kirill Nikolaev Leave a comment

If you need to defragment your Exchange database at a computer without Exchange components, you will probably use this MS KB: 244525: How to run Eseutil on a computer without Exchange Server. I tried it when I was defragmenting Exchange 2003 database at a Windows Server 2008 R2 machine.

Eseutil was running fine, but after some time (when it used just over 2 GBs of RAM), I received an error described here: 273087: Error With Jcb.dll While Running Eseutil. Unfortunately, none of the methods described there were useful: when I pressed «Cancel» button, I received an «Operation terminated with error -2102 JET_errCallbackNotResolved, A callback function could not be found) after 1168.136 seconds.» error.

Eventually, I grabbed Process Monitor and found out that files described in Microsoft’s KB 244525 and in this thread «Re: JCB.DLL Not Found Error» weren’t enough – you need another one file: ntlsapi.dll. I copied it from a nearest Windows Server 2003 R2 SP2 box to a system where eseutil were working and everything went smoothly.

NetBackup

NetBackup: Client Attributes: The error is ” (1)”

June 27, 2011 Kirill Nikolaev Leave a comment

SYMPTOMS:

When you are trying to change or delete client attributes from client list in master server properties, you get an error:
Unable to save data on some hosts An error occurred on host example.com. The error is " (1)".

RESOLUTION:

Go to NetBackup installation folder (usually it’s %PROGRAMFILES%VeritasNetBackup). Then proceed to dbclient folder.
If there is only one file and it is named “GP_”, go to step 3. If not, I can’t guarantee anything.
Delete dbclient folder.
Restart NBU services.