Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

vmware backups locking out backup exec service account.

Created: 22 May 2013 | 4 comments

Hi All, we have a very odd problem I hope you can help with!

We are using backupexec 2012 to backup our vmware environment.  Everytime we backup a vm we get a audit failure for logon in the security logs on the vcentre server (see below), but the backup runs fine!  The problem we have is that if we try and run 10 vm backup jobs in a half hour strech the logon failure locks out the service account created for backup exec and then all the following jobs fail until the account lockout period expires.  Our account lockout policy is set to lock the account after 10 failed attemps.

So I'm after some advice as to what is causing the logon failures on the vcentre server.  The really odd thing is is that if I am logged into the backup exec server with my own account (not the backup exec service account), we still get the login failures but the account doesn't lock out.

Some details:

Domain is 2008r2
Backup Exec is on a physical 2008r2 server
The credentials test in backup Exec is all ticks for every vm in Vcentre
The backup exec service account (sv.backupexec.admin) has all the rights and permissions as per the symantec documentation (domain admin, logon on as a service etc), these are set in a gpo that goes to all servers.  the service account is also a full vmware administrator.
We do not get the logon failure on any physical servers, they only show in on the two vcentre servers.
The backup jobs are created to backup indervidual vm's through one of two vcentre servers.  There is no GRT, AOFO is set to use snapshot tech and auto select (I have tried with no snapshot and the same things happens), there is no advanced disk-base backup comfigured.

So thanks in advance for any help you can give.

Ed
Here's the windows error:

Log Name:      Security
Source:        Microsoft-Windows-Security-Auditing
Date:          21/05/2013 23:30:19
Event ID:      4625
Task Category: Logon
Level:         Information
Keywords:      Audit Failure
User:          N/A
Computer:      S2VLESXVC01.universe.local
Description:
An account failed to log on.
 
Subject:
Security ID: UNIVERSE\sv.S2vlesxvc01
Account Name: sv.S2vlesxvc01
Account Domain: UNIVERSE
Logon ID: 0xda90e
 
Logon Type: 2
 
Account For Which Logon Failed:
Security ID: NULL SID
Account Name: sv.backupexec.admin
Account Domain: UNIVERSE
 
Failure Information:
Failure Reason: Unknown user name or bad password.
Status: 0xc000006d
Sub Status: 0xc000006a
 
Process Information:
Caller Process ID: 0xb7c
Caller Process Name: C:\Program Files\VMware\Infrastructure\VirtualCenter Server\vpxd.exe
 
Network Information:
Workstation Name: S2VLESXVC01
Source Network Address: -
Source Port: -
 
Detailed Authentication Information:
Logon Process: Advapi  
Authentication Package: Negotiate
Transited Services: -
Package Name (NTLM only): -
Key Length: 0
 
This event is generated when a logon request fails. It is generated on the computer where access was attempted.
 
The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.
 
The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network).
 
The Process Information fields indicate which account and process on the system requested the logon.
 
The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.
 
The authentication information fields provide detailed information about this specific logon request.
- Transited services indicate which intermediate services have participated in this logon request.
- Package name indicates which sub-protocol was used among the NTLM protocols.
- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
    <EventID>4625</EventID>
    <Version>0</Version>
    <Level>0</Level>
    <Task>12544</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8010000000000000</Keywords>
    <TimeCreated SystemTime="2013-05-21T22:30:19.733033600Z" />
    <EventRecordID>836879</EventRecordID>
    <Correlation />
    <Execution ProcessID="516" ThreadID="12892" />
    <Channel>Security</Channel>
    <Computer>S2VLESXVC01.universe.local</Computer>
    <Security />
  </System>
  <EventData>
    <Data Name="SubjectUserSid">S-1-5-21-2195173279-127488908-3912173888-1125</Data>
    <Data Name="SubjectUserName">sv.S2vlesxvc01</Data>
    <Data Name="SubjectDomainName">UNIVERSE</Data>
    <Data Name="SubjectLogonId">0xda90e</Data>
    <Data Name="TargetUserSid">S-1-0-0</Data>
    <Data Name="TargetUserName">sv.backupexec.admin</Data>
    <Data Name="TargetDomainName">UNIVERSE</Data>
    <Data Name="Status">0xc000006d</Data>
    <Data Name="FailureReason">%%2313</Data>
    <Data Name="SubStatus">0xc000006a</Data>
    <Data Name="LogonType">2</Data>
    <Data Name="LogonProcessName">Advapi  </Data>
    <Data Name="AuthenticationPackageName">Negotiate</Data>
    <Data Name="WorkstationName">S2VLESXVC01</Data>
    <Data Name="TransmittedServices">-</Data>
    <Data Name="LmPackageName">-</Data>
    <Data Name="KeyLength">0</Data>
    <Data Name="ProcessId">0xb7c</Data>
    <Data Name="ProcessName">C:\Program Files\VMware\Infrastructure\VirtualCenter Server\vpxd.exe</Data>
    <Data Name="IpAddress">-</Data>
    <Data Name="IpPort">-</Data>
  </EventData>
</Event>
Operating Systems:

Comments 4 CommentsJump to latest comment

Colin Weaver's picture

Are you creating your backup jobs (in teeh BE console) using the vCenter Server or using the individual ESX hosts?

If using the vCenter server what credentials have you specifically defined for the VCenter server object in the selections?

Also are there any domain and workgroup boundaries being crossed.

i.e. is media server in different domain form vcenter, is media server in different domain from VMs, are VMs in different domain from vCenter , is anything in a workgroup

Just as an Example in my testsetup

Media server is in Domain 1

VM's are in Domain 2

VCenter is in a workgroup

My backups are defined via the vCenter object and I have specified the local workgroup administrator for the job to use to achive this - then against the VM objects themselves (in the vcenter job) I have specified Domain2 admins so that I can use GRT capabilities within the VM

EdHousing21's picture

Hi Colin,

We are creating the backup job in BE by connection to a vcentre server.
We are using a single service account for all backups.  That service account (refrenced in the ms error log above sv.backupexec.admin) is a domain admin, it is also a local admin on the vcentre server and is a full administrator in vmcentre itself.

All the servers are domain server (vcenter, be server and vm's) in universe.local

Thanks for your help so far!

Ed.

VJware's picture

Which account is used by vxpd (Virtual Center service) ? If it is the same Backup Exec logon account, try re-entering the credentials.

Were SSL certs recently changed/replaced on the vCenter ? Was the password for the Backup Exec logon account recently changed ? ( 0xc000006a refers to user name is correct but the password is wrong)

EdHousing21's picture

Hi VJware, the account used by vxpd isn't the backup exec account, it's sv.S2vlesxvc01 which is also mentioned in the error log.  The account details for that are fine as the services start without any problems.

No new SSL certs for months.

The backup exec account has never had it's password changed.  The details for the account are correct as the backup exec account check on the jobs all pass fine.

Grrrrr, this is really annoying me now!

Ed.