Endpoint Protection

 View Only
Expand all | Collapse all

SVA not working or communicating VMWare VDI clients or VM management

  • 1.  SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 17, 2014 09:15 AM

    Before you post with simple links to the "how to" or KB articles or all of the "about" documents, please note the following:
    I've read and followed all of the links and suggestions at this thread:https://www-secure.symantec.com/connect/forums/shared-insight-cache-1212-vshield-security-virtual-appliance-status-unknown
     

    I've read and followed the links in this page,
    http://www.verious.com/article/symantec-endpoint-protection-terminology-guide-concepts-technologies-terms-part-3-s-z/#Security Virtual Appliance (SVA)
    which also include these links:
    About the Symantec Endpoint Protection Security Virtual Appliance

    http://www.symantec.com/docs/HOWTO81080

    VMware software requirements to install a Symantec Security Virtual Appliance

    http://www.symantec.com/docs/HOWTO81081

    Installing a Symantec Endpoint Protection Security Virtual Appliance

    http://www.symantec.com/docs/HOWTO81083

    Configuring the Symantec Endpoint Protection Security Virtual Appliance installation settings file

    http://www.symantec.com/docs/HOWTO81082

     

    We have 2 VM hosts. We placed a Symantec SVA under each host as per instructions. We properly configured the install XML files as per instructions.
    We have not only done that, but we've done it at least 3 times. When there were issues, we properly uninstalled/removed using the correct documented process and tried again.
    They are running, they are accessable, they can be "pinged". When I use VMWare's interface/console and shut one down, I get an email stating it is offline.

    When a client or other "entity" can't communicate with one, it shows a log entry stating it can't communicate with it.
    So they are up, they are running.

    HOWEVER, they are doing nothing.

    I get the status unknown in the console, and there is never any incrementing of the stats in vshield or the SVA itself. 0 accesses. They aren't being utilized.
     

    Here are screenshots to show what we have, how it's configured to a point, and what I mean by certain errors. I can supply much more if/as needed.
    Bottom line - after 8 months of trying, we can't make these things do anything other than sit there taking resources and looking pretty.
    This one shows that I have the proper settings enabled in SEPM - telling the AV settings to use the shared insight cache via vShield.


    sepm-sva-setting.png

    This one shows that the VDI clients/desktops are alive, online, status good in SEPM, they ARE communicating exactly like physical desktops as far as SEPM - as far as policies, management, etc. This also shows the current SEP version on the VDI desktops, and current date and time showing they have been updating their "status" to SEPM just fine. No problems at all there. The ONLY problem is lack of use of the SVA.

     

    vdi-sva-status-versions.png

    This shows that the properties of the client as SEP sees it is correct - it is virtual, it is VMWare, ah, but the SVA is "unknown".

    sva-unknown-vmware-known.png

     

    And this is what the SVA looks like in the VMWare console - (sorry but due to our nature I had to redact some things for security and privacy,etc. but this shows the meat of the beast and what's needed)

    sva-vm-info.png

     

    So, what's up?

    Unless you have new info - please don't repost the same links that were in those other pages or threads. I always do extensive and exhaustive searches before posting here. If there is something new since say last August, or later, that's different. I GAVE UP as making these work seemed hopeless - no one has any good info at all, no details, including VMWare - and Symantec themselves. It's almost as if this was a purchased product and not fully documented as so far few people know squat about the Symantec SVA - and the info you do find is totally incorrect! I still see VM posts that state the SVA off-loads scanning and so on - direct from VMWare people right in their KB docs!! WOW, they don't even know their own partner.

     



  • 2.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 20, 2014 11:47 PM

    I've never used the SVA personally because I've always considered it unnecessary, but can I just ask how many VDI clients you're trying to service with it?

    Just curious as to why you wouldn't use the simpler network based SIC. If it's a large environment I can understand the reluctance in using network based SIC (increased network traffic).

    I've also heard that the SVA component isn't something VMware/Symantec are planning on using in the future (I might be misinformed though) which might explain the limited documentation available.

    Have you logged a job with Symantec Support?



  • 3.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 21, 2014 08:51 AM

    Right now just 2 or 3 to try to MAKE IT WORK. We have not gotten into VDI yet short of trying to figure it out and make it functional in testing. Numbers are irrelavant at this time. Doesn't matter. 2 or 10 or 100. None work. We've had 6 VDI desktops running at a time, and we are down to 2 now - none work.
    Traffic is of no concern as there's no network traffic involved in VDI other than what goes out to the field offices which is almost 100% PC over IP protocols, screen/mouse/keyboard. VDI is all under the host and at backplane speeds. It's for the most part over a virtual network and virtual switches. Even the SVA is under the same host as the VDI desktops so there's no network traffic. If we did have a shared insight cache, which I understand is a totally different animal, still no network traffic as it would be virtual too, under the same hosts.

    It's my understanding that the SVA is different than what I assume you are referring to as "Shared Insight Cache" as the SVA is for SCANS and functionally different - although info is sketchy at best.

    ***But the primary reason we need the SVAs to functin is because our admin believes we really must have something otherwise he believes SEP will kill VDI clients/hosts once we are in full production down the road, and frankly although I don't really agree with his reasoning, it's nonsense to have SEP doing it's full thing on all the VDI desktops when risks are more limited there and it's a shared environment. Why have 10 or 300 VDI desktops all scan the same file on the same day if it's clean, it's clean and 1 computer can tell us that, thus the SVA.

    Network traffic? how? It's all under the VM host servers, there's no network traffic to an SVA or even the shared insightcache as VDI desktops run inside a server, not across a network, as would any virtual appliance.(assuming as I have no real idea what a "SIC" is as far as machine, service, etc.)

    Dropping it? How or why would they not use an SVA - McAfee is already light-years ahead in this game fully off-loading things to a virtual appliance off of the clients. 
    the SVA is part of VM's vision for the future - take it off the virtual clients and move it to an appliance.  MAYBE you mean in Symantec's version which is just a glorified cache of client scan info, they don't plan oin moving it forward - first to scan shares that info so the others don't need to scan. It's not insight really, it's the AV scanning. It's not downloading or dependend on what others out there have downloaded, it's sharing info on files in YOUR own environment. If several of us have a file called YYY.EXE, and 1 scans it and finds it clean, it shares that info with the rest so the other 300 don't scan it with the same defs.
    I don't believe a SIC is the same thing.

    What I read recently in a blog done by a VMWare employee person that there are plans to take Symantec's SVA further and actually off-load some client functions to a virtual appliance. I don't know, doesn't matter - it's the here and now I must make work - those are my orders.

    Doesn't matter, my question isn't about other devices or the future of SVA - my question is how to make this function. Symantec says it will, VMWare says it will, but there's no real info out there.
    VMWare is terrible about furnishing any details - even when setting up VDI to begin with they say "you should do this" or "you need to set that" but there's no clue as to where or how! There are no blueprints, just vague references in both the VMWare docs and Symantec's documents.
    The VMWare documents are full of holes regarding VDI anything and we've been almost a year just trying to make a stable desktop and stream apps with thinapp.

    Symantec just doesn't seem to care about the SVA - or they don't know anything about it themselves, which is what I am starting to believe - they purchased all this technology and aren't clear as to how to make it work. I've been 8 months in asking - with no solution to date.

    - doesn't matter why, we just need to make it work. 

     



  • 4.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 21, 2014 09:23 AM

    Oh boy, you really like writing smiley

    try to run Run > Cmd> fltmc

    Do you see SymEPSecFlt driver?!



  • 5.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 21, 2014 09:35 AM

    One more thing, if you want to use it for non-persistant VDI, SVA is not the solution. VIE is your savior.



  • 6.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 21, 2014 10:04 AM

    I'm a technical writer at times...  wink  ... and feel the need to explain when asked things already outlined before   surprise.  

    Interesting on the commant fltmc - have not seen that before. Will try (when we get a couple of the VDI desktops up again - VMWare is also causing us a lot of troubles!)  I'll check that to see if that driver is there as soon as another VDI desktop is available and running properly.  Thanks for that tip - that's one to print and save in my SEP document folder.



  • 7.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 21, 2014 10:11 AM

    Please explain - why do you say that, and what is VIE - alphabet soup seems to be the norm these days but if you aren't using VIE, you might not know what VIE is (such as in my case! HA  smiley )


    First rule of journalism with acronyms and abbreviations - use it once in a sentence fully spelled out, then refer to the acronym, then use the acronym from there on.  laugh
    Trouble with Security Virtual Appliance (SVA)   (now I can use SVA from here on.........)

    We will be running linked clones - the machines will "be there" and in active directory sitting and idling ready to be logged into. This way the machine doesn't have to be built and joined to the domain and configured while the user waits for a desktop - there will be some ready to go. At least that's my feable understanding of VDI - obviously you know a bit more than I on that topic!  I prefer my feet to be planted on solid real ground I can touch and feel. But for all I know, there are no real people and never have been, we're all virtual and exist in someone's strange imagination.



  • 8.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 21, 2014 10:26 AM

    As you already using SEP, I assumed you know VIE :-)

    VIE stands for Virtual Image Exception. It means, once you create the master image for VDI, you scan it, make sure it is clean, then exclude all from being scanned again via VIE (a separate tool from Symantec).

    I have worked with VDI with SEP12.1, and believe me, it is not easy to make it all work, especially when it comes to updating SEP definitions.

    for VDI, I recommend you to:

    • use VIE
    • disable scheduled scans
    • increase heartbeat
    • plan how you will keep SEP definitions per cloned Virtual Desktop. If the cloned VD will go back as it was before (non-persistant)


  • 9.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 22, 2014 03:02 PM


    Ok, momentary lapse of acronym memory. Yes, we use the VIE tool and the last minute clone prep as well.
    There is no way to "rely" on VIE as the saving feature. Yes, you can "certify" the basic image - but there's always the fact that they are "on the network" and anything downloaded via web, link clicked in email, email attachment, access to their personal email via web interface or some other means won't infect their profile area and get to the network or across shares, etc. Face it, today's risks target the person, the user profile area as that's a free-for-all thanks to MICROSOFT. To make it simple for dummies to run a computer and make the computer a do-it-all as simply as possible without interference, they allow anything and everything to run in the profile area with no real restrictions. Even APPLICATIONS now install into the profile and not the programfiles area like they should. Examples - how about just about any GOOGLE app - especially the Chrome browser - in the user profile area. How about toolbars, add-ins, plug-ins, or webinar and meeting software - installs under the unprotected user profile where it can run and grab email addresses, content of the Outlook mail box including addresses, can set up a service to send mail/spam, can cruise through documents and gather info and send it back to a waiting web server - all from the user profile area which is pretty much the same be it a real computer or VDI. It's also been shown that VDI desktops are vulnerable in other ways, including the ability to hammer or break hosts since the processors share certain instruction sets regardless of how you THINK they are isolated. It's been demonstrated that although in the traditional ways VDI is more secure, in other ways it's not secure, and because too many BELIEVE it to be VERY secure, it's less secure as folks don't think it needs to be watched. But just click on that bad link or be redirected by an ad and see what happens even with VDI. If SEP wasn't there, the network data or documents or email is at risk. Being a public agency with confidential information that includes taxpayer info, personal medical records, and information obtained directly from the SSA themselves, we must be as secure as technically and humanly possible. We are audited for such security measures. We must audit who has access to information, when and from where. Without SEP running in force, it's not all that easy to do, thank you Microsoft.
    And just like the Apple fans once thought "what, me worry"? VDI admins and fans will be the cause of the first real VDI attack just because they made it popular enough to be worth attacking. The bad guys don't waste their time - but once it's no longer a waste of time, they'll be there. And any personal info that can be accessed via a profile will be vunlerable. I'm not at all concerned with INFECTIONS these days, that's so old-skool, I'm concerned about theft, corruption, etc. of PII. And that is vulnerable just as much under VDI as a real computer.|
    All that being said - I need SEP working to its fullest ability protecting the vulnerable user profile against malware or attacks that are aimed at theft more than destruction. When you see that end-users are taking their annual security tests in IE in one tab while shopping and clicking ad links in another tab, well...... VDI is just as vulnerable and SEP must be there with full current defs.

    Scheduled scans will be at "off-hours" and will scan only high-risk areas. SEP also allows off-set and detects the VM world around it so won't storm the hosts. I may get to the point of NO scans at all but our users are EXTREMELY high risk. In fact if it wasn't for my collection of custom IPS and my application and device control coupled with some tight SEP configuration we'd be hammered at least 2 or 3 times a year.
    As it is I can easily claim 3 full years - yes, over 36 months with NO INFECTIONS of any sort (other than the boss getting shareware infested with toolbars, BHOs and other garbage, but I took care of that as well) Not too many organizations can state their policies with and in SEP have kept any true malware or viruses out.

    I have yet to see where anyone has ever made reference to SEP and VDI as far as definitions. I've not seen it mentioned, talked about or documented so that bridge has not yet been crossed. I haven't found any Symantec documents on that topic yet.  With our very very small test environment currently in place where we have 2 or 4 virtual desktops/computers going at a given time, I can't even tell what's happening with them at this time. The admin makes and destroys them, and there's little real chance yet to observe how SEP is reacting other than to state they communicate with SEPM, they are online when they should be, and defs always seem to be current. I suspect that due to the fact that the SEPM servers are virtual and under the same hosts SO FAR (that will change, it will have to!) that the defs updates takes place SO fast across the backplane or inside network we never notice it happening.

    I will be checking logs later today, though.

    Oh, here is the result of the test you suggested if I did it correctly -

    driver.png

     

    Heartbeat already increased over what it was/is for physical computers.



  • 10.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 23, 2014 04:24 AM
      |   view attached

    You know, VIE is to exclude the whole image from being scanned. And anything comes new or modified, it will be handled by Auto-Protect and SONAR. From experience, the main issue you will face, is updating definitions. I had to use a workaround and it is listed here:

    https://www-secure.symantec.com/connect/articles/sep-non-persistent-vdi#comment-9721741

    From Symantec side, they a white paper for virtualization (attached) where VDI are mentioned in section 5.

    For SVA issue, it seems the endpoint driver is installed but still not reporting the SVA properly. I would suggest to increase the logging for SVA and see if you have any errors here. You can refer to Admin guide, they have a whole section for troubleshooting.

    Attachment(s)



  • 11.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 23, 2014 08:23 AM

    I've seen the whitepaper IF that is the same one. Quite generic, really. It's ok, but not specific, simply covering generic settings you can change or tweak. I will review it again though just in case I missed something or you posted a better or newer one so thanks for the reminder and attachment.

    VIE excludes the basic gold image. Yes, but that is it. That's the OS only, period,. That's not apps, that's not profiles which these days comprises a large part of any "desktop". Much of the activity and recording is done in the user profile area or in the "current user" part of the registry. Unless those are protected by SEP or some other means, that's where the real risk is. I have SEP pay little attention to the OS even on physical images and real computers as most modern risks don't even mess with the OS itself. It's too bloody simple to access and leverage the user profile area which is a free-for-all and security nightmare.

    Install Chrome or webinar/meeting software and watch closely where it goes, what it does. Watch for near-misses on malware - where does it go, what does it access or try to access? NOT the core OS or core OS files at all - it stays in the wide-open user profile area!  %UserProfile%\ and the localappdata, roaming and locallow areas. There's a lot of stuff there and that's where a lot of modern apps including malware run from these days. Not the core OS. So although I can use VIE to certify and exclude the core OS or gold image, that part wasn't as at risk anyway.

    I need to protect the rest - the apps which will be streamed and the user profile area and you can't possibly certify that. To exclude scanning/checking the user profile area is security suicide. It's the highest risk part of any system.

    The SVA is not reporting errors. The SVA says it's fine. The OS with SEP says it's fine - but there's still no interaction at all between them. If I take an SVA down, say I turn it off, I get an error email almost instantly from SEPM and the client/VDI desktop will log that it can't communicate with the SVA. Interesting indeed since it doesn't seem to know the SVA exists until I turn one of them off then suddenly it complains.

    I'll review the docs again but I've got, literally no joke, a pile of printed papers over 6" high on troubleshooting from Symantec and it's still too generic. And the "what you need to install an SVA is almost a joke.



  • 12.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 24, 2014 03:12 AM

    I agree with you, SVA is pretty new and not widely used. you don't find enough KBs or whitepapers about it. But from experience, if you open a case with Symantec, they will ask first for logs :-)

    and for VIE, do you prefer to scan always the whole image and personal profile/drive. Besides, you don't need to scan anything, except on-access attemps attempts.



  • 13.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 24, 2014 08:58 AM

    I do agree on the logs - one should never contact any tech support for any software until they have anticipated and prepared for the questions and requests. So if you make contact and they ask for any logs or info, you do not "play 20 questions" with a lot of back and forth, you immediately say "here are all the detailed logs and these are the specific settings we use". Saves them time, saves you time. Why waste it on back and forth when you know they need the information anyway. Present it right up front.

    VIE - we create the base or gold image and get it how we want it, then we scan and "certify" it - VIE, then the clone prep tool to remove all SEP serialization and shut it down.

    We don't have SEP scan the base image of any clone - any scans will be only for what is not part of the base image. User profiles are not part of the base image. We will be using Persona Manager and keep the profiles off on another drive when the user is not logged in. It's VMware's version of roaming profiles. When someone is logged in their profile area, which is not part of the base or gold image, will be checked just like on any real computer. But because VIE will have checked the base image, it will never be checked. SEPM policy is setup to tell SEP on the desktop to ignore that certified image. This is how I have it set for that group that will have the VDI desktops in it -

    vie.png

     

    sic.png

     

    (to back up just a bit - I am sure this is just like every other time I've run into something I was stumped on, it's going to be something simple I should have known about, or that I missed, or that someone else saw that I have not yet seen or found. I have seen cases where I do things with SEP others never use or didn't even know existed, and I know not a lot of people use VDI/VMware AND SEP, but there has to be someone!)
    At this point I truly believe it's a communication issue with VMWare itself - there is something missing or not set correctly and yet VM tech support says it's all fine. I knew they were wrong when I discovered the default VM HOSTS firewalls BLOCKED the traffic required for vShield!!! Yet they said "all is fine". Intil I UNblocked the VM host firewalls the vShield never populated any inventory at all! Now it does, but not well and not consistantly and not all of it. And I see sometimes it states the hosts are protected and in vShield manager itself it states the hosts are not protected. There is no SEP client communication with the SVA - and yet they know if an SVA is down or they cna't communicate with an SVA! How can it claim it's not communicating then at the same time tell in the logs it has lost communication with the SVA?

    IF I read the documents correctly - from Symantec and VM - the communication is NOT directly between SEP on a client and the SVA. Is that correct? Does it go through vShield or something VM first, or does it TRULY communicate DIRECTLY, one on one, VDI desktop SEP client right straight to the SVA? OR, does vShield act as an intermediary??? Do communications have to go through the VMWare stuff like vShield and not direct SEP client to SEP SVA?

    That's the part no one can state clearly or for sure.

     



  • 14.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 28, 2014 03:06 AM

    To answer you last concerns, the coomunication goes through vShield Endpoint driver on the SEP client host. That's I sugguest to check the driver in the first place. But you are right, a problem with VMware configuration could be causing this, but I have no knowledge with it. If SVA is working find, SEP client is working fine with the proper porlicy and the right drivers, then I am out of ideas :-)



  • 15.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 28, 2014 11:05 AM

    Well you tried and I did actually learn about how to check for a filter being loaded.....

    My only other question since you DO deal with VDI in general - how do you handle apps and user profiles?
    What is on your base or "gold image" as far as apps, etc. - and what about user profiles? How are those managed? I ask because many apps these days do install in the profile area as well as keeping settings for specific users in the profile. Outlook is an example, Mocha terminal emulation keeps a user-specific config file in the user profile area, and under application data, there is local, local low and roaming - ALL have user specific and app specific settings...



  • 16.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 30, 2014 09:54 AM

    To be honest, the profiles are not scanned unles something has been accessed. so if you start with basic profile, it should be clean and anything is added is scanned only, via auto protect. Also, usually, admins don't allow any user to install whatever they want.



  • 17.  RE: SVA not working or communicating VMWare VDI clients or VM management

    Posted Jan 30, 2014 10:17 AM

    With Windows, Profiles ALWAYS have something accessed. Check the %LocalAppData%\temp folder.

    Check the browser cache, check the Windows OS and browser history areas, unhide folders and you'll see that if there is someone logged in, it's a flurry of activity and file touches, always, constantly.

    Then check the user profile if you run Chrome or Firefox browsers - yup, if they open the browser, it's many hits in the user profile.

    Open Outlook or Word - again, this is all info swirling around in the user profile.
    Take part in a Webinar or online meeting - the software installs there, the meeting or whatever is logged there. EXE and DLL files are INSTALLED there. Terminal emulation packages such as Mocha - config files, settings, logs - it's in the user profile. Some profiles can reach a gig or more in size.

    Anything you do in Windows touches your user profile. In short, you can't even log in to Windows without the user profile changing. You can't log out with it changing. You can't install software of ANY sort even WEBEX or GoToMeeting without the profile having folders and files, some of them EXE and DLL files put there. And you DO NOT NEED ANY ADMIN rights for this. Get into a WEBEX session or an online meeting - check your profile. I bet you have installations there - even with no admin rights. That's because the user profile is wide-open for that user. "Anything goes" in the user profile. Create an email signature, choose a "stationary" for Outlook messages, set your defaults and preferences, it's in the user profile.

    Here is one of the most simple examples of a single user profile I can find - and if I dig into these folders, I find a lot of activity. The TEMP folder is LOADED with files, sometimes dozens, sometimes hundreds. If you have Outlook running, it logs there, if you have Symantec Vault, it logs there. Windows itself logs there. Look at the other folders - application stuff. The Citrix folder has EXE and DLL files.

    The user profile area is in a constant state of flux, always changing, always being accessed - always. Even Windows keeps registry files and profile DAT files in there. You have to unhide things to see them, but if you blocked access to anything here, you'd kill windows and the apps.

    So, I'm saying all that as THIS IS THE EXACT place that most malware today hits. Modern malware targets the user profile as you do not need elevated or admin rights to do so. If they click a bad link or ad, WHAM, THIS is exactly where it goes, at times installing to and running from the TEMP folder as all users have full rights there to do anything. And Windows doesn't keep it cleaned out.

    If you are not guarding this area carefully, you are remiss in your security.... every piece of malware I've seen in the last couple of years targets something here - something that is NOT locked in a gold image in VDI, and yet if the user clicks and gets malware in their profile area, say the temp folder, it has access to the network and files exactly as the user who is logged in has. And so what if VDI means the base is locked and reverts back each time of a logout... .the damage is probably already done as the malware installs SMTP or other services in the profile, runs it from there, and grabs all the personal, confidential info it can and sends it out from that safe VDI image. Infected? Possibly not - but data loss, data theft, identiy theft, spam, etc - ALL POSSIBLE from VDI, especially if the user profile is not protected and locked down by SEP and app control. Forget viruses or worms that install and must have admin access or rights to the program files area - that's antique old-skool stuff. Modern stuff targets what you see below - just like Chrome and Firefox browsers do. A user with NON-ADMIN rights can install that software. Malware only needs to get in and run for a few seconds. It doesn't care if the image is wiped every so often or locked in the apps area. The profile can't be locked down (without SEP and app control) so it's wide open. Malware gets in, does its stuff, perhaps totally messing up your network, and then leaves. No installs necessary, no admin rights needed.

     

    profile.png