Citrix Xenapp 7.6 Logon Slow Long Black Screen Phase

Citrix XenApp 7.6 Logon Slow – Long Black Screen Phase by Helge Klein on April 27, 2015 in Citrix / Terminal Services / Remote Desktop Services

Update 2015-04-28: Citrix provides the limited release hotfix ICATS760WX64009 that fixes this issue. More information below.

During the research for my session about the XenApp 7.6 logon process, to be presented at Citrix

Synergy and BriForum London, I noticed that the logon to my XenApp 7.6 lab server was taking a bit long. Longer, in fact, than the combined durations of the main logon phases user profile loading, group policy processing, logon script execution and shell startup. Much longer. Also much longer than on an otherwise similar XenApp 6.5 machine.

Examining uberAgent’s brand-new logon process performance dashboard I found a 10 second gap:

After the user profile has been loaded and group policy has been processed, which are both finished after three seconds, nothing happens for about ten seconds. Finally, at 13 seconds into the login is the Userinit process started which ultimately loads the shell, Explorer.exe. During most of that time the end user is presented with a black screen, getting no feedback as to what might or might not be happening as you can see in this video: I experiemented a lot trying to find the root cause of the prolonged black screen phase, disabling all kinds of redirection, printer mapping, even IPv6. None of that fundamentally changed the duration of the black screen phase. Then I had the idea to log in via RDP (for which you have to make the user a member of the group Direct Access Users on the XenApp machine):

Now, that is a lot faster! RDP logons are finished in less than 4 seconds compared to around 14 seconds for ICA logons.

Digging around some more I stumbled upon CTX135782 which suggests setting the following registry value:

Key: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Citrix\Logon Value: DisableStatus Type: REG_DWORD Data: 1

The effect was immediate, bringing the ICA logon from 14 seconds down to around 5. Happy End?

Not quite. With DisableStatus set to 1, every time I logged on the LogonUI.exe process crashed due to a fault in the module CtxWinlogonProv.dll. Events like the following were logged:

Faulting application name: LogonUI.exe, version: 6.3.9600.16384, time stamp: 0x5215f6c5 Faulting module name: CtxWinlogonProv.dll_unloaded, version: 7.6.0.5018, time stamp: 0x541cfd05 Exception code: 0xc0000005 Fault offset: 0x00000000000019e1 Faulting process id: 0x1e2c Faulting application start time: 0x01d080da8b0d6ae2 Faulting application path: C:\Windows\system32\LogonUI.exe Faulting module path: CtxWinlogonProv.dll Report Id: cb285960-eccd-11e4-80c0-00155d001b10 Faulting package full name: Faulting package-relative application ID:

CtxWinlogonProv.dll, also called CitrixRemoteLogonFilter, is registered as a credential provider filter via the registry keyHKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Authentication\Credential Provider

Filters\{3571A91D-713C-427c-AA0C-BBF4F618A819}. When that key is renamed or deleted LogonUI.exe does not try to load CtxWinlogonProv.dll and consequently does not crash any more while logons are still fast.

However, since I do not know what the consequences of disabling the registration of CtxWinlogonProv.dll are I cannot really recommend that approach. Ultimately Citrix are the only ones who can truly fix this. Update 2015-04-28: a Hotfix

Shortly after the initial publication of this article Shane Kleinert pointed me to limited release hotfix ICATS760WX64009

(CTX142036) which seems to be available for server OS (i.e. XenApp) only. As he explains in the comments below

Shane had noticed similar issues at customer sites and found CTX142036 in spite of the more than vague problem description.

I am happy to report that CTX142036 fixed this long black screen phase issue on my machines, too. So this story does have a happy end, after all! https://helgeklein.com/blog/2015/04/citrix-xenapp-7-6-logon-slow-long-black-screen-phase/

For those wondering the hotfix is “ICATS760WX64009” This is specifically applied to the Server VDA, NOT the workstation VDA.

Installing uberAgent for Splunk is quick and straightforward. Due to the product’s flexibility there is often more than one way of doing things. In such a case we describe the recommended configuration here and link to supported alternatives. Prerequisites  Read about the architecture options.  If you do not have a running Splunk installation yet set up Splunk.  Download uberAgent and extract the contents of the archive. You should have a directoryuberAgent components with the following content: o A subdirectory uberAgent_endpoint o Packed Splunk app uberAgent_indexer.tgz o Packed Splunk app uberAgent_searchhead.tgz Preparing the Splunk Server Manual Installation  Go to the Splunk console’s home page by navigating to http://servername:8000 in your browser.  Click Manage apps:

 Click Install app from file:

 Select uberAgent_indexer.tgz and click Upload  The uberAgent indexer app is now installed  Install uberAgent_searchhead.tgz in the same way  Click Settings -> Server controls -> Restart Splunk Distributed Splunk Deployment If you have a distributed Splunk deployment with separate search heads and indexers please deploy the indexer app to all indexers and the search head app to all search heads.

Alternative Architectures Note: This is optional and not required for the recommended architecture. If you decided to have uberAgent send data to Splunk through a locally installed Universal Forwarder on the monitored endpoints you need to enable receiving Universal Forwarder data as described here, i.e. through Settings -> Forwarding and receiving -> Receive data -> Add new -> 9997 -> Save. Installing uberAgent on the Endpoints The agent installer is available as an MSI package. The MSI can either be installed manually or unattended through existing software deployment tools or Splunk’s Deployment Server.

Expected result after the installation of the MSI: the service uberAgent is installed and running. Manual Installation  Run the batch file uberAgent_endpoint\bin\manual-install.cmd  On the screen Receiver Configuration specify the name(s) of your Splunk indexer(s) and the port configured earlier (default: 19500) Installation Through a Software Deployment Tool  Install the appropriate MSI file from the directory uberAgent_endpoint\bin depending on the bitness of your machine: uberAgent-32.msi or uberAgent-64.msi  Specify the following MSI parameters: o INSTALLDIR [optional]: installation directory o SERVERS [required]: comma-separated list of receiving servers/ports. o Example:

o SERVERS="splunk1:19500,splunk2:19500"

Alternative Architectures Note: This is optional and not required for the recommended architecture. If you decided to implement one of the alternative architectures you need to install Universal Forwarder on each endpoint. In order to deploy uberAgent through Splunk’s Deployment Server follow these steps. Alternatively, if you want to run uberAgent in legacy mode (not recommended) follow this documentation. Configuration uberAgent can be configured very flexibly. By editing the configuration file you can switch metrics on or off, change the data collection frequency and significantly reduce the data volume. License File If you have a license file for uberAgent copy it to the installation directory (default: C:\Program Files\vast limits\uberAgent). Without a license file uberAgent displays a splash screen during logon. Contact us for an evaluation license. Imaging / Citrix PVS If you intend to copy the agent installation via an imaging method or Citrix PVS we recommend you remove instance-specific information. To do that follow these steps right before capturing the image:  Stop the service uberAgent (but leave the start type at automatic)  Open an administrative command prompt  Run the command: reg delete “HKLM\SOFTWARE\vast limits\uberAgent” /f /reg:64  Prepare the machine for cloning as necessary, but do not reboot If you have Splunk Universal Forwarder installed, please follow the steps listed here, too. Monitoring uberAgent The operation of the uberAgent service can be monitored by checking the log file. Centralized monitoring is possible through the Splunk app uberAgent Log Collector. All Done! You have successfully deployed uberAgent. Now take a look at the dashboards by pointing your browser athttp://servername:8000/app/uberAgent. Have fun!

If you do not see data from some or all endpoints please follow these troubleshooting steps. Questions?

Do you have questions that were not answered here? Please ask us, we are happy to help!

https://uberagent.com/documentation/install-and-deploy-uberagent/

Hotfix ICATS760WX64009 - For VDA Core Services 7.6 for Windows Server OS (64-bit) - English

Software Updates | Public | 16 found this helpful | Created: 10 Apr 2015 | Modified: 03 Aug 2015 Languages N/A Download

ICATS760WX64009.msp

Was this helpful? Yes No

Filename: ICATS760WX64xxx.msp

For: XenApp 7.6; XenDesktop 7.6 VDA Core Services for Windows Server OS (64-bit). or operating system

requirements, seeSystem requirements on the Citrix Product Documentation site.

Replaces: None Date: November, 2014

Languages supported: English (US), German (DE), Spanish (ES), French (FR), Japanese (JA), Simplified

Chinese (SC)

Readme version: 1.01

Readme Revision History Version Date Change Description 1.01 May, 2015 Updating for general availability 1.00 March, 2015 Initial release

Important Note(s)

 This release is based on XenApp 7.6/XenDesktop 7.6 VDA Core Services for Windows Server (64-bit).

It patches only systems where Version 7.6 of the product is installed. Any known issues in Version 7.6 of

the product, except for the specific issues resolved in this limited release, still apply.

 You can upgrade the VDA from a console session or from an RDP session. Upgrading from within an

ICA session is not supported.

 Caution! This release may require you to edit the registry. Using Registry Editor incorrectly can cause

serious problems that may require you to reinstall your operating system. Citrix cannot guarantee that

problems resulting from the incorrect use of Registry Editor can be solved. Use Registry Editor at your

own risk.

Where to Find Documentation

This document describes the issue(s) resolved by this release and includes installation instructions. For additional product information, including supported operating systems and system requirements, see Citrix Product

Documentation.

New Fixes in This Release

1. When starting a published application, users can experience slow logons.

[From ICATS760WX64009][#LC0596]

Fixes from Replaced Hotfixes

No hotfixes were replaced by this release.

Installing and Uninstalling this Release

Notes:  This release is packaged with Microsoft Windows Installer 3.0 as a .msp file. For more information

about deploying .msp files, see Microsoft article 884016 or visit the Microsoft Web site and search on

keyword msiexec.

 This installer program complies with Microsoft User Account Control (UAC). If UAC is enabled, you

must run the installer program in elevated mode; that is, with administrative privileges enabled. For more

information about UAC, see Microsoft TechNet or visit the Microsoft Web site and search on keyword

 To install this release successfully, computers must not have registry modification restrictions in place.

 Slipstreaming - installations of the VDA for Windows Server OS base product packaged with one or

more individual hotfixes - is not supported and might leave servers in an unstable state.

 You must restart the server after the installation of this hotfix completes. However, if you are installing

multiple hotfixes at the same time, there is no need to restart the server after the installation of each hotfix.

It is sufficient to restart the server after the installation of the final hotfix completes.

 If the need arises to restore the original settings and functionality provided by this release, you must

uninstall the release before reinstalling it according to the installation instructions below.

To install this release:

1. Copy the release package to an empty folder on the hard drive of the computer you want to update.

2. Close all applications.

3. Run the executable.

4. Restart the computer.

To uninstall this release:

1. From the Start menu, select Control Panel > Programs and Features.

2. Highlight the release you want to uninstall and click Uninstall.

3. Follow the directions on-screen.

Files Updated (All Dates/Times UTC)

File Name Date Time Size

CPatch.exe 01/26/2015 13:42 338,288

CtxHfLoader.dll 01/26/2015 13:42 395,648

ctxregasm.exe 01/26/2015 13:42 26,576 msi50ca.dll 01/26/2015 13:42 633,184

SDKv06_RunTime_Xml 01/26/2015 13:42 334 statui.dll 01/26/2015 13:42 103,192

CPatchUI.dll (x32\de) 01/26/2015 13:42 14,200

CPatchUI.dll (x32\en) 01/26/2015 13:42 13,688

Error.idt (x32\en) 01/26/2015 13:42 35,673

CPatchUI.dll (x32\es) 01/26/2015 13:42 14,712

CPatchUI.dll (x32\fr) 01/26/2015 13:42 14,200

CPatchUI.dll (x32\ja) 01/26/2015 13:42 12,152

CPatchUI.dll (x32\zh-cn) 01/26/2015 13:42 11,640

File Name MD5 Checksum

ICATS760WX64009.msp 992F0BEC3BFA0FCC24DA274F479AE857

CPatch.exe B552C4ADFC4EBDBC198EB2620F282709

CtxHfLoader.dll 3342F05F9EE6F862172AC3150F804149 ctxregasm.exe 07207F40A7704D248E14594672B72CC4 msi50ca.dll C0E3ED1736A69FE8B6D6EBC6E5F2421E

SDKv06_RunTime_Xml 02C86CD1A05CE7F5BF41291C7AA6B595 statui.dll 8976157104431E08A2530F1026D0192C

CPatchUI.dll (x32\de) E4F3BD678E21553FD6B3BB48F2BDB297

CPatchUI.dll (x32\en) 1D5BC99B2A180E49381C086804B70F69 Error.idt (x32\en) 8370B4898257DD82C2D81D3EF1BE893E

CPatchUI.dll (x32\es) EAFEEF8AAA7BEBE89139D1AA0150A5B5

CPatchUI.dll (x32\fr) E2AC887A998BBC67CC3BEC4A7705EECB

CPatchUI.dll (x32\ja) A86BB78D0F01E8A8D82868E7B3FFF9E5

CPatchUI.dll (x32\zh-cn) 4D5B49F6F74FAF217C543D21DBCD3F91