[THIN] Re: Session hangs/resets
- From: "Jeremy Saunders" <jeremy@xxxxxxxxxxxxxxxxxxxx>
- To: <thin@xxxxxxxxxxxxx>
- Date: Fri, 1 Jul 2011 21:51:11 +0800
Hi Ang,
I don't think you've understood my questions on the NAS. I know what a NAS
is, but you can get them as different OSs, such as Windows as the OS running
Windows Storage Server 2003 or 2008, meaning that you still join it to your
domain and log in and manage it just like a normal Windows server. Anyway.we
digress.
I assume from your answer you're using something like a NetApp Filer. Whilst
it may be underutilised and your NAS admin says there's nothing wrong with
it, everything will look normal if you're not looking in the right place.
Read something like this:
http://netappsky.com/storage-management/cifs-performance-is-slow-after-inves
tigating-performance-issues-cifs-considerations/
There is definitely a problem\bottleneck with SMB, which is killing the box.
The thing is, you can change those parameters on the Citrix servers and get
more aggressive, but unless the NAS is setup correctly, it may be to no
avail.
Refer to this KB article for tuning the Citrix servers:
http://support.microsoft.com/kb/324446
Cheers,
Jeremy.
From: thin-bounce@xxxxxxxxxxxxx [mailto:thin-bounce@xxxxxxxxxxxxx] On Behalf
Of Angela Smith
Sent: Friday, 1 July 2011 1:39 PM
To: thin@xxxxxxxxxxxxx
Subject: [THIN] Re: Session hangs/resets
Hi Jeremy
Thanks for the detailed email..
We are using Windows 2003 Enterprise (32bit). Servers have 10GB RAM, Dual
Quad core CPU's
File Storage is on a NAS Appliance (Network Attached Storage) so its an
appliance (not Windows based).
Citrix Server
MaxMpxCt is set to 1024
MaxWork Items is set to 4096
NAS
CIFS MPX is set to 1124
Perfmon results
Redirector\Current Commands: Min 155, Max 246. Average 170
Server\Work Item Shortages: 0
Server Work Queues\Available Work Items: Min 30, Max 30, Average 30
As you can see from above The Redirector Current Commands are quite high.
Our NAS Admin says there is no issue on the NAS side. What can I do to fix
this on the Citrix Servers?
Thanks
Ang
_____
From: jeremy@xxxxxxxxxxxxxxxxxxxx
To: thin@xxxxxxxxxxxxx
Subject: [THIN] Re: Session hangs/resets
Date: Thu, 30 Jun 2011 18:45:09 +0800
Hi Ang,
Ronnie has a good point. I know all about Notes 8.5 as I used to work for
IBM J
Notes 8.5 also uses heaps of memory. Are you using 2003 Enterprise or
Standard? 32-bit or 64-bit?
As for the SMB tuning, what is the NAS? Is it still a Windows 2003 or 2008
server?
On the Citrix server you can use PerfMon to monitor the following:
Redirector\Current Commands: Current Commands counts the number of requests
to the Redirector that are currently queued for service. If this number is
much larger than the number of network adapter cards installed in the
computer, then the network(s) and/or the server(s) being accessed are
seriously bottlenecked. The number indicated by the Current Commands counter
indicates the number of pending commands that are waiting to be serviced by
the local computer. This is the number of pending commands from the local
computer to all destination servers. This means that if one of the
destination servers does not respond in a timely manner, the number of
current commands on the local computer may increase. The goal of a
Terminal/Citrix Server deployment is to keep the Current Commands value to
about 60 on heavily utilised servers. A Current Commands value of 80 means
that on average you've got 80 pending SMB/RPC network I/O commands in the
queue. With standard out-of-the-box tuning, it means that you're probably
getting pretty close to the edge at times. When the pending requests queue
is full, things stop (pause) momentarily until the queue has free slots. If
settings such as the MaxMpxCt and MaxWorkItems values are not increased on
the file server(s), anything at all that slows down your file server will
potentially impact on your Terminal/Citrix Servers. Tuning MaxCmds on the
Terminal/Citrix Server end won't provide any enhancement unless the file
server end is increased as well.
On the file server, assuming it's a Windows server, you can use PerfMon to
monitor the following:
Server\Work Item Shortages: It represents the number of times no work items
were available or couldn't be allocated to service a file request. It
indicates whether or not the InitWorkItems or MaxWorkItems parameters need
to be adjusted. If you see any other value than zero, you need to start
worrying. Upping the InitWorkItems or MaxWorkItems could help out here.
Server Work Queues\Available Work Items: Shows the instantaneous number of
available work items per CPU. A work item is typically a request from a
client to the server; the server maintains a pool of available work items
per CPU to speed processing. Sustained values smaller than ten means that
the File server is running out of work items. When it does, performance
really starts to plummet. If it reaches zero, clients may be unable to
access files. Make sure this doesn't happen by upping the MinFreeworkItems
value.
There are a lot more things you can monitor on the Citrix servers, but this
is a start. If you are experiencing an SMB bottleneck, I would expect to see
a high number of Current Commands.
Hope that helps.
Cheers,
Jeremy.
From: thin-bounce@xxxxxxxxxxxxx [mailto:thin-bounce@xxxxxxxxxxxxx] On Behalf
Of Hamilton, Ronnie
Sent: Thursday, 30 June 2011 4:19 PM
To: thin@xxxxxxxxxxxxx
Subject: [THIN] Re: Session hangs/resets
Hi Angela,
I remember looking at issues relating to many open files and needed to do
some SMD tuning at the time
<http://blogs.citrix.com/2010/10/21/smb-tuning-for-xenapp-and-file-servers-o
n-windows-server-2008/>
http://blogs.citrix.com/2010/10/21/smb-tuning-for-xenapp-and-file-servers-on
-windows-server-2008/
This blog might give you an idea, It's for 2008. I was using server 2003 at
the time but don't have exact MS article at hand
Ronnie
From: thin-bounce@xxxxxxxxxxxxx [mailto:thin-bounce@xxxxxxxxxxxxx] On Behalf
Of Angela Smith
Sent: 30 June 2011 08:45
To: thin@xxxxxxxxxxxxx
Subject: [THIN] Session hangs/resets
Hi
Im having an issue on my Windows 2003 Xenapp 5 Farm where Sessions keep
locking up and just disconnecting, event logs say Event ID 50, and 26 for
delayed write failed. This is particularly noticable with Lotus Notes 8.5
client which relies heavily on File Server. I've been doing research on
this all last week with no luck at all. File server hosting user home
drives/profiles is on the NAS. NAS is underutilised and Citrix Servers are
all connected to Gig network again which is underutilised.
No other applications get errors except for Lotus Notes, and when it crashes
all of their applications crash. It just crashes the user's entire
connection. There is no latency that I can see
Has anyone seen this issue before? Ive had to reset over 300 connections in
1 month because of this issue. We have Session reliability enabled.
Thanks
Ang
Visit our website : <http://www.ltai.ie> www.ltai.ie
__________________________________________
Lufthansa Technik Airmotive Ireland Limited. Registered in Ireland. Reg. No.
45999. Registered Office: Naas Road, Rathcoole, Co.Dublin.
Lufthansa Technik Airmotive Ireland Leasing Limited. Registered in Ireland.
Reg. No. 140891. Registered Office: Naas Road, Rathcoole, Co.Dublin.
__________________________________________
The information in this email and in any attachments is confidential and may
be privileged. If you are not the intended recipient, please destroy this
message, delete any copies held on your systems and notify the sender by
return email. You should not read, retain, copy, disseminate, distribute,
disclose or use this email or its contents in any way. Any such action is
strictly prohibited. Thank you.
Other related posts: