[ExchangeList] Re: Messages stuck in "local delivery" queue

  • From: James Chong <jchong@xxxxxxxxxxxxxx>
  • To: "exchangelist@xxxxxxxxxxxxx" <exchangelist@xxxxxxxxxxxxx>
  • Date: Thu, 20 Nov 2008 09:49:34 -0500

http://www.msexchange.org
-------------------------------------------------------As far as your question 
on what incurs more IO large file vs. smaller files; in essence it's not the 
size of files or whether you actually break it up; it's the IO pattern. I 
passed this question along to John Fullbright MVP who's much more knowledgeable 
in storage than I am. Here's his explanation:


"That's a loaded question.  
 
Each piece of data has metadata associated with it.  If I have one large piece 
vs. several small pieces, arguably the large piece has less metadata.  On this 
level, copying many small files the sum of which equals the size of one large 
file would still incur the overhead of the additional metadata.  In reality the 
metadata only adds up to a few percentage points.  
 
When I look at the total data transferred, the difference isn't much.  What is 
drastically different between the twoi scenarios is the IO pattern.  If I copy 
one large file then the IO pattern is a mostly sequential one.  If I copy a set 
of files the sum of which is the same as that one large file, then as the 
number of members of the set increases the IO pattern becomes less sequential 
and more random.  This assumes that I have multiple concurrent streams.  If I 
copy the files one after another, then the IO pattern is still mostly 
sequential.  
 
Disks have far better throughput when writing sequential data than they do when 
writing random data.  This is mainly because of the difference between the 
random seek time and the track to track seek time.  While it takes about 5ms to 
position the head for a random seek, it only takes about half a ms for a track 
to track.  Modern disks pack anywhere between 128 and 1000 sectors per track 
depending on where exactly the track is located on the platter.  A purely 
sequential write workload presents the sectors to a disk in LBN order which can 
easily be staged and written with a minimum of head movement.  As the workload 
becomes less sequential and more random, this becomes more and more difficult 
to do.  Add to that the problem of overwrites, in most systems if the data is 
overwritten it is physically overwritten, and increased randomness of a 
workload results in significantly longer IO times.  A disk may be able to 
sustain 1000 8K IOPS at a 20ms response time for a purely sequential workload, 
but the number for random workloads looks more like 160.  If you look at the 
amount of data written in the two scenarios, it's not that much different.  If 
you look at the time it takes to write the data for the two scenarios, there's 
a large difference."


Most high activity users in exmon culprits are:

1. User downloading ost file
2. User has blackberry
3. User has desktop search indexing software or other Outlook third party plug 
ins could be Outlook spam software etc
4. User has large mailbox\folder with many items 5000+


Hope this helps. 

James Chong
Sr. Systems Engineer
Simplexity, LLC.
11130 Sunrise Valley Drive, Suite 300
Reston, VA 20191
O (703) 657-4612
C (703) 863-1483

-----Original Message-----
From: exchangelist-bounce@xxxxxxxxxxxxx 
[mailto:exchangelist-bounce@xxxxxxxxxxxxx] On Behalf Of Nimori, Nathan
Sent: Wednesday, November 19, 2008 3:30 PM
To: exchangelist@xxxxxxxxxxxxx
Subject: [ExchangeList] Re: Messages stuck in "local delivery" queue

http://www.msexchange.org
-------------------------------------------------------Thanks James.  

When using exmon, what fields would show IO usage?  There is obviously
bytes in and bytes out but what about packets or operations?  Our disk
is being slammed by IO and I sometimes wonder is it really the amount of
data being transferred or could it be lots of searches, indexing, etc
where people aren't necessarily transferring large attachments but are
just doing a lot of manipulation of e-mail.  For example, which would
make more IO - copying one large file from one disk to another or
thousands of small files equal to the large file.  

Users on the top of the list in bytes in and bytes out are always
changing.  I noticed that users in the top of the list of packets and
operations are usually the same bunch.

Thanks again.

___________________________
Nathan Nimori
Systems Engineer
East Side Union High School District

-----Original Message-----
From: exchangelist-bounce@xxxxxxxxxxxxx
[mailto:exchangelist-bounce@xxxxxxxxxxxxx] On Behalf Of James Chong
Sent: Wednesday, May 28, 2008 11:34 AM
To: exchangelist@xxxxxxxxxxxxx
Subject: [ExchangeList] Re: Messages stuck in "local delivery" queue

http://www.msexchange.org
-------------------------------------------------------Could be several
reasons

1. DSACCESS related issues. Turn up diagnostic logging.

2. Sites and services are misconfigured and Exchange is using a remote
GC for lookups

3. You are incurring additional IO for some reason. See if there is some
process running; some large mail sent etc. Run perform and check your
physical disk avg disk sec reads and avg disk sec write. If you're over
50ms sustained you need to investigate. Also can look into using exmon
to see if a particular user is incurring the additional IO

4. Turn up diagnostic logging msexchangetransport

5. Run Exchange best practice analyzer


James Chong
Sr. Systems Engineer
Simplexity, LLC.
11130 Sunrise Valley Drive, Suite 300
Reston, VA 20191
O (703) 657-4612
C (703) 863-1483


-----Original Message-----
From: exchangelist-bounce@xxxxxxxxxxxxx
[mailto:exchangelist-bounce@xxxxxxxxxxxxx] On Behalf Of Enoque Modesto
dos Santos
Sent: Wednesday, May 28, 2008 1:46 PM
To: exchangelist@xxxxxxxxxxxxx
Subject: [ExchangeList] Messages stuck in "local delivery" queue

http://www.msexchange.org
-------------------------------------------------------Hi, ExchangeList,

I'm facing a problem related to a very large amount of messages stuck in
my Exchange serve 2003 "local delivery" queue;  this is caunsig the
Server performance to slow down and delaying the delivery of my user
messages.

Has someone faced such a thing ?

Thanks in advance,

Emo.

-------------------------------------------------------
List Archives: //www.freelists.org/archives/exchangelist/
MSExchange Newsletter: http://www.msexchange.org/pages/newsletter.asp
MSExchange Articles and Tutorials:
http://www.msexchange.org/articles_tutorials/
MSExchange Blogs: http://blogs.msexchange.org/
-------------------------------------------------------
Visit TechGenix.com for more information about our other sites:
http://www.techgenix.com
-------------------------------------------------------
To unsubscribe visit http://www.msexchange.org/pages/exchangelist.asp
Report abuse to listadmin@xxxxxxxxxxxxxx

-------------------------------------------------------
List Archives: //www.freelists.org/archives/exchangelist/  
MSExchange Newsletter: http://www.msexchange.org/pages/newsletter.asp 
MSExchange Articles and Tutorials:
http://www.msexchange.org/articles_tutorials/ 
MSExchange Blogs: http://blogs.msexchange.org/ 
-------------------------------------------------------
Visit TechGenix.com for more information about our other sites:
http://www.techgenix.com 
-------------------------------------------------------
To unsubscribe visit http://www.msexchange.org/pages/exchangelist.asp
Report abuse to listadmin@xxxxxxxxxxxxxx 

-------------------------------------------------------
List Archives: //www.freelists.org/archives/exchangelist/  
MSExchange Newsletter: http://www.msexchange.org/pages/newsletter.asp 
MSExchange Articles and Tutorials: 
http://www.msexchange.org/articles_tutorials/ 
MSExchange Blogs: http://blogs.msexchange.org/ 
-------------------------------------------------------
Visit TechGenix.com for more information about our other sites:
http://www.techgenix.com 
-------------------------------------------------------
To unsubscribe visit http://www.msexchange.org/pages/exchangelist.asp
Report abuse to listadmin@xxxxxxxxxxxxxx 

-------------------------------------------------------
List Archives: //www.freelists.org/archives/exchangelist/
MSExchange Newsletter: http://www.msexchange.org/pages/newsletter.asp
MSExchange Articles and Tutorials: http://www.msexchange.org/articles_tutorials/
MSExchange Blogs: http://blogs.msexchange.org/
-------------------------------------------------------
Visit TechGenix.com for more information about our other sites:
http://www.techgenix.com
-------------------------------------------------------
To unsubscribe visit http://www.msexchange.org/pages/exchangelist.asp
Report abuse to listadmin@xxxxxxxxxxxxxx

Other related posts: