[recoll-user] peaks in IO that freeze the machine
- From: Ramon Diaz-Uriarte <rdiaz02@xxxxxxxxx>
- To: recoll-user@xxxxxxxxxxxxx
- Date: Fri, 02 Sep 2011 12:26:13 +0200
Dear All,
I've been a very happy user of recoll. Lately, however, I've started to
notice that I periodically get peaks in IO that freeze my systems.
Details:
- I index my home directory (my recoll.conf is at the bottom) using real
time indexing. (I also monitor /usr/share/doc with another recollindex,
but that rarely causes trouble).
- My current recoll version is 1.15.9-1 (as packed by Debian).
- I am monitoring IO usage with iotop.
- From time to time (e.g., when I delete/refile messages in my Maildir, or
when offlineimap deletes messages) I can see recollindex doing lots of
IO (iotop shows up to 99%).
- This seems to lead to many other processes being left waiting (as shown
by top). For instance, while answering email, Emacs freezes and you
cannot see what your are typing. Or the command line freezes if you are
doing a simple "ls".
- This can last from barely less than a second to a few seconds, but it is
very intrusive.
- The log output shows all sorts of messages, many of them "No such file
or directory", but not all. In particular, it does not seem related (as
far as I can tell) to processing some very huge text file.
- This problem happens in several machines, from a tiny Asus netbook to a
quad-core workstation. All run Debian GNU/Linux and recent stock Debian
kernels (e.g., 2.6.39).
I am attaching a screen capture that shows (left to right, top to bottom):
* tail -f recolltrace
* iotop -o
* iotop -o -a
* top
it shows a case where IO from recollindex seems to do lots of IO and a
large fraction of processes (23%, but in other cases as many as 50%) are
waiting, I think because of IO.
What am I doing wrong? Is there a way to avoid these problems?
Thanks a lot, and thanks for such a great program.
Best,
R.
P.S. My ~/.recoll/recoll.conf
****************
indexstemminglanguages = english spanish
skippedNames = *.RData *.Rdata *~ .beagle .bzr .git .hg .svn .thumbnails CV\
S Cache bin cache* caughtspam loop.ps recollrc tmp .xsessio\
n-errors .recoll* xapiandb recollrc #*
skippedPaths = /home/ramon/.local/share/data /home/ramon/.wuala /home/ramon\
/WualaDrive /home/ramon/caca /home/ramon/tmp
topdirs = /home/ramon
## /usr/share/doc indexed in a separate process
loglevel = 3
logfilename = /home/ramon/.recoll/recolltrace
textfilemaxmbs = 100
****************
--
Ramon Diaz-Uriarte
Department of Biochemistry
Universidad Autónoma de Madrid
Spain
http://ligarto.org/rdiaz
Temporarily at:
Structural Biology and Biocomputing Programme
Spanish National Cancer Centre (CNIO)
Phone: +34-91-732-8000 ext. 3019
Fax: +-34-91-224-6972
Other related posts:
