[mira_talk] Re: memory seetings
- From: David Lambert <dlamber2@xxxxxxxxxxxxxxxxxx>
- To: mira_talk@xxxxxxxxxxxxx
- Date: Tue, 22 Dec 2009 12:22:28 -0500
Hi,
Can you tell based on the log file from an aborted run (pasted below)
whether these new parameters are likely to help, or if I have another
problem. I am running an est assembly with 10K sanger reads and 1.3
million 454 titanium reads. I am using a two processors on a
cluster where I should have 8 gb RAM available.
Thanks once again!
Dave
At 05:47 PM 12/21/2009, you wrote:
On Montag 21 Dezember 2009 David
Lambert wrote:
> Hi all,
> I am having similar memory management problems running the
current
> version 2.8.3 on a cluster, and I would like to try these
parameters
> ((-AS:amm:kpmf:mps), but I can't find them in the manual or
usage
> docs, can someone point me to where they are discribed? Or are
they
> only for an earlier version?
Nope, these have been added only very recently. Latest incarnations of
release
candidate 3 (and hopefully the last before the real thing (now
really)):
http://www.chevreux.org/tmp/mira_3rc4e_dev_linux-gnu_x86_64_static.tar.bz2
http://www.chevreux.org/tmp/mira_3rc4e_dev_linux-gnu_i686_32_static.tar.bz2
http://www.chevreux.org/tmp/mira-3rc4e.tar.bz2
http://www.chevreux.org/tmp/mira_3rdparty_21-12-2009.tar.bz2
Regards,
Bastien
--
You have received this mail because you are subscribed to the mira_talk
mailing list. For information on how to subscribe or unsubscribe, please
visit
http://www.chevreux.org/mira_mailinglists.html
LAST SECTION OF LOG FOR ABORTED RUN:
Sanger 454 Solexa SOLiD
---------------------------------
Total reads 10434 1387166
0 0
Reads wo qual 0
0
0 0
Used reads 10431 1385446
0 0
Avg len used 1209
350 0
0
With strain 0
0
0 0
W/o clips
0 19056
0 0
===========================================================================
Starting clips: done.
Clipping or tagging poly A/T stretches at ends of reads ...
done.
===========================================================================
Pool statistics:
Backbones: 0 Backbone rails: 0
Sanger 454 Solexa SOLiD
---------------------------------
Total reads 10434 1387166
0 0
Reads wo qual 0
0
0 0
Used reads 9193 1368835
0 0
Avg len used 668
344 0
0
With strain 0
0
0 0
W/o clips
0 18478
0 0
===========================================================================
Writing temporary hstat files:
[0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|....
[40%] ....|.... [50%] ....|
.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|....
[100%] done
Analysing hstat files:
Ouch, out of memory detected.
========================== Memory self assessment
==============================
Running in 64 bit mode.
Dump from /proc/meminfo
--------------------------------------------------------------------------------
MemTotal: 16440136 kB
MemFree: 8384628 kB
Buffers: 66244 kB
Cached: 320580 kB
SwapCached: 0
kB
Active: 7686008 kB
Inactive: 246400 kB
HighTotal: 0
kB
HighFree:
0 kB
LowTotal: 16440136 kB
LowFree: 8384628 kB
SwapTotal: 1959920 kB
SwapFree: 1959920 kB
Dirty:
0 kB
Writeback: 0
kB
AnonPages: 7545532 kB
Mapped: 27080
kB
Slab:
67720 kB
PageTables: 17216 kB
NFS_Unstable: 0 kB
Bounce:
0 kB
CommitLimit: 10179988 kB
Committed_AS: 8732100 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 274980 kB
VmallocChunk: 34359463311 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
Hugepagesize: 2048 kB
--------------------------------------------------------------------------------
Dump from /proc/self/status
--------------------------------------------------------------------------------
Name: mira
State: R (running)
SleepAVG: 78%
Tgid: 10387
Pid: 10387
PPid: 10384
TracerPid: 0
Uid: 26544243
26544243
26544243 26544243
Gid: 26544243
26544243
26544243 26544243
FDSize: 256
Groups: 10013 10067 26544243
VmPeak: 7586148 kB
VmSize: 7474240 kB
VmLck: 0 kB
VmHWM: 7582924 kB
VmRSS: 7405748 kB
VmData: 7469892 kB
VmStk: 88 kB
VmExe: 4224 kB
VmLib: 0 kB
VmPTE: 14492 kB
StaBrk: 02013000 kB
Brk: 1b139d000 kB
StaStk: 7fff11ef18b0 kB
Threads: 1
SigQ: 0/137216
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000001000
SigCgt: 0000000180000000
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
Cpus_allowed:
00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff
Mems_allowed: 00000000,00000001
--------------------------------------------------------------------------------
Information on current assembly object:
AS_readpool: 1397600 reads.
AS_contigs: 0 contigs.
AS_bbcontigs: 0 contigs.
Mem used for reads: 6883962888 (6.4 GiB)
Memory used in assembly structures:
Eff. Size Free cap. LostByAlign
AS_writtenskimhitsperid:
0 24
B 0
B 0 B
AS_skim_edges:
0 24
B 0
B 0 B
AS_adsfacts:
0 24
B 0
B 0 B
AS_confirmed_edges:
0 24
B 0
B 0 B
AS_permanent_overlap_bans:
0 64
MiB 0
B 0 B
AS_readhitmiss:
0 24
B 0
B 0 B
AS_readhmcovered:
0 24
B 0
B 0 B
AS_count_rhm:
0 24
B 0
B 0 B
AS_clipleft:
1397600 5
MiB 0
B 0 B
AS_clipright:
1397600 5
MiB 0
B 0 B
AS_used_ids:
1397600 1
MiB 0
B 0 B
AS_multicopies:
0 1
MiB 1
MiB 0 B
AS_hasmcoverlaps:
0 1
MiB 1
MiB 0 B
AS_maxcoveragereached:
1397600 5
MiB 0
B 0 B
AS_coverageperseqtype:
0 24
B 0
B 0 B
AS_istroublemaker:
1397600 1
MiB 0
B 0 B
AS_isdebris:
1397600 1
MiB 0
B 0 B
AS_needalloverlaps:
1397600 1
MiB 32
B 0 B
AS_readsforrepeatresolve:
0 40
B 0
B 0 B
AS_allrmbsok:
0 5
MiB 5
MiB 0 B
AS_probablermbsnotok:
0 5
MiB 5
MiB 0 B
AS_weakrmbsnotok:
0 5
MiB 5
MiB 0 B
AS_readmaytakeskim:
0 40
B 0
B 0 B
AS_skimstaken:
0 40
B 0
B 0 B
AS_numskimoverlaps:
0 24
B 0
B 0 B
AS_numleftextendskims:
0 24
B 0
B 0 B
AS_rightextendskims:
0 24
B 0
B 0 B
AS_skimleftextendratio:
0 24
B 0
B 0 B
AS_skimrightextendratio:
0 24
B 0
B 0 B
AS_usedlogfiles:
4 144
B 0
B 0 B
Total: 6992976624 (6.5 GiB)
================================================================================
========================== Memory self assessment
==============================
Running in 64 bit mode.
Dump from /proc/meminfo
--------------------------------------------------------------------------------
MemTotal: 16440136 kB
MemFree: 8384628 kB
Buffers: 66244 kB
Cached: 320580 kB
SwapCached: 0
kB
Active: 7686008 kB
Inactive: 246400 kB
HighTotal: 0
kB
HighFree:
0 kB
LowTotal: 16440136 kB
LowFree: 8384628 kB
SwapTotal: 1959920 kB
SwapFree: 1959920 kB
Dirty:
0 kB
Writeback: 0
kB
AnonPages: 7545532 kB
Mapped: 27080
kB
Slab:
67728 kB
PageTables: 17216 kB
NFS_Unstable: 0 kB
Bounce:
0 kB
CommitLimit: 10179988 kB
Committed_AS: 8732100 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 274980 kB
VmallocChunk: 34359463311 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
Hugepagesize: 2048 kB
--------------------------------------------------------------------------------
Dump from /proc/self/status
--------------------------------------------------------------------------------
Name: mira
State: R (running)
SleepAVG: 73%
Tgid: 10387
Pid: 10387
PPid: 10384
TracerPid: 0
Uid: 26544243
26544243
26544243 26544243
Gid: 26544243
26544243
26544243 26544243
FDSize: 256
Groups: 10013 10067 26544243
VmPeak: 7586148 kB
VmSize: 7474240 kB
VmLck: 0 kB
VmHWM: 7582924 kB
VmRSS: 7405760 kB
VmData: 7469892 kB
VmStk: 88 kB
VmExe: 4224 kB
VmLib: 0 kB
VmPTE: 14492 kB
StaBrk: 02013000 kB
Brk: 1b139d000 kB
StaStk: 7fff11ef18b0 kB
Threads: 1
SigQ: 0/137216
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000001000
SigCgt: 0000000180000000
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
Cpus_allowed:
00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff
Mems_allowed: 00000000,00000001
--------------------------------------------------------------------------------
Information on current assembly object:
AS_readpool: 1397600 reads.
AS_contigs: 0 contigs.
AS_bbcontigs: 0 contigs.
Mem used for reads: 6883962888 (6.4 GiB)
Memory used in assembly structures:
Eff. Size Free cap. LostByAlign
AS_writtenskimhitsperid:
0 24
B 0
B 0 B
AS_skim_edges:
0 24
B 0
B 0 B
AS_adsfacts:
0 24
B 0
B 0 B
AS_confirmed_edges:
0 24
B 0
B 0 B
AS_permanent_overlap_bans:
0 64
MiB 0
B 0 B
AS_readhitmiss:
0 24
B 0
B 0 B
AS_readhmcovered:
0 24
B 0
B 0 B
AS_count_rhm:
0 24
B 0
B 0 B
AS_clipleft:
1397600 5
MiB 0
B 0 B
AS_clipright:
1397600 5
MiB 0
B 0 B
AS_used_ids:
1397600 1
MiB 0
B 0 B
AS_multicopies:
0 1
MiB 1
MiB 0 B
AS_hasmcoverlaps:
0 1
MiB 1
MiB 0 B
AS_maxcoveragereached:
1397600 5
MiB 0
B 0 B
AS_coverageperseqtype:
0 24
B 0
B 0 B
AS_istroublemaker:
1397600 1
MiB 0
B 0 B
AS_isdebris:
1397600 1
MiB 0
B 0 B
AS_needalloverlaps:
1397600 1
MiB 32
B 0 B
AS_readsforrepeatresolve:
0 40
B 0
B 0 B
AS_allrmbsok:
0 5
MiB 5
MiB 0 B
AS_probablermbsnotok:
0 5
MiB 5
MiB 0 B
AS_weakrmbsnotok:
0 5
MiB 5
MiB 0 B
AS_readmaytakeskim:
0 40
B 0
B 0 B
AS_skimstaken:
0 40
B 0
B 0 B
AS_numskimoverlaps:
0 24
B 0
B 0 B
AS_numleftextendskims:
0 24
B 0
B 0 B
AS_rightextendskims:
0 24
B 0
B 0 B
AS_skimleftextendratio:
0 24
B 0
B 0 B
AS_skimrightextendratio:
0 24
B 0
B 0 B
AS_usedlogfiles:
4 144
B 0
B 0 B
Total: 6992976624 (6.5 GiB)
================================================================================
Dynamic allocs: 0
Align allocs: 0
Out of memory detected, exception message is: std::bad_alloc
If you have questions on why this happened, please send the last
1000
lines (or better, the complete log) of the output log to the author
(together with a short summary of your assembly project).
For general help, you will probably get a quicker response on the
MIRA talk mailing list
than if you mailed the author directly.
To report bugs or ask for features, please use the new ticketing system
at:
http://sourceforge.net/apps/trac/mira-assembler/
This ensures that requests don't get lost.
/var/spool/pbs/mom_priv/jobs/32907.bhsn-int.bluehive.crc.private.SC: line
15: 10387 Aborted
mira -project=2009 -fasta -job=denovo,est,normal,sanger,454
COMMON_SETTING
S -SK:not=4 SANGER_SETTINGS -OUT:sssip=1 454_SETTINGS -OUT:sssip=1
[dlamber2@bluehive debug]$
-------------------------------------------------------------------------
David Lambert
Assistant Professor
Department of Biology
University of Rochester
http://www.rochester.edu/College/BIO/labs/LambertLab/index.html
585 273-2482 (office)
585 275-7521 (lab)
585 275-2070 (fax)
Shipping address:
University of Rochester
Department of Biology
213 Hutchison Hall
River Campus
Rochester, NY 14627-0211
Mailing address:
University of Rochester
Hutchison Hall
RC Box # 270211
Rochester NY 14627-0211
--
You have received this mail because you are subscribed to the mira_talk mailing list. For information on how to subscribe or unsubscribe, please visit http://www.chevreux.org/mira_mailinglists.html
Other related posts:
