Hi Everybody, I am running a denovo assembly of a bacterial genome having ~150X coverage (sampled from 2000X+ coverage dataset) and 7 million Illumina paired end 100bp reads on a 64GB RAM machine using MIRA 3.4.0. Here are my parameters mira --project=9gb_28e6_def --job=denovo,genome,accurate,solexa --noclipping COMMON_SETTINGS -GE:number_of_threads=8 -MI:stop_on_nfs=no:somrnl=0 SOLEXA_SETTINGS -LR:file_type=fastq I am having to run it on a NFS but that cannot be the problem. MIRA stops due to an internal error ("Error while loading adsfacts, less facts in file than calculated earlier!"). It was using <30 GB or RAM at that point so memory is not an issue. Any ideas about what could be going on? The STDOUT is pasted below. Thanks! -Surya This is MIRA V3.4.0 (production version). Please cite: Chevreux, B., Wetter, T. and Suhai, S. (1999), Genome Sequence Assembly Using Trace Signals and Additional Sequence Information. Computer Science and Biology: Proceedings of the German Conference on Bioinformatics (GCB) 99, pp. 45-56. To (un-)subscribe the MIRA mailing lists, see: http://www.chevreux.org/mira_mailinglists.html After subscribing, mail general questions to the MIRA talk mailing list: mira_talk@xxxxxxxxxxxxx To report bugs or ask for features, please use the new ticketing system at: http://sourceforge.net/apps/trac/mira-assembler/ This ensures that requests don't get lost. Compiled by: bach Sun Aug 21 17:50:30 CEST 2011 On: Linux arcadia 2.6.38-11-generic #48-Ubuntu SMP Fri Jul 29 19:02:55 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux Compiled in boundtracking mode. Compiled in bugtracking mode. Compiled with ENABLE64 activated. Runtime settings (sorry, for debug): Size of size_t : 8 Size of uint32 : 4 Size of uint32_t: 4 Size of uint64 : 8 Size of uint64_t: 8 Current system: Linux lmcompute-4-4.v4linux 2.6.18-238.19.1.el5 #1 SMP Fri Jul 15 07:31:24 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux Parsing parameters: --project=9gb_28e6_def --job=denovo,genome,accurate,solexa --noclipping COMMON_SETTINGS -GE:number_of_threads=8 -MI:stop_on_nfs=no:somrnl=0 SOLEXA_SETTINGS -LR:file_type=fastq Using quickmode switch -noclipping : SANGER_SETTINGS -CL:msvs=no:pvlc=no:qc=no:bsqc=no:mbc=no:lcc=no:emlc=no:emrc=no:c3pp=no:cpat=no:mqtfer=0:ckar=no 454_SETTINGS -CL:msvs=no:pvlc=no:qc=no:bsqc=no:mbc=no:lcc=no:emlc=no:emrc=no:c3pp=no:cpat=no:mqtfer=0:ckar=no IONTOR_SETTINGS -CL:msvs=no:pvlc=no:qc=no:bsqc=no:mbc=no:lcc=no:emlc=no:emrc=no:c3pp=no:cpat=no:mqtfer=0:ckar=no PACBIO_SETTINGS -CL:msvs=no:pvlc=no:qc=no:bsqc=no:mbc=no:lcc=no:emlc=no:emrc=no:c3pp=no:cpat=no:mqtfer=0:ckar=no SOLEXA_SETTINGS -CL:msvs=no:pvlc=no:qc=no:bsqc=no:mbc=no:lcc=no:emlc=no:emrc=no:c3pp=no:cpat=no:mqtfer=0:ckar=no SOLID_SETTINGS -CL:msvs=no:pvlc=no:qc=no:bsqc=no:mbc=no:lcc=no:emlc=no:emrc=no:c3pp=no:cpat=no:mqtfer=0:ckar=no COMMON_SETTINGS -CL:ascdc=no:asjdc=no:cbse=no Parameters parsed without error, perfect. -CL:pec and -CO:emeas1clpec are set, setting -CO:emea values to 1. ------------------------------------------------------------------------------ Parameter settings seen for: Sanger data (also common parameters), Solexa data Used parameter settings: General (-GE): Project name in (proin) : 9gb_28e6_def Project name out (proout) : 9gb_28e6_def Number of threads (not) : 8 Automatic memory management (amm) : yes Keep percent memory free (kpmf) : 15 Max. process size (mps) : 0 EST SNP pipeline step (esps) : 0 Use template information (uti) : [san] yes [sxa] yes Template insert size minimum (tismin) : [san] -1 [sxa] -1 Template insert size maximum (tismax) : [san] -1 [sxa] -1 Template partner build direction (tpbd) : [san] -1 [sxa] -1 Colour reads by hash frequency (crhf) : yes Load reads options (-LR): Load sequence data (lsd) : [san] no [sxa] yes File type (ft) : [san] fasta [sxa] fastq External quality (eq) : from SCF (scf) Ext. qual. override (eqo) : no Discard reads on e.q. error (droeqe): no Solexa scores in qual file (ssiqf) : no FASTQ qual offset (fqqo) : [san] 0 [sxa] 0 Wants quality file (wqf) : [san] yes [sxa] yes Read naming scheme (rns) : [san] Sanger Institute (sanger) [sxa] Solexa (solexa) Merge with XML trace info (mxti) : [san] no [sxa] no Filecheck only (fo) : no Assembly options (-AS): Number of passes (nop) : 3 Skim each pass (sep) : yes Maximum number of RMB break loops (rbl) : 1 Maximum contigs per pass (mcpp) : 0 Minimum read length (mrl) : [san] 80 [sxa] 20 Minimum reads per contig (mrpc) : [san] 2 [sxa] 10 Base default quality (bdq) : [san] 10 [sxa] 10 Enforce presence of qualities (epoq) : [san] yes [sxa] yes Automatic repeat detection (ard) : yes Coverage threshold (ardct) : [san] 2 [sxa] 2.5 Minimum length (ardml) : [san] 400 [sxa] 300 Grace length (ardgl) : [san] 40 [sxa] 20 Use uniform read distribution (urd) : no Start in pass (urdsip) : 3 Cutoff multiplier (urdcm) : [san] 1.5 [sxa] 1.5 Keep long repeats separated (klrs) : no Spoiler detection (sd) : yes Last pass only (sdlpo) : yes Use genomic pathfinder (ugpf) : yes Use emergency search stop (uess) : yes ESS partner depth (esspd) : 500 Use emergency blacklist (uebl) : yes Use max. contig build time (umcbt) : no Build time in seconds (bts) : 10000 Strain and backbone options (-SB): Load straindata (lsd) : no Assign default strain (ads) : [san] no [sxa] no Default strain name (dsn) : [san] StrainX [sxa] StrainX Load backbone (lb) : no Start backbone usage in pass (sbuip) : 3 Backbone file type (bft) : fasta Backbone base quality (bbq) : 30 Backbone strain name (bsn) : ReferenceStrain Force for all (bsnffa) : no Backbone rail from strain (brfs) : Backbone rail length (brl) : 0 Backbone rail overlap (bro) : 0 Also build new contigs (abnc) : yes Dataprocessing options (-DP): Use read extensions (ure) : [san] yes [sxa] no Read extension window length (rewl) : [san] 30 [sxa] 30 Read extension w. maxerrors (rewme) : [san] 2 [sxa] 2 First extension in pass (feip) : [san] 0 [sxa] 0 Last extension in pass (leip) : [san] 0 [sxa] 0 Clipping options (-CL): Merge with SSAHA2/SMALT vector screen (msvs): [san] no [sxa] no Gap size (msvsgs) : [san] 10 [sxa] 1 Max front gap (msvsmfg) : [san] 60 [sxa] 2 Max end gap (msvsmeg) : [san] 120 [sxa] 2 Strict front clip (msvssfc) : [san] 0 [sxa] 0 Strict end clip (msvssec) : [san] 0 [sxa] 0 Possible vector leftover clip (pvlc) : [san] no [sxa] no maximum len allowed (pvcmla) : [san] 18 [sxa] 18 Min qual. threshold for entire read (mqtfer): [san] 0 [sxa] 0 Number of bases (mqtfernob) : [san] 0 [sxa] 15 Quality clip (qc) : [san] no [sxa] no Minimum quality (qcmq) : [san] 20 [sxa] 20 Window length (qcwl) : [san] 30 [sxa] 30 Bad stretch quality clip (bsqc) : [san] no [sxa] no Minimum quality (bsqcmq) : [san] 20 [sxa] 5 Window length (bsqcwl) : [san] 30 [sxa] 20 Masked bases clip (mbc) : [san] no [sxa] no Gap size (mbcgs) : [san] 20 [sxa] 5 Max front gap (mbcmfg) : [san] 40 [sxa] 12 Max end gap (mbcmeg) : [san] 60 [sxa] 12 Lower case clip (lcc) : [san] no [sxa] no Clip poly A/T at ends (cpat) : [san] no [sxa] no Keep poly-a signal (cpkps) : [san] no [sxa] no Minimum signal length (cpmsl) : [san] 12 [sxa] 12 Max errors allowed (cpmea) : [san] 1 [sxa] 1 Max gap from ends (cpmgfe) : [san] 9 [sxa] 9 Clip 3 prime polybase (c3pp) : [san] no [sxa] no Minimum signal length (c3ppmsl) : [san] 12 [sxa] 12 Max errors allowed (c3ppmea) : [san] 2 [sxa] 2 Max gap from ends (c3ppmgfe) : [san] 9 [sxa] 9 Clip known adaptors right (ckar) : [san] no [sxa] no Ensure minimum left clip (emlc) : [san] no [sxa] no Minimum left clip req. (mlcr) : [san] 25 [sxa] 0 Set minimum left clip to (smlc) : [san] 30 [sxa] 0 Ensure minimum right clip (emrc) : [san] no [sxa] no Minimum right clip req. (mrcr) : [san] 10 [sxa] 10 Set minimum right clip to (smrc) : [san] 20 [sxa] 20 Apply SKIM chimera detection clip (ascdc) : no Apply SKIM junk detection clip (asjdc) : no Propose end clips (pec) : yes Bases per hash (pecbph) : 31 Handle Solexa GGCxG problem (pechsgp) : yes Clip bad solexa ends (cbse) : no Parameters for SKIM algorithm (-SK): Number of threads (not) : 8 Also compute reverse complements (acrc) : yes Bases per hash (bph) : 17 Hash save stepping (hss) : 1 Percent required (pr) : [san] 65 [sxa] 95 Max hits per read (mhpr) : 2000 Max megahub ratio (mmhr) : 0 SW check on backbones (swcob) : no Freq. est. min normal (fenn) : 0.4 Freq. est. max normal (fexn) : 1.6 Freq. est. repeat (fer) : 1.9 Freq. est. heavy repeat (fehr) : 8 Freq. est. crazy (fecr) : 20 Mask nasty repeats (mnr) : yes Nasty repeat ratio (nrr) : 100 Repeat level in info file (rliif) : 6 Max hashes in memory (mhim) : 15000000 MemCap: hit reduction (mchr) : 4096 Pathfinder options (-PF): Use quick rule (uqr) : [san] yes [sxa] yes Quick rule min len 1 (qrml1) : [san] 200 [sxa] -95 Quick rule min sim 1 (qrms1) : [san] 90 [sxa] 100 Quick rule min len 2 (qrml2) : [san] 100 [sxa] -85 Quick rule min sim 2 (qrms2) : [san] 95 [sxa] 100 Backbone quick overlap min len (bqoml) : [san] 150 [sxa] 20 Max. start cache fill time (mscft) : 5 Align parameters for Smith-Waterman align (-AL): Bandwidth in percent (bip) : [san] 20 [sxa] 20 Bandwidth max (bmax) : [san] 130 [sxa] 80 Bandwidth min (bmin) : [san] 25 [sxa] 20 Minimum score (ms) : [san] 30 [sxa] 15 Minimum overlap (mo) : [san] 17 [sxa] 25 Minimum relative score in % (mrs) : [san] 65 [sxa] 90 Solexa_hack_max_errors (shme) : [san] 0 [sxa] 0 Extra gap penalty (egp) : [san] no [sxa] no extra gap penalty level (egpl) : [san] low [sxa] low Max. egp in percent (megpp) : [san] 100 [sxa] 100 Contig parameters (-CO): Name prefix (np) : 9gb_28e6_def Reject on drop in relative alignment score in % (rodirs) : [san] 25 [sxa] 30 Mark repeats (mr) : yes Only in result (mroir) : no Assume SNP instead of repeats (asir) : no Minimum reads per group needed for tagging (mrpg) : [san] 2 [sxa] 4 Minimum neighbour quality needed for tagging (mnq) : [san] 20 [sxa] 20 Minimum Group Quality needed for RMB Tagging (mgqrt) : [san] 30 [sxa] 30 End-read Marking Exclusion Area in bases (emea) : [san] 1 [sxa] 1 Set to 1 on clipping PEC (emeas1clpec) : yes Also mark gap bases (amgb) : [san] yes [sxa] yes Also mark gap bases - even multicolumn (amgbemc) : [san] yes [sxa] yes Also mark gap bases - need both strands (amgbnbs): [san] yes [sxa] yes Force non-IUPAC consensus per sequencing type (fnicpst) : [san] no [sxa] no Merge short reads (msr) : [san] no [sxa] yes Keep ends unmerged (msrkeu) : [san] -1 [sxa] -1 Gap override ratio (gor) : [san] 66 [sxa] 66 Edit options (-ED): Automatic contig editing (ace) : [san] no [sxa] no Sanger only: Strict editing mode (sem) : no Confirmation threshold in percent (ct) : 50 Misc (-MI): Stop on NFS (sonfs) : no Extended log (el) : no Large contig size (lcs) : 500 Large contig size for stats(lcs4s) : 5000 Stop on max read name length (somrnl) : 0 Directories (-DI): Working directory : When loading EXP files : When loading SCF files : Top directory for writing files : 9gb_28e6_def_assembly For writing result files : 9gb_28e6_def_assembly/9gb_28e6_def_d_results For writing result info files : 9gb_28e6_def_assembly/9gb_28e6_def_d_info For writing tmp files : 9gb_28e6_def_assembly/9gb_28e6_def_d_tmp Tmp redirected to (trt) : For writing checkpoint files : 9gb_28e6_def_assembly/9gb_28e6_def_d_chkpt File names (-FN): When loading sequences from FASTA : [san] 9gb_28e6_def_in.sanger.fasta [sxa] 9gb_28e6_def_in.solexa.fasta When loading qualities from FASTA quality : [san] 9gb_28e6_def_in.sanger.fasta.qual [sxa] 9gb_28e6_def_in.solexa.fasta.qual When loading sequences from FASTQ : [san] 9gb_28e6_def_in.sanger.fastq [sxa] 9gb_28e6_def_in.solexa.fastq When loading project from CAF : 9gb_28e6_def_in.sanger.caf When loading project from MAF (disabled) : 9gb_28e6_def_in.sanger.maf When loading EXP fofn : 9gb_28e6_def_in.sanger.fofn When loading project from PHD : 9gb_28e6_def_in.phd.1 When loading strain data : 9gb_28e6_def_straindata_in.txt When loading XML trace info files : [san] 9gb_28e6_def_traceinfo_in.sanger.xml [sxa] 9gb_28e6_def_traceinfo_in.solexa.xml When loading SSAHA2 vector screen results : 9gb_28e6_def_ssaha2vectorscreen_in.txt When loading SMALT vector screen results : 9gb_28e6_def_smaltvectorscreen_in.txt When loading backbone from MAF : 9gb_28e6_def_backbone_in.maf When loading backbone from CAF : 9gb_28e6_def_backbone_in.caf When loading backbone from GenBank : 9gb_28e6_def_backbone_in.gbf When loading backbone from GFF3 : 9gb_28e6_def_backbone_in.gff3 When loading backbone from FASTA : 9gb_28e6_def_backbone_in.fasta Output files (-OUTPUT/-OUT): Save simple singlets in project (sssip) : [san] no [sxa] no Save tagged singlets in project (stsip) : [san] yes [sxa] yes Remove rollover tmps (rrot) : yes Remove tmp directory (rtd) : no Result files: Saved as CAF (orc) : yes Saved as MAF (orm) : yes Saved as FASTA (orf) : yes Saved as GAP4 (directed assembly) (org) : no Saved as phrap ACE (ora) : yes Saved as GFF3 (org3) : no Saved as HTML (orh) : no Saved as Transposed Contig Summary (ors) : yes Saved as simple text format (ort) : no Saved as wiggle (orw) : yes Temporary result files: Saved as CAF (otc) : yes Saved as MAF (otm) : no Saved as FASTA (otf) : no Saved as GAP4 (directed assembly) (otg) : no Saved as phrap ACE (ota) : no Saved as HTML (oth) : no Saved as Transposed Contig Summary (ots) : no Saved as simple text format (ott) : no Extended temporary result files: Saved as CAF (oetc) : no Saved as FASTA (oetf) : no Saved as GAP4 (directed assembly) (oetg) : no Saved as phrap ACE (oeta) : no Saved as HTML (oeth) : no Save also singlets (oetas) : no Alignment output customisation: TEXT characters per line (tcpl) : 60 HTML characters per line (hcpl) : 60 TEXT end gap fill character (tegfc) : HTML end gap fill character (hegfc) : File / directory output names: CAF : 9gb_28e6_def_out.caf MAF : 9gb_28e6_def_out.maf FASTA : 9gb_28e6_def_out.unpadded.fasta FASTA quality : 9gb_28e6_def_out.unpadded.fasta.qual FASTA (padded) : 9gb_28e6_def_out.padded.fasta FASTA qual.(pad): 9gb_28e6_def_out.padded.fasta.qual GAP4 (directory): 9gb_28e6_def_out.gap4da ACE : 9gb_28e6_def_out.ace HTML : 9gb_28e6_def_out.html Simple text : 9gb_28e6_def_out.txt TCS overview : 9gb_28e6_def_out.tcs Wiggle : 9gb_28e6_def_out.wig ------------------------------------------------------------------------------ Creating directory 9gb_28e6_def_assembly ... done. Creating directory 9gb_28e6_def_assembly/9gb_28e6_def_d_tmp ... done. Creating directory 9gb_28e6_def_assembly/9gb_28e6_def_d_results ... done. Creating directory 9gb_28e6_def_assembly/9gb_28e6_def_d_info ... done. Creating directory 9gb_28e6_def_assembly/9gb_28e6_def_d_chkpt ... done. Tmp directory is not on a NFS mount, good. Localtime: Thu Aug 25 10:19:49 2011 Loading data (Solexa) from FASTQ files, Localtime: Thu Aug 25 10:19:49 2011 Counting sequences in FASTQ file: found 14000000 sequences. Localtime: Thu Aug 25 10:20:05 2011 Solexa will load 14000000 reads. Longest Sanger: 0 Longest 454: 0 Longest IonTor: 0 Longest PacBio: 0 Longest Solexa: 100 Longest Solid: 0 Longest overall: 100 Total reads to load: 14000000 Reserving space for reads (this may take a while) Reserved space for 14000010 reads. Loading data (Solexa) from FASTQ files, Localtime: Thu Aug 25 10:20:05 2011 Counting sequences in FASTQ file: found 14000000 sequences. Localtime: Thu Aug 25 10:20:21 2011 Using calculated FASTQ quality offset: 66 Localtime: Thu Aug 25 10:20:21 2011 Loading data from FASTQ file: [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] Done. Loaded 14000000 reads, Localtime: Thu Aug 25 10:21:20 2011 Loaded 14000000 Solexa reads. Total reads loaded: 14000000 Checking reads for trace data: [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] No SCF data present in any read, automatic contig editing for Sanger data is now switched off. 14000000 reads with valid data for assembly. Localtime: Thu Aug 25 10:21:35 2011 Generated 7000000 unique template ids for 14000000 valid reads. Localtime: Thu Aug 25 10:22:07 2011 Generated 0 unique strain ids for 14000000 reads. Strain "default" has 14000000 reads. Have read pool with 14000000 reads. =========================================================================== Pool statistics: Backbones: 0 Backbone rails: 0 Sanger 454 IonTor PacBio Solexa SOLiD ---------------------------------------- Total reads 0 0 0 0 14000000 0 Reads wo qual 0 0 0 0 0 0 Used reads 0 0 0 0 14000000 0 Avg tot rlen 0 0 0 0 88 0 Avg rlen used 0 0 0 0 87 0 With strain 0 0 0 0 0 0 W/o clips 0 0 0 0 14000000 0 Sanger total bases:0 used bases in used reads: 0 454 total bases:0 used bases in used reads: 0 IonTor total bases:0 used bases in used reads: 0 PacBio total bases:0 used bases in used reads: 0 Solexa total bases:1238800892 used bases in used reads: 1224800892 Solid total bases:0 used bases in used reads: 0 =========================================================================== =========================================================================== Pool statistics: Backbones: 0 Backbone rails: 0 Sanger 454 IonTor PacBio Solexa SOLiD ---------------------------------------- Total reads 0 0 0 0 14000000 0 Reads wo qual 0 0 0 0 0 0 Used reads 0 0 0 0 14000000 0 Avg tot rlen 0 0 0 0 88 0 Avg rlen used 0 0 0 0 87 0 With strain 0 0 0 0 0 0 W/o clips 0 0 0 0 14000000 0 Sanger total bases:0 used bases in used reads: 0 454 total bases:0 used bases in used reads: 0 IonTor total bases:0 used bases in used reads: 0 PacBio total bases:0 used bases in used reads: 0 Solexa total bases:1238800892 used bases in used reads: 1224800892 Solid total bases:0 used bases in used reads: 0 =========================================================================== Tmp directory is not on a NFS mount, good. Hash analysis for proposed cutbacks:Localtime: Thu Aug 25 10:22:25 2011 Writing temporary hstat files: [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] done Localtime: Thu Aug 25 10:26:53 2011 Analysing hstat files: [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] Localtime: Thu Aug 25 10:34:13 2011 clean up temporary stat files...Localtime: Thu Aug 25 10:34:34 2011 Localtime: Thu Aug 25 10:34:41 2011 Hash statistics: ========================================================= Measured avg. frequency coverage: 131 Deduced thresholds: ------------------- Min normal cov: 52.4 Max normal cov: 209.6 Repeat cov: 248.9 Heavy cov: 1048 Crazy cov: 2620 Mask cov: 13100 Repeat ratio histogram: ----------------------- 0 4687450 1 8913532 2 1264412 3 148666 4 46536 5 23400 6 13612 7 8478 8 5394 9 4072 10 1456 11 628 12 256 13 274 14 360 15 232 16 200 17 276 18 166 19 150 20 260 21 228 22 146 23 178 24 246 25 140 26 178 27 214 28 136 29 82 30 40 31 50 32 104 33 64 34 14 35 26 36 28 37 72 38 64 39 50 40 38 41 40 42 16 43 34 44 14 60 2 61 10 62 4 ========================================================= Assigning statistics values: Localtime: Thu Aug 25 10:34:41 2011 [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] Localtime: Thu Aug 25 10:41:36 2011 Localtime: Thu Aug 25 10:41:36 2011 Looking for proposed cutbacks ... done. Performed clips: Num reads cliped left: 19717 Num reads cliped right: 253190 Num reads completely killed: 3410 Total bases clipped : 2929488 Clipping dubious poly-base stretches at end of reads ... done. Hash analysis for proposed cutbacks:Localtime: Thu Aug 25 10:41:51 2011 Writing temporary hstat files: [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] done Localtime: Thu Aug 25 10:44:06 2011 Analysing hstat files: [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] Localtime: Thu Aug 25 10:51:34 2011 clean up temporary stat files...Localtime: Thu Aug 25 10:51:55 2011 Localtime: Thu Aug 25 10:52:02 2011 Hash statistics: ========================================================= Measured avg. frequency coverage: 132 Deduced thresholds: ------------------- Min normal cov: 52.8 Max normal cov: 211.2 Repeat cov: 250.8 Heavy cov: 1056 Crazy cov: 2640 Mask cov: 13200 Repeat ratio histogram: ----------------------- 0 3563778 1 8937479 2 1241779 3 145600 4 45534 5 23238 6 13478 7 8282 8 5330 9 3870 10 1392 11 532 12 264 13 270 14 374 15 212 16 216 17 262 18 162 19 170 20 264 21 210 22 152 23 190 24 242 25 140 26 172 27 214 28 118 29 76 30 44 31 68 32 102 33 36 34 22 35 22 36 48 37 66 38 62 39 36 40 54 41 26 42 22 43 30 44 8 60 8 61 8 ========================================================= Assigning statistics values: Localtime: Thu Aug 25 10:52:02 2011 [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] Localtime: Thu Aug 25 10:58:51 2011 Localtime: Thu Aug 25 10:58:51 2011 Looking for proposed cutbacks ... done. Performed clips: Num reads cliped left: 747 Num reads cliped right: 3948 Num reads completely killed: 111 Total bases clipped : 38308 Clipping dubious poly-base stretches at end of reads ... done. =========================================================================== Pool statistics: Backbones: 0 Backbone rails: 0 Sanger 454 IonTor PacBio Solexa SOLiD ---------------------------------------- Total reads 0 0 0 0 14000000 0 Reads wo qual 0 0 0 0 0 0 Used reads 0 0 0 0 13996386 0 Avg tot rlen 0 0 0 0 88 0 Avg rlen used 0 0 0 0 87 0 With strain 0 0 0 0 0 0 W/o clips 0 0 0 0 13726232 0 Sanger total bases:0 used bases in used reads: 0 454 total bases:0 used bases in used reads: 0 IonTor total bases:0 used bases in used reads: 0 PacBio total bases:0 used bases in used reads: 0 Solexa total bases:1238800892 used bases in used reads: 1221772035 Solid total bases:0 used bases in used reads: 0 =========================================================================== Dump from /proc/self/status -------------------------------------------------------------------------------- Name: mira State: R (running) SleepAVG: 0% Tgid: 28680 Pid: 28680 PPid: 28674 TracerPid: 0 Uid: 16777813 16777813 16777813 16777813 Gid: 10000 10000 10000 10000 FDSize: 256 Groups: 10000 10157 10321 VmPeak: 15787988 kB VmSize: 15787924 kB VmLck: 0 kB VmHWM: 15587236 kB VmRSS: 15587068 kB VmData: 15782564 kB VmStk: 88 kB VmExe: 5236 kB VmLib: 0 kB VmPTE: 30488 kB StaBrk: 036cb000 kB Brk: 3c6b0c000 kB StaStk: 7ffff20e63e0 kB Threads: 1 SigQ: 0/530432 SigPnd: 0000000000000000 ShdPnd: 0000000000000000 SigBlk: 0000000000000000 SigIgn: 0000000000001000 SigCgt: 0000000180000000 CapInh: 0000000000000000 CapPrm: 0000000000000000 CapEff: 0000000000000000 Cpus_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,ffffffff Mems_allowed: 00000000,00000001 -------------------------------------------------------------------------------- Pass: 1 / 3 Performing snapshot 1 Localtime: Thu Aug 25 10:59:09 2011 Localtime: Thu Aug 25 11:00:45 2011 =========================================================================== Pool statistics: Backbones: 0 Backbone rails: 0 Sanger 454 IonTor PacBio Solexa SOLiD ---------------------------------------- Total reads 0 0 0 0 14000000 0 Reads wo qual 0 0 0 0 0 0 Used reads 0 0 0 0 13996386 0 Avg tot rlen 0 0 0 0 88 0 Avg rlen used 0 0 0 0 87 0 With strain 0 0 0 0 0 0 W/o clips 0 0 0 0 13726232 0 Sanger total bases:0 used bases in used reads: 0 454 total bases:0 used bases in used reads: 0 IonTor total bases:0 used bases in used reads: 0 PacBio total bases:0 used bases in used reads: 0 Solexa total bases:1238800892 used bases in used reads: 1221772035 Solid total bases:0 used bases in used reads: 0 =========================================================================== Localtime: Thu Aug 25 11:00:48 2011 Writing temporary hstat files: [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] done Localtime: Thu Aug 25 11:03:49 2011 Analysing hstat files: [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] Localtime: Thu Aug 25 11:13:08 2011 clean up temporary stat files...Localtime: Thu Aug 25 11:13:34 2011 Localtime: Thu Aug 25 11:13:40 2011 Hash statistics: ========================================================= Measured avg. frequency coverage: 162 Deduced thresholds: ------------------- Min normal cov: 64.8 Max normal cov: 259.2 Repeat cov: 307.8 Heavy cov: 1296 Crazy cov: 3240 Mask cov: 16200 Repeat ratio histogram: ----------------------- 0 2761088 1 9185807 2 1214837 3 148260 4 50870 5 27552 6 15636 7 9168 8 6502 9 4004 10 1520 11 868 12 352 13 330 14 400 15 226 16 300 17 304 18 328 19 194 20 268 21 218 22 214 23 278 24 246 25 168 26 216 27 210 28 122 29 84 30 46 31 96 32 100 33 74 34 16 35 40 36 86 37 80 38 52 39 20 40 104 41 14 42 32 43 42 46 2 48 2 49 4 50 2 52 2 53 2 55 2 60 2 61 2 66 4 69 8 70 4 72 2 81 2 85 2 88 2 ========================================================= Assigning statistics values: Localtime: Thu Aug 25 11:13:40 2011 [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] Localtime: Thu Aug 25 11:21:56 2011 Buntifying reads (this may take a while) ... done. Writing read repeat info to: 9gb_28e6_def_assembly/9gb_28e6_def_d_info/9gb_28e6_def_info_readrepeats.lst ... 156402 sequences with 187550 masked stretches. Localtime: Thu Aug 25 11:22:28 2011 Searching for possible overlaps: Localtime: Thu Aug 25 11:22:28 2011 Now running threaded and partitioned skimmer with 82 partitions in 8 threads: [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ..truncating 9gb_28e6_def_assembly/9gb_28e6_def_d_tmp/9gb_28e6_def_int_posmatchf_pass.1.bin from 11138087184 to 8823555000 .truncating 9gb_28e6_def_assembly/9gb_28e6_def_d_tmp/9gb_28e6_def_int_posmatchc_pass.1.bin from 10756888968 to 8690135736 .|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] .truncating 9gb_28e6_def_assembly/9gb_28e6_def_d_tmp/9gb_28e6_def_int_posmatchf_pass.1.bin from 21705369888 to 18382481664 ..truncating 9gb_28e6_def_assembly/9gb_28e6_def_d_tmp/9gb_28e6_def_int_posmatchc_pass.1.bin from 21559777416 to 18455396856 .|.... [80%] ....|.... [90%] ....|.... [100%] done. truncating 9gb_28e6_def_assembly/9gb_28e6_def_d_tmp/9gb_28e6_def_int_posmatchf_pass.1.bin from 27265753872 to 24855669672 truncating 9gb_28e6_def_assembly/9gb_28e6_def_d_tmp/9gb_28e6_def_int_posmatchc_pass.1.bin from 26681440536 to 24590752800 Skim summary: accepted: 2954471258 possible: 201784884 permbans: 0 Hits chosen: 2698156192 Localtime: Thu Aug 25 15:19:23 2011 Total megahubs: 0 System memory: 67587063808 Mem2keepfree: 10138059571 Used by MIRA: 19259060224 Mem avail: 38189944013 rsh increased memtouse to: 38189944013 Can load up to 563714375 skim edges at once. Partitioning into 8 blocks. We have 4120535206 skims in file. Localtime: Thu Aug 25 15:19:26 2011 De-normalising SKIM hits ... (this will take a while) Loading skims in id range 0 to 943933 Localtime: Thu Aug 25 15:24:55 2011 Loaded 536870834 elements. Sorting ... done. Localtime: Thu Aug 25 15:29:14 2011 Writing normalised skimblock 0 ( 16.0 GiB) ... done. Loading skims in id range 943934 to 1966089 Localtime: Thu Aug 25 15:38:35 2011 Loaded 536870748 elements. Sorting ... done. Localtime: Thu Aug 25 15:42:54 2011 Writing normalised skimblock 943934 ( 16.0 GiB) ... done. Loading skims in id range 1966090 to 3068540 Localtime: Thu Aug 25 15:52:21 2011 Loaded 536870088 elements. Sorting ... done. Localtime: Thu Aug 25 15:56:39 2011 Writing normalised skimblock 1966090 ( 16.0 GiB) ... done. Loading skims in id range 3068541 to 4304483 Localtime: Thu Aug 25 16:06:07 2011 Loaded 536870724 elements. Sorting ... done. Localtime: Thu Aug 25 16:10:22 2011 Writing normalised skimblock 3068541 ( 16.0 GiB) ... done. Loading skims in id range 4304484 to 5725668 Localtime: Thu Aug 25 16:19:37 2011 Loaded 536870440 elements. Sorting ... done. Localtime: Thu Aug 25 16:23:56 2011 Writing normalised skimblock 4304484 ( 16.0 GiB) ... done. Loading skims in id range 5725669 to 7463490 Localtime: Thu Aug 25 16:33:12 2011 Loaded 536870762 elements. Sorting ... done. Localtime: Thu Aug 25 16:37:37 2011 Writing normalised skimblock 5725669 ( 16.0 GiB) ... done. Loading skims in id range 7463491 to 9849829 Localtime: Thu Aug 25 16:47:12 2011 Loaded 536870762 elements. Sorting ... done. Localtime: Thu Aug 25 16:51:35 2011 Writing normalised skimblock 7463491 ( 16.0 GiB) ... done. Loading skims in id range 9849830 to 13999999 Localtime: Thu Aug 25 17:01:44 2011 Loaded 362440848 elements. Sorting ... done. Localtime: Thu Aug 25 17:04:33 2011 Writing normalised skimblock 9849830 ( 10.8 GiB) ... done. Localtime: Thu Aug 25 17:07:01 2011 Loading block 0 Loading 536870834 elements at offset 0 Loaded 536870834 elements Loading block 1 Loading 536870748 elements at offset 17179866688 Loaded 536870748 elements Loading block 2 Loading 536870088 elements at offset 34359730624 Loaded 536870088 elements Loading block 3 Loading 536870724 elements at offset 51539573440 Loaded 536870724 elements Loading block 4 Loading 536870440 elements at offset 68719436608 Loaded 536870440 elements Loading block 5 Loading 536870762 elements at offset 85899290688 Loaded 536870762 elements Loading block 6 Loading 536870762 elements at offset 103079155072 Loaded 536870762 elements Loading block 7 Loading 362440848 elements at offset 120259019456 Loaded 362440848 elements Localtime: Thu Aug 25 17:25:37 2011 Step 0 Loading block 0 Loading 536870834 elements at offset 0 Loaded 536870834 elements Loading block 1 Loading 536870748 elements at offset 17179866688 Loaded 536870748 elements Loading block 2 Loading 536870088 elements at offset 34359730624 Loaded 536870088 elements Loading block 3 Loading 536870724 elements at offset 51539573440 Loaded 536870724 elements Loading block 4 Loading 536870440 elements at offset 68719436608 Loaded 536870440 elements Loading block 5 Loading 536870762 elements at offset 85899290688 Loaded 536870762 elements Loading block 6 Loading 536870762 elements at offset 103079155072 Loaded 536870762 elements Loading block 7 Loading 362440848 elements at offset 120259019456 Loaded 362440848 elements Localtime: Thu Aug 25 17:58:10 2011 Only short reads Step 10 Loading block 0 Loading 536870834 elements at offset 0 Loaded 536870834 elements Loading block 1 Loading 536870748 elements at offset 17179866688 Loaded 536870748 elements Loading block 2 Loading 536870088 elements at offset 34359730624 Loaded 536870088 elements Loading block 3 Loading 536870724 elements at offset 51539573440 Loaded 536870724 elements Loading block 4 Loading 536870440 elements at offset 68719436608 Loaded 536870440 elements Loading block 5 Loading 536870762 elements at offset 85899290688 Loaded 536870762 elements Loading block 6 Loading 536870762 elements at offset 103079155072 Loaded 536870762 elements Loading block 7 Loading 362440848 elements at offset 120259019456 Loaded 362440848 elements Total skims taken: 43278705 Step 20 Loading block 0 Loading 536870834 elements at offset 0 Loaded 536870834 elements Loading block 1 Loading 536870748 elements at offset 17179866688 Loaded 536870748 elements Loading block 2 Loading 536870088 elements at offset 34359730624 Loaded 536870088 elements Loading block 3 Loading 536870724 elements at offset 51539573440 Loaded 536870724 elements Loading block 4 Loading 536870440 elements at offset 68719436608 Loaded 536870440 elements Loading block 5 Loading 536870762 elements at offset 85899290688 Loaded 536870762 elements Loading block 6 Loading 536870762 elements at offset 103079155072 Loaded 536870762 elements Loading block 7 Loading 362440848 elements at offset 120259019456 Loaded 362440848 elements Total skims taken: 94355820 Step 30 Loading block 0 Loading 536870834 elements at offset 0 Loaded 536870834 elements Loading block 1 Loading 536870748 elements at offset 17179866688 Loaded 536870748 elements Loading block 2 Loading 536870088 elements at offset 34359730624 Loaded 536870088 elements Loading block 3 Loading 536870724 elements at offset 51539573440 Loaded 536870724 elements Loading block 4 Loading 536870440 elements at offset 68719436608 Loaded 536870440 elements Loading block 5 Loading 536870762 elements at offset 85899290688 Loaded 536870762 elements Loading block 6 Loading 536870762 elements at offset 103079155072 Loaded 536870762 elements Loading block 7 Loading 362440848 elements at offset 120259019456 Loaded 362440848 elements Total skims taken: 94355820 Step 40 Loading block 0 Loading 536870834 elements at offset 0 Loaded 536870834 elements Loading block 1 Loading 536870748 elements at offset 17179866688 Loaded 536870748 elements Loading block 2 Loading 536870088 elements at offset 34359730624 Loaded 536870088 elements Loading block 3 Loading 536870724 elements at offset 51539573440 Loaded 536870724 elements Loading block 4 Loading 536870440 elements at offset 68719436608 Loaded 536870440 elements Loading block 5 Loading 536870762 elements at offset 85899290688 Loaded 536870762 elements Loading block 6 Loading 536870762 elements at offset 103079155072 Loaded 536870762 elements Loading block 7 Loading 362440848 elements at offset 120259019456 Loaded 362440848 elements Total skims taken: 94355820 Step 50 Loading block 0 Loading 536870834 elements at offset 0 Loaded 536870834 elements Loading block 1 Loading 536870748 elements at offset 17179866688 Loaded 536870748 elements Loading block 2 Loading 536870088 elements at offset 34359730624 Loaded 536870088 elements Loading block 3 Loading 536870724 elements at offset 51539573440 Loaded 536870724 elements Loading block 4 Loading 536870440 elements at offset 68719436608 Loaded 536870440 elements Loading block 5 Loading 536870762 elements at offset 85899290688 Loaded 536870762 elements Loading block 6 Loading 536870762 elements at offset 103079155072 Loaded 536870762 elements Loading block 7 Loading 362440848 elements at offset 120259019456 Loaded 362440848 elements Total skims taken: 604663052 Step 55 Loading block 0 Loading 536870834 elements at offset 0 Loaded 536870834 elements Loading block 1 Loading 536870748 elements at offset 17179866688 Loaded 536870748 elements Loading block 2 Loading 536870088 elements at offset 34359730624 Loaded 536870088 elements Loading block 3 Loading 536870724 elements at offset 51539573440 Loaded 536870724 elements Loading block 4 Loading 536870440 elements at offset 68719436608 Loaded 536870440 elements Loading block 5 Loading 536870762 elements at offset 85899290688 Loaded 536870762 elements Loading block 6 Loading 536870762 elements at offset 103079155072 Loaded 536870762 elements Loading block 7 Loading 362440848 elements at offset 120259019456 Loaded 362440848 elements Total skims taken: 613899161 Step 56 Loading block 0 Loading 536870834 elements at offset 0 Loaded 536870834 elements Loading block 1 Loading 536870748 elements at offset 17179866688 Loaded 536870748 elements Loading block 2 Loading 536870088 elements at offset 34359730624 Loaded 536870088 elements Loading block 3 Loading 536870724 elements at offset 51539573440 Loaded 536870724 elements Loading block 4 Loading 536870440 elements at offset 68719436608 Loaded 536870440 elements Loading block 5 Loading 536870762 elements at offset 85899290688 Loaded 536870762 elements Loading block 6 Loading 536870762 elements at offset 103079155072 Loaded 536870762 elements Loading block 7 Loading 362440848 elements at offset 120259019456 Loaded 362440848 elements Total skims taken: 613926272 Step 60 Loading block 0 Loading 536870834 elements at offset 0 Loaded 536870834 elements Loading block 1 Loading 536870748 elements at offset 17179866688 Loaded 536870748 elements Loading block 2 Loading 536870088 elements at offset 34359730624 Loaded 536870088 elements Loading block 3 Loading 536870724 elements at offset 51539573440 Loaded 536870724 elements Loading block 4 Loading 536870440 elements at offset 68719436608 Loaded 536870440 elements Loading block 5 Loading 536870762 elements at offset 85899290688 Loaded 536870762 elements Loading block 6 Loading 536870762 elements at offset 103079155072 Loaded 536870762 elements Loading block 7 Loading 362440848 elements at offset 120259019456 Loaded 362440848 elements Total skims taken: 613936555 Step solexa by critlevel rsh4_takeSolexaByCritLevel. Loading block 0 Loading 536870834 elements at offset 0 Loaded 536870834 elements Loading block 1 Loading 536870748 elements at offset 17179866688 Loaded 536870748 elements Loading block 2 Loading 536870088 elements at offset 34359730624 Loaded 536870088 elements Loading block 3 Loading 536870724 elements at offset 51539573440 Loaded 536870724 elements Loading block 4 Loading 536870440 elements at offset 68719436608 Loaded 536870440 elements Loading block 5 Loading 536870762 elements at offset 85899290688 Loaded 536870762 elements Loading block 6 Loading 536870762 elements at offset 103079155072 Loaded 536870762 elements Loading block 7 Loading 362440848 elements at offset 120259019456 Loaded 362440848 elements Taken 26995105 hits. Step template overlaps Loading block 0 Loading 536870834 elements at offset 0 Loaded 536870834 elements Loading block 1 Loading 536870748 elements at offset 17179866688 Loaded 536870748 elements Loading block 2 Loading 536870088 elements at offset 34359730624 Loaded 536870088 elements Loading block 3 Loading 536870724 elements at offset 51539573440 Loaded 536870724 elements Loading block 4 Loading 536870440 elements at offset 68719436608 Loaded 536870440 elements Loading block 5 Loading 536870762 elements at offset 85899290688 Loaded 536870762 elements Loading block 6 Loading 536870762 elements at offset 103079155072 Loaded 536870762 elements Loading block 7 Loading 362440848 elements at offset 120259019456 Loaded 362440848 elements Step NAO rsh4_takeNeedAllOverlaps. Loading block 0 Loading 536870834 elements at offset 0 Loaded 536870834 elements Loading block 1 Loading 536870748 elements at offset 17179866688 Loaded 536870748 elements Loading block 2 Loading 536870088 elements at offset 34359730624 Loaded 536870088 elements Loading block 3 Loading 536870724 elements at offset 51539573440 Loaded 536870724 elements Loading block 4 Loading 536870440 elements at offset 68719436608 Loaded 536870440 elements Loading block 5 Loading 536870762 elements at offset 85899290688 Loaded 536870762 elements Loading block 6 Loading 536870762 elements at offset 103079155072 Loaded 536870762 elements Loading block 7 Loading 362440848 elements at offset 120259019456 Loaded 362440848 elements Taken 0 hits. Total skims taken: 642968188 Filtering forward skims. Localtime: Thu Aug 25 22:03:31 2011 Writing reduced skim file: [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] Done. Done. Filtering complement skims. Localtime: Thu Aug 25 22:17:47 2011 Writing reduced skim file: [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] Done. Done all filtering. Localtime: Thu Aug 25 22:31:41 2011 Making alignments. Localtime: Thu Aug 25 22:31:47 2011 Aligning possible forward matches: [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] Alignment stats: Potential: 338871153 Calculated: 9468 Evaded (PB): 0 Rejected (checkfun): 0 Trans 100 saved: 338861684 Banned overlap pairs: 349 in 127 sets. Localtime: Thu Aug 25 22:49:50 2011 Aligning possible complement matches: [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] Alignment stats: Potential: 304097035 Calculated: 37183 Evaded (PB): 54 Rejected (checkfun): 0 Trans 100 saved: 304059796 Banned overlap pairs: 931 in 508 sets. Localtime: Thu Aug 25 23:01:19 2011 Localtime: Thu Aug 25 23:01:19 2011 Counting number of alignments in file ... done. Expecting 512423414 alignments. Localtime: Thu Aug 25 23:03:51 2011 Loading confirmed overlaps from disk (will need approximately 28.6 GiB RAM): [0%] ....|.... [10%] ....|.... [20%] ....|.... [30%] ....|.... [40%] ....|.... [50%] ....|.... [60%] ....|.... [70%] ....|.... [80%] ....|.... [90%] ....|.... [100%] Internal logic/programming/debugging error (*sigh* this should not have happened). Please file a bug report on http://sourceforge.net/apps/trac/mira-assembler/ "Error while loading adsfacts, less facts in file than calculated earlier!" ->Thrown: void Assembly::loadAlignmentsFromFile(int32 version, const string prefix, const string postfix, const string tmpfname) ->Caught: void Assembly::loadAlignmentsFromFile(int32 version, const string prefix, const string postfix, const string tmpfname) Aborting process, probably due to an internal error. If you want to report the error, please do so on http://sourceforge.net/apps/trac/mira-assembler/ and also give a short notice on the mira talk mailing list. If reporting, please do not delete the log and checkpoint directories, there may be files in them which could be needed to find the problem. Subscribing / unsubscribing to mira talk, see: //www.freelists.org/list/mira_talk CWD: /tmp/1106239.scheduler.v4linux Thank you for noticing that this is *NOT* a crash, but a controlled program stop.