[mira_talk] Re: Log files

OK, here we go, the result of
./mira_2.9.42_dev_linux-gnu_x86_64/bin/mira --project=clado
--job=denovo,genome,accurate,454 -GE:not=4 -CL:pvlc=no
-SKIM:max_megahub_ratio=1:mchr=5120 -OUT:rrol=yes

is this :
sysop@dev3:/data/jvh_mira/mira$ du -shc *
4.0K    454linker.fasta
51M     bin
8.0K    clado_d_info
199G    clado_d_log
4.0K    clado_d_results
1.2G    clado_in.454.fasta
3.4G    clado_in.454.fasta.qual
1.1G    clado_traceinfo_in.454.xml
28M     log_assembly
28M     log_assembly_pvlc
28M     log_assembly_pvlc2
28M     log_assembly_pvlc3
28M     log_assembly_SKIM
28M     log.assembly.txt
39M     mira_2.9.42_dev_linux-gnu_x86_64
9.7M    mira_2.9.42_dev_linux-gnu_x86_64.tar.bz2
15M     mira.memtrack
36K     mkvtree.memtrack
205G    total

The contents of the log directory looks like this :
sysop@dev3:/data/jvh_mira/mira/clado_d_log$ ls -lSh
total 199G
-rw-r--r-- 1 sysop sysop  85G 2009-03-28 05:02
clado_int_posmatchf_preassembly.0.lst
-rw-r--r-- 1 sysop sysop  84G 2009-03-28 05:02
clado_int_posmatchc_preassembly.0.lst
-rw-r--r-- 1 sysop sysop 2.5G 2009-03-27 23:56 stattmpc
-rw-r--r-- 1 sysop sysop 2.3G 2009-03-27 23:56 stattmp7
-rw-r--r-- 1 sysop sysop 2.2G 2009-03-27 23:56 stattmp2
-rw-r--r-- 1 sysop sysop 1.9G 2009-03-27 23:56 stattmpd
-rw-r--r-- 1 sysop sysop 1.9G 2009-03-27 23:56 stattmp8
-rw-r--r-- 1 sysop sysop 1.9G 2009-03-27 23:56 stattmp3
-rw-r--r-- 1 sysop sysop 1.8G 2009-03-27 23:56 stattmp9
-rw-r--r-- 1 sysop sysop 1.8G 2009-03-27 23:56 stattmpf
-rw-r--r-- 1 sysop sysop 1.8G 2009-03-27 23:56 stattmp0
-rw-r--r-- 1 sysop sysop 1.8G 2009-03-27 23:56 stattmp6
-rw-r--r-- 1 sysop sysop 1.7G 2009-03-27 23:56 stattmp1
-rw-r--r-- 1 sysop sysop 1.7G 2009-03-27 23:56 stattmpb
-rw-r--r-- 1 sysop sysop 1.6G 2009-03-27 23:56 stattmp5
-rw-r--r-- 1 sysop sysop 1.6G 2009-03-27 23:56 stattmpa
-rw-r--r-- 1 sysop sysop 1.5G 2009-03-27 23:56 stattmpe
-rw-r--r-- 1 sysop sysop 1.5G 2009-03-27 23:56 stattmp4
-rw-r--r-- 1 sysop sysop 1.2G 2009-03-28 00:09 hashstat
-rw-r--r-- 1 sysop sysop 345M 2009-03-28 00:19 clado_int_clippings.0.txt
-rw-r--r-- 1 sysop sysop 125M 2009-03-27 23:48 clado_readpoolinfo.lst
-rw-r--r-- 1 sysop sysop 2.7M 2009-03-27 23:48 clado_info_reads_tooshort
-rw-r--r-- 1 sysop sysop    0 2009-03-27 23:47 clado_error_reads_invalid
-rw-r--r-- 1 sysop sysop    0 2009-03-28 05:02
clado_int_posmatch_megahubs_preassembly.0.lst
-rw-r--r-- 1 sysop sysop    0 2009-03-28 05:02
clado_int_posmatch_multicopystat_preassembly.0.txt
-rw-r--r-- 1 sysop sysop    0 2009-03-27 23:48 miralog.noqualities

I have attached the log.

I hope you can fix this.

With kind regards,
Jan

On Mon, Mar 23, 2009 at 18:08, Bastien Chevreux <bach@xxxxxxxxxxxx> wrote:

> On Monday 23 March 2009 Jan van Haarst wrote:
> > During assembly mira generates several hundred gigabytes in log files.
>
> Hello Jan,
>
> several 100 GiB? This is the first time I hear of such dimensions in the
> log
> directory.
>
> Can you please send me:
>
> 1) the first 2000 lines from the file you pipe the mira output to (e.g. for
>  "mira ...alotofoptions...  >&log.assembly.txt"
> then it's log_assembly.txt).
>
> 2) a "ls -l" of the log directory
>
> > Is there a way to stop this ?
>
> Partly yes. "-OUT:rrol=yes" should remove most of the things which are not
> needed, but it should be on by default.
>
> > I know that working without logs isn't optimal, but right now I'm not
> able
> > to finish anyway...
>
> Many of the files in the log directory are also temporary files from which
> MIRA
> needs to read again, so getting rid of them completely is not possible,
>
> > It would also help if I could pipe the generation of the logs through
> > gzip/bzip.
>
> That'd take too long for the temporary files. Might be an idea for the real
> logs though, I'll see what can be done.
>
> But first I'm curious to know which files are currently getting so big in
> your
> project ...
>
> Regards,
>  Bastien
>
>
> --
> You have received this mail because you are subscribed to the mira_talk
> mailing list. For information on how to subscribe or unsubscribe, please
> visit http://www.chevreux.org/mira_mailinglists.html
>



-- 
Dag,
Jan

Other related posts: