[mira_talk] R: Re: PC with 64 GB RAM: update
- From: "Davide Sassera (davide.sassera)" <davide.sassera@xxxxxxxx>
- To: mira_talk@xxxxxxxxxxxxx
- Date: Mon, 30 Jan 2012 09:37:54 +0100
I also have a 64G machine,
I was able to accurate de novo 5 mil pairs (10 Mil total) solexa reads (100bp).
When I tried to go 7.5Mil pairs it died saying something about too many skim
hits if I remember correctly.
I can send the log if needed.
best
D.
Il 30/01/12, Dimitar Kenanov <dimitar@xxxxxxxxxxxxxx> ha scritto:
> Hi Bastien,
> for now i ran MIRA with a 2M torrent reads and A. Thaliana ref genome with
> that command:
>
> mira --project=NON-316c1_17 --job=mapping,est,accurate,iontor -AS:nop=1
> -SB:bft=gbf:bbq=30 IONTOR_SETTINGS -SB:ads=yes:dsn=athali >& log_assembly.log
>
> It took 1.15 h and around 26Gb RAM.
>
> I tried the same data and with that one:
>
> mira --project=DIV-316c1_16 --job=mapping,genome,accurate,iontor -AS:nop=1
> -SB:bft=gbf:bbq=30 IONTOR_SETTINGS -SB:ads=yes:dsn=athali
>
> This one however did not finish correctly. I got the following error message
> which i have no idea how to interpret:
> ----------- LOG -------------
> "offsets1tos2<0 && abs(offsets1tos2)>=len2 ?"
>
> ->Thrown: void ADSEstimator::calcNewEstimate(int32 offsets1tos2, uint32 len1,
> uint32 len2, int32 id1, int32 id2, int8 id1dir, int8 id2dir)
> ->Caught: main
>
> Aborting process, probably due to an internal error.
> ----------- LOG --------------
>
>
> And here is the some info before the error occured:
>
>
> ------------- LOG -------------
>
> Backbone SW checks reverse:
> System memory: 67542007808
> Mem2keepfree: 10131301171
> Used by MIRA: 27041959936
> Mem avail: 30368746701
> rsh increased memtouse to: 30368746701
> tcmalloc: large alloc 2672017408 bytes == 0x57276e000 @
> Can load up to 83500326 skim edges at once.
> Partitioning into 1 blocks.
> We have 79524120 skims in file.
> Localtime: Fri Jan 27 18:25:24 2012
> De-normalising SKIM hits ... (this will take a while)
> Only one block ... first loop ...not loaded yet.
> Loading skims in id range 0 to 2004021
> Localtime: Fri Jan 27 18:25:28 2012
> Loaded 79524120 elements.
> Sorting ... done.
> Localtime: Fri Jan 27 18:25:39 2012
> Writing normalised skimblock 0 ( 2.4 GiB) ... done.
> Only one block ... not in first loop, return 0.
> Localtime: Fri Jan 27 18:25:40 2012
> Loading block 0
> Only one block, already loaded.
> Localtime: Fri Jan 27 18:25:41 2012
> Step rail
> Loading block 0
> Only one block, already loaded.
> something is really wrong with that ADSE!
> -922
> 108
> 140
> 1246186
> 594085
> 1
> 1
> adse: id1: 1246186 id2: 594085 le1: 922 re1: 0 le2: 0 re2: 890 ln1: 108 ln2:
> 140 tl: 1030 ol: 4294966514 dr1: 1 dr2: 1
> ========================== Memory self assessment
> ==============================
> Running in 64 bit mode.
>
> Dump from /proc/meminfo
> --------------------------------------------------------------------------------
> MemTotal: 65958992 kB
> MemFree: 9889588 kB
> Buffers: 43640 kB
> Cached: 28225132 kB
> SwapCached: 0 kB
> Active: 39364792 kB
> Inactive: 15763672 kB
> Active(anon): 26729828 kB
> Inactive(anon): 137540 kB
> Active(file): 12634964 kB
> Inactive(file): 15626132 kB
> Unevictable: 36 kB
> Mlocked: 36 kB
> SwapTotal: 1951888 kB
> SwapFree: 1951888 kB
> Dirty: 2485144 kB
> Writeback: 0 kB
> AnonPages: 26859904 kB
> Mapped: 147260 kB
> Shmem: 7648 kB
> Slab: 821940 kB
> SReclaimable: 796700 kB
> SUnreclaim: 25240 kB
> KernelStack: 3392 kB
> PageTables: 62572 kB
> NFS_Unstable: 0 kB
> Bounce: 0 kB
> WritebackTmp: 0 kB
> CommitLimit: 34931384 kB
> Committed_AS: 27618376 kB
> VmallocTotal: 34359738367 kB
> VmallocUsed: 307036 kB
> VmallocChunk: 34359427580 kB
> DirectMap4k: 53248 kB
> DirectMap2M: 3051520 kB
> DirectMap1G: 63963136 kB
> --------------------------------------------------------------------------------
>
> Dump from /proc/self/status
> --------------------------------------------------------------------------------
> Name: mira
> State: R (running)
> Tgid: 4927
> Pid: 4927
> PPid: 3975
> TracerPid: 0
> Uid: 1000 1000 1000 1000
> Gid: 100 100 100 100
> FDSize: 256
> Groups: 11 17 18 19 93 100
> VmPeak: 26408168 kB
> VmSize: 26408164 kB
> VmLck: 0 kB
> VmHWM: 26351184 kB
> VmRSS: 26341524 kB
> VmData: 26377236 kB
> VmStk: 340 kB
> VmExe: 3336 kB
> VmLib: 4580 kB
> VmPTE: 51548 kB
> Threads: 1
> SigQ: 0/515273
> SigPnd: 0000000000000000
> ShdPnd: 0000000000000000
> SigBlk: 0000000000000000
> SigIgn: 0000000000000000
> SigCgt: 0000000180000000
> CapInh: 0000000000000000
> CapPrm: 0000000000000000
> CapEff: 0000000000000000
> CapBnd: ffffffffffffffff
> Cpus_allowed: fff
> Cpus_allowed_list: 0-11
> voluntary_ctxt_switches: 63789
> nonvoluntary_ctxt_switches: 33490
> --------------------------------------------------------------------------------
>
> Information on current assembly object:
>
> AS_readpool: 2004022 reads.
> AS_contigs: 0 contigs.
> AS_bbcontigs: 5 contigs.
> Mem used for reads: 6526343832 (6.1 GiB)
>
> Memory used in assembly structures:
> Eff. Size Free cap. LostByAlign
> AS_writtenskimhitsperid: 2004022 8 MiB 0 B 0 B
> AS_skim_edges: 79524120 2.5 GiB 121 MiB 0 B
> AS_adsfacts: 0 24 B 0 B 0 B
> AS_confirmed_edges: 0 24 B 0 B 0 B
> AS_permanent_overlap_bans: 1 24 B 0 B 0 B
> AS_readhitmiss: 0 24 B 0 B 0 B
> AS_readhmcovered: 0 24 B 0 B 0 B
> AS_count_rhm: 0 24 B 0 B 0 B
> AS_clipleft: 2004022 8 MiB 0 B 0 B
> AS_clipright: 2004022 8 MiB 0 B 0 B
> AS_used_ids: 2004022 2 MiB 0 B 2 B
> AS_multicopies: 2004022 2 MiB 0 B 2 B
> AS_hasmcoverlaps: 0 2 MiB 2 MiB 2 B
> AS_maxcoveragereached: 2004022 8 MiB 0 B 0 B
> AS_coverageperseqtype: 0 24 B 0 B 0 B
> AS_istroublemaker: 2004022 2 MiB 0 B 2 B
> AS_isdebris: 2004022 2 MiB 0 B 2 B
> AS_needalloverlaps: 2004022 2 MiB 10 B 0 B
> AS_readsforrepeatresolve: 0 40 B 0 B 0 B
> AS_allrmbsok: 0 8 MiB 8 MiB 0 B
> AS_probablermbsnotok: 0 8 MiB 8 MiB 0 B
> AS_weakrmbsnotok: 0 8 MiB 8 MiB 0 B
> AS_readmaytakeskim: 2004022 2 MiB 10 B 0 B
> AS_skimstaken: 79524120 76 MiB 40 B 0 B
> AS_numskimoverlaps: 2004022 8 MiB 0 B 0 B
> AS_numleftextendskims: 2004022 8 MiB 0 B 0 B
> AS_rightextendskims: 2004022 8 MiB 0 B 0 B
> AS_skimleftextendratio: 2004022 2 MiB 0 B 2 B
> AS_skimrightextendratio: 2004022 2 MiB 0 B 2 B
> AS_usedtmpfiles: 10 336 B 0 B 0 B
> Total: 9376076632 (8.7 GiB)
>
> ================================================================================
> Dynamic allocs: 6
> Align allocs: 22
> ----------------------- LOG ------------------------
>
>
> Any idea what might have gone wrong?
>
> Dimitar
>
>
> On 01/28/2012 11:30 PM, Bastien Chevreux wrote:
> >On Jan 27, 2012, at 6:30 , Dimitar Kenanov wrote:
> >
> >>i got the new PC. It has CPU - i7 3930K and 64GB RAM. MBoard - Gigabyte
> >>X79-UD5.
> >>Well it works. From my Slackware linux i can see all of the RAM. I attach a
> >>JPG where in a window which runs 'htop' program one can see all the CPU
> >>threads and the RAM amount.
> >>Unfortunately didnt test it with MIRA yet. But i will for sure and report
> >>how the machine behaves.
> >>
> >Uh, that thing is making me jealous. Really :-)
> >
> >Have you tried whether a single process can allocate and use>32 GiB?
> >
> >B.
> >
> >
> >
>
>
>
> --
> You have received this mail because you are subscribed to the mira_talk
> mailing list. For information on how to subscribe or unsubscribe, please
> visit http://www.chevreux.org/mira_mailinglists.html
>
>
Other related posts:
- » [mira_talk] R: Re: PC with 64 GB RAM: update - Davide Sassera (davide.sassera)
