[mira_talk] Re: Out of memory detected : std::bad_alloc

  • From: Jeremiah Davie <jdavie@xxxxxxxxxxx>
  • To: mira_talk@xxxxxxxxxxxxx
  • Date: Tue, 17 Aug 2010 14:00:49 -0400

Hi Martin,
I believe George's system specification isn't intended to completely swap out as much space as he has allocated, but rather to increase the speed of swap by taking advantage of the increased I/O speed of SSD drives and then further dividing that time by a 1/4 through the use of the RAID10. While I agree that for a few MB of disk space, the time savings would be minimal, the amount of swap on a really big Illumina project could totally justify that set up (provided I didn't have to pay for it myself ;) That said, I'm not terribly well versed in systems administration, so I would be very interested in knowing if my interpretation of George's set-up intentions are correct; I have been wrong before :) -Jeremiah


On Aug 17, 2010, at 1:41 PM, Martin MOKREJŠ wrote:

Hi,

George Marselis wrote:
On 17/8/10 5:28 PM, "Bastien Chevreux" <bach@xxxxxxxxxxxx> wrote:

[snip]
Your machine has a bad set-up: only 4 GiB swap for 36 GiB RAM is not ideal.

I am not quite sure about that: at 36GB RAM 99.9% of the [l][u| i]n[u|i]x out
there will not touch the swap, unless they really have to do, like in
specialized circumstances as such. If you have a 3TB smp machine, for
example, what do you use for swap?

Swap is necessary for linux kernel to be able to handle memory management. It needs swap to determine it has not enough memory to start cleanup of allocated RAM by other tasks. This is quite commonly discussed at the linux- kernel
mailing list.

Processes reserve memory in advance, so that is where reserving swap memory gives you the advantage of saving the only RAM you have. Just make swap (2-3)*36 GiB,
"swapon -s /dev/sdXY" and be happy.


Having said that, a neat idea I have been playing with includes a raid 10 on
4 intel SSDs, 160GB each, just for swap. The machine has 128GB RAM.

Why RAID to slow down the swap allocation? You really need swap just to swap out few MB, you never want your drives to be constantly reading out and writing in the data out/into swap disks. Machines are usually not responding by that time and
once that starts happening, you rather press the power button.

And when you run out of swap space, you know that linux kernel runs the random process killer and kills some process, one by one until it helps? Do yourself a favor and google-out the patches to modify this default behavior to your preference. Certainly people do not agree with this but definitely, this is for a long-time existing problem and the opinion of linux kernel gatekeepers is clear. Provide enough swap and make you you do not need it, and if one application eats too much memory, do not start. Linux kernel has no way to know which process cause
the memory pressure.

So, why do you want to have RAID over swapspace I do not understand.

Best,
Martin


--
You have received this mail because you are subscribed to the mira_talk mailing list. For information on how to subscribe or unsubscribe, please visit http://www.chevreux.org/mira_mailinglists.html




--
You have received this mail because you are subscribed to the mira_talk mailing 
list. For information on how to subscribe or unsubscribe, please visit 
http://www.chevreux.org/mira_mailinglists.html

Other related posts: