Hi, I think we are moving a bit off the mira_talk topic. I am not a linux kernel guru, but I do remember some discussion I had on this in the past. I tried to find out the old email threads ... Decide yourself whether it still applies to your kernel version. George Marselis wrote: > On 17/8/10 8:41 PM, "Martin MOKREJŠ" <mmokrejs@xxxxxxxxxxxxxxxxxx> wrote: > >> Hi, > > Hello :) > > >> Swap is necessary for linux kernel to be able to handle memory management. >> It needs swap to determine it has not enough memory to start cleanup of >> allocated >> RAM by other tasks. This is quite commonly discussed at the linux-kernel >> mailing list. > > Actually, no. Swap is not needed in any modern Unix/Linux operating system. > You can perfectly setup a Linux system without any swap. Swap is a > secondary storage medium, left over from the days where memory cost as much > as a house to buy. Processes which step over the bounds of physical memory, > get swapped out to disk. It might be good practice to use swap, in case you > do run out of memory. That way the kernel will not panic and halt as fast. > But it is definitely not necessary. If you reach the end of both your I am quite sure I communicated about that with Alan Cox or Marcelo Tossati, I cannot find the email thread now. But it turned out I have to have at least _some_ swap assigned to the kernel. > physical and swap space, the kernel will panic, by default. It cannot free > RAM taken by other processes, just like that, as that would make it lose > data (bad). http://lwn.net/Articles/111409/ http://lkml.org/lkml/2004/11/22/57 http://lkml.org/lkml/2007/12/7/128 > > If the kernel determines that there is not enough memory (physical or on a > swap partition), it will first start flushing its own buffers, first. If the > memory does not suffice, AKA SOME FIRST YEAR STUDENT HAS BEEN MALLOCBOMBING > THE STUDENT MACHINE AGAIN (you know who you are), and your friendly > neighborhood sysadmin has not setup ulimits or is monitoring the machine And people trying to do bioinformatics usually set ulimit to unlimited once they figure out where to configure all the relevant places (sysctl, .profile, ^H^H^H^H^H .bashrc, etc.). ;) > with [insert favorite monitoring program, mine is nagios], well, good look > and I hope you got terminal open so you can start killing processes. But the And this is why this Russian patch was made for: so that you know that mira will be the first candidate on the particular machine to be killed: https://patchwork.kernel.org/patch/1900/ > kernel will NOT kill other processes. It may kill itself, (after it flushes > all the buffers to disk), so it will maintain the integrity of the data. No, it does kill other processes, even sshd, init, xdm. http://linux.derkeiler.com/Mailing-Lists/Kernel/2010-02/msg00518.html You can score candidate processes nowadays but you can only affect the likelihood the process gets killed: http://lkml.indiana.edu/hypermail/linux/kernel/0901.2/03144.html It is different from the Russian patch above, you still risk kernel will kill init, sshd, root's bash, or similarly critical process. > > Now, that is Unix. > > In the microsoft world, yes, swap is a requirement. The Windows kernel (up > to server 2008) preempts all processes to swap, so it will free main memory. > Hence us oldtimers remembers the time it was taking to switch between two > tasks in older versions of windows. > > I remember, in my days ;), where 16MB was 250 USD (1996 prices). These days, > where I got 8GB on my laptop, why would take precious space from my SSD for > swap? Let's say you got an Altix UV1000, with 3TB RAM. Is it really cost > effective to buy four times the amount of the memory and store it on FC/SAS > disks, with RAID, so it can keep up? Of course not. I would much rather have > that cash as a night out in Amsterdam :D > > >> Processes reserve memory in advance, so that is where reserving swap memory >> gives >> you the advantage of saving the only RAM you have. Just make swap (2-3)*36 >> GiB, >> "swapon -s /dev/sdXY" and be happy. > > Uh, no. No process reserves all the memory it needs in advance, unless > Tanenbaum finally came up with that latent telepathy module he was talking > about fifteen years ago. Hence malloc and dynamic memory allocation. Well, I was not very exact, you are right of course. ;-) > Processes request memory memory from the kernel and if the programmer has > not been stringent enough (you know who you are - *cough*newbler*cough*) it > will catch that failed malloc and it will fail gracefully and inform the > user that there is no more memory *cough*newbler*cough*Error9*cough*. When > they die, they release (hopefully/Use the Valgrind, Luke!) the memory, and > the kernel puts the memory back in the free buffer pool. > > >> Why RAID to slow down the swap allocation? You really need swap >> just to swap out few MB, you never want your drives to be >> constantly reading out and writing in the data out/into swap disks. >> Machines are usually not responding by that time and once that >> starts happening, you rather press the power button. > > Erh, swap allocation? Sorry? And if you swap, in large projects you do not > swap out "a few megabytes". > > And how will RAID slow down swapping memory, exactly? The mirror in RAID10 means it must be written twice instead of once. RAID5 gives you chance to have one drive failed, but I wonder who has e.g. 5 disks in raid5 configuration to sacrifice one of them for the parity. > > >> And when you run out of swap space, you know that linux kernel runs the >> random >> process killer and kills some process, one by one until it helps? Do yourself >> a favor and google-out the patches to modify this default behavior to your >> preference. Certainly people do not agree with this but definitely, this is >> for >> a long-time existing problem and the opinion of linux kernel gatekeepers is >> clear. > > ... The Debian-flavor of the kernel does not kill random processes. It > panics. It tells you you messed up. You did not monitory the machine > adequately. And for me, rightfully so. Last thing I need is a kernel that > kills random processes. I have heard this already from sysadmins: they are glad when the machine panics/reboots because that prevents user to start nasty jobs. ;-) Martin -- You have received this mail because you are subscribed to the mira_talk mailing list. For information on how to subscribe or unsubscribe, please visit http://www.chevreux.org/mira_mailinglists.html