[py-lmdb] Allocating Max RAM memory to LMDB.

  • From: Offer Sharabi <offer@xxxxxxxxxxxx>
  • To: py-lmdb@xxxxxxxxxxxxx
  • Date: Tue, 24 Mar 2015 11:47:57 -0400

Allocating Max RAM memory to LMDB.

Hi David,

my name is Offer Sharabi, and I am a developer working for Magnetic, in
Julian Berman’s team.

We are planning to use the LMDB in our production environment where we have
limited RAM resource. In our current implementation we are using the python
LMDB client, and instantiate the DB from the python application.

When testing LMDB we found that it used up to 99% RAM capacity before it
started hitting disk. Unfortunately, this has severely slowed down the
servers and affected all other services including LMDB swap which jumped
from several ms to several seconds or even minutes.

I was hoping that there is some parameter that can be given to LMDB in
order to allocate max memory usage, and in this way better control
resources. However, I failed to find one in docs and was wondering :

Did I just missed it ?

If not, is there a way to control memory allocation ?

If not, is there a plan to introduce one ?

Finally, is there any other solution that you can think of for the problem?

Thanks in advance for your help on this,

Best ,

Offer Sharabi.


*Offer Sharabi *|* Software Developer*

magnetic.com <http://www.magnetic.com/>  | blog
<http://www.magnetic.com/magnetic-culture/>  |  t
<https://twitter.com/MagneticIs>witter <https://twitter.com/MagneticIs>  |
facebook <https://www.facebook.com/pages/Magnetic/297609126995926>  |
linkedin <http://www.linkedin.com/company/magnetic_2>  |  youtube

Other related posts: