[py-lmdb] Re: MapFullError - MDB_MAP_FULL

  • From: David Wilson <dw@xxxxxxxx>
  • To: py-lmdb@xxxxxxxxxxxxx
  • Date: Fri, 23 May 2014 16:03:35 +0000

On Fri, May 23, 2014 at 08:52:19AM -0700, Dinesh Vadhia wrote:

> Got a mdb_map_full error twice on a 64-bit Python 2.7 environment with a
> map_size=2gb.  Using the guidance of 8+key+value, the populating data should
> have comfortably fitted around 1.7gb.  Best solution is to increase the
> map_size but wondered if I'm missing something?

Hey Dinesh,

Actual environment size depends a lot on the access pattern used to
build the database. I am a little fuzzy on why this is the case, but
it's common in B-tree implementations. Expect as much as 50% difference
between your data size and the resulting database size.

You may notice after fully building your database, that running
"python -m lmdb rewrite -e source.lmdb -E dest.lmdb" produces a much
smaller database, since "rewrite" writes everything sequentally.


Other related posts: