[py-lmdb] Re: MapFullError - MDB_MAP_FULL
- From: David Wilson <dw@xxxxxxxx>
- To: py-lmdb@xxxxxxxxxxxxx
- Date: Fri, 23 May 2014 16:03:35 +0000
On Fri, May 23, 2014 at 08:52:19AM -0700, Dinesh Vadhia wrote:
> Got a mdb_map_full error twice on a 64-bit Python 2.7 environment with a
> map_size=2gb. Using the guidance of 8+key+value, the populating data should
> have comfortably fitted around 1.7gb. Best solution is to increase the
> map_size but wondered if I'm missing something?
Hey Dinesh,
Actual environment size depends a lot on the access pattern used to
build the database. I am a little fuzzy on why this is the case, but
it's common in B-tree implementations. Expect as much as 50% difference
between your data size and the resulting database size.
You may notice after fully building your database, that running
"python -m lmdb rewrite -e source.lmdb -E dest.lmdb" produces a much
smaller database, since "rewrite" writes everything sequentally.
David
Other related posts:
