Hello,
I wanted to ask about using Python multiprocessing / threading to prefetch
data using py-lmdb.
At the moment I have a loop that reads a chunk from a lmdb file using
py-lmdb and then performs a computation on that data. To speed up I would
like to 'hide' the IO by reading the next chunk of the lmdb while the
computation is happening.
In the documentation I found the statement `Environments may be opened by
multiple processes on the same host, making it ideal for working around
Python’s GIL.` But I found no example code and reading (and only partly
understanding) https://github.com/dw/py-lmdb/issues/86 and
https://github.com/dw/py-lmdb/issues/65 I got the impression that I what I
want is not 'easily' possible.
Appreciate any pointers
Thanks
Ludwig
--
*EyeEm*
Ludwig Schmidt-Hackenberg
Computer Vision Engineer
@shackenberg <http://www.eyeem.com/shackenberg>
EyeEm Mobile GmbH
Kohlfurter Strasse 4I/43, I0999 Berlin, Germany
Geschäftsführer: Florian Meissner & Lorenz Aschoff
Handelsregister Amtsgericht Charlottenburg HRBI3253I
