[yunqa.de] Re: Out of Memory
- From: Dominique Devienne <ddevienne@xxxxxxxxx>
- To: yunqa@xxxxxxxxxxxxx
- Date: Thu, 19 Sep 2019 09:11:08 +0200
On Wed, Sep 18, 2019 at 5:36 PM Delphi Inspiration <delphi@xxxxxxxx> wrote:
The current SQLiteSpy 1.9.13 is Win32, not Win64. I do not know
Dependency Walker and can not say why it gives you different information.
http://www.dependencywalker.com/
The tool shows you DLL dependencies of an EXE, and shows a different icon
for 64-bit DLLs. Most MLs don't allow attachments, so I'm not embedding a
picture.
Based on your feedback, I double-checked, and the DLLs it depends on are
64-bit,
but the EXE itself is not (I compared with a real 64-bit exe). So my
mistake indeed.
Win32 has a 2 GB memory limit. Your largest JSON doc apparently fits
into that limit. However, SQLite parses JSON strings into a node tree
before answering questions like json_array_length(). Depending on the
structure of your JSON string, this node tree can consume much more
memory than the JSON string. It is possible that this causes SQLiteSpy
to run out of memory. However, this is only a guess and may not be the
exact reason.
Well, the UI itself is also using memory I'm sure, so there's that too.
2 GB is just tiny with the kind of databases I'm dealing with.
How can SQLiteSpy use more RAM, if available? Two options:
* I can provide a Win64 binary. So far I did not do this to avoid
frustrations if users download the 64-bit SQLiteSpy for their 32-bit
Windows.
Yes please!
The fact you have both a Win32 and Win64 link fooled me into thinking this
was a real 64-bit process, which can use tons of RAM, which I have. I'm
actually
surprised you'd do that, have two links pointing to the same EXE honestly.
The two things I like most about SQLiteSpy* is its speed and simplicity, but
its current inability to deal with large databases is just a deal breaker
for me I'm afraid.
And since it's closed source, I can't rebuild it myself (and I'm not a
Delphi dev anyway)
so I would really appreciate a true 64-bit EXE, only restricted by the
available RAM on
the machine and not the limited address space of a 32-bit EXE.
* I wrote simplicity above, but I did send requests for enhancements,
because there are
definitely a things more things I'd like from SQLiteSpy. I was a bit
bummed but your lack
of answer, I thought at one point the list had gone dead in fact...
* For SQLiteSpy Win32, I can enable the IMAGE_FILE_LARGE_ADDRESS_AWARE
flag. This allows Win32 applications to use up to 4 GB of memory on
Win64. For details, see
https://docs.microsoft.com/en-us/windows/win32/memory/4-gigabyte-tuning
That wouldn't do me much good I'm afraid. 4GB is still too small for the
kind of data I'm dealing with.
For reference, can you run your query using the Win32 sqlite3.exe
command line tool and report the result? Here is the download:
https://www.sqlite.org/download.html
I have many versions of SQLite, but the pre-built Windows binaries are also
Win32 there.
I can easily build my own 64-bit shell, but the query I mentioned above is
just one. Another
returns 26M rows, and at the DOS prompt (which is famously slow at
printing) that's just not
manageable. I know of LIMIT and OFFSET too, I've been using SQLite for
years FWIW.
Ralf
Thanks for your answer. --DD
On 17.09.2019 16:37, Dominique Devienne wrote:
Working while a large 26GB DB, I keep getting out of memory errors,
[snip]
Other related posts:
