The times I mentioned were from logging into CX, to the record / comic count
reaching 2.7k – I referenced the donut chart or comic view just to give you an
idea of what sort of navigation was happening (i.e. none).
“Another part to look into is to have Angular not try to load all cover images
but only those for the current page of comics being viewed. That should also
drop the performance hits on initial load.”
Ah...if Angular is loading every cover image, that could be a problem – and
would explain why I was seeing so many “get cover” messages - 2.7k of images
being loaded would definitely cause performance issues on a hardware light
machine. I’ve formatted all my comics to webp, so the file sizes are pretty
small...but I’d imagine a full HD cover in JPG format would be quite large
(relatively speaking).
I just created a new library with 300 comics....and CX is nice and snappy – I
was seeing lots of cover loads, but the menu lag I see with my full library
didn’t exist. For sure I can see why the problems I’m coming across would
surprise you.
Here’s a log link (let me know if you can’t access it):
https://1drv.ms/u/s!AnMfIf1tfCd8gckixBSRkFgbs2CEww?e=8pY5bd
At line 196, or time-stamp 2021-04-03 17:18:15.600, you’ll see a bunch of “get
cover” messages that refer to the comic file name. There are three sections
like that – obviously corresponding with the total number of comics in the
library (getting 100 comics in each load). But...the CX DB is only displaying
the default 10 comic images on the first library screen.
There’s also a bunch of “Looking for cached image” messages...about 72 give or
take. I’m pretty sure I didn’t open a comic, so not sure why CX would be
looking for more than 10 images, which is also odd seeing as it’s already
searched the files for cover images.
I can get a log from my full library if that would be useful – it’d just take
around 12 odd minutes to do >_<
From: Darryl L. Pierce
Sent: Saturday, 3 April 2021 1:44 AM
To: comixed-dev@xxxxxxxxxxxxx
Subject: [comixed-dev] Re: Performance and the initial library load
My test library is a few hundred comics, but I drop them, add a bunch, etc. as
I'm testing. Also, the old chart was displayed before the library was fully
loaded, so that's likely not a fair comparison. Likely 0.8 is loading at the
same rate as 0.7 (not much changed in that regard between the two versions),
it's just that 0.8 takes you right to the library view and you see straight
away that it's in the process of loading comics. That's also likely why you're
seeing the "loading covers" output right away, since it's actively doing that
for the library view, whereas the old chart view was only loading records and
not retrieve images out of them.
I'd appreciate anything you can capture in a log to share to help understand
the performance issues.
BTW, I'm (again, ::sigh::) reworking the initial loading. I had dropped they
old HTTP loading and monitor for a pure message bus loading and monitoring. But
the load via message bus felt sluggish, so I've refactored things to a hybrid
model (I emailed the list about this). I'm hoping it can be made more
streamlined and faster.
Another part to look into is to have Angular not try to load all cover images
but only those for the current page of comics being viewed. That should also
drop the performance hits on initial load.
On Wed, Mar 31, 2021 at 11:30 PM Guy Incognito <dmarc-noreply@xxxxxxxxxxxxx>
wrote:
Out of curiosity, how large is your test library?
On a library of 2.7k archives, 0.7 completes the initial load in about 40
seconds. 0.8 took a few seconds over 12 minutes.
In both cases that’s the time from pressing the login button to the initial
screen being loaded (the donut chart in 0.7, and the library display (set to
only show 10 comics) in 0.8) and the library file count to reach 2.7k.
I’d need to capture a log for anybody else to see, but watching the CMD window
with 0.8, there was a lot of “getting cover” type messages. Way more than 10.
If I had to guess, I’d say around 2.7k. Once that was done, then the window
was filled with similar messages – more along the lines of “getting hash” type
though, but it was always the first page in the archive as far as I could
tell....again, I’d guess around 2.7k of them.
After that the log settled down – and was doing the “checking for changes” type
things, but while everything was going on the menu’s weren’t snappy at
all...0.7 is very responsive, while 0.8 displayed lags of a few seconds before
the desired action was completed (even if this was simply expanding the menu).
I’m not entirely sure if there’s a problem with my system – but this happens on
an aging Surface Pro and a desktop machine, both running Windows 10 pro.
I should get some free time over the long weekend, so I’ll set up a smaller
test library to see how that goes.
From: Darryl L. Pierce
Sent: Thursday, 1 April 2021 2:44 PM
To: comixed-dev@xxxxxxxxxxxxx
Subject: [comixed-dev] Performance and the initial library load
So Guy's email really has me thinking about this topic, and how we can improve
the initial load time for the library.
The old method performed ongoing HTTP requests to the backend to load a maximum
of 100 comics that have last-updated date greater than a specified timestamp.
This was a pretty robust way to load the initial set of comics, but a horrible
means for keeping the frontend notified of changes to the library afterward.
With 0.9 I've moved things to a messaging busy using STOMP to both load the
comics and also receive updates as comics are changed in the library. This is
fairly quick but I've not collected any metrics to compare the load times. The
main goal with going to the messaging bus is the reduce the complexity for
getting updates on the various parts of the library as they change.
What I'm thinking about doing is moving us to a more hybrid approach: the
initial load would use HTTP requests until all comics are loaded, and subscribe
to the messaging bus for updates as things change.
Does anybody have any opinion on this? MY work-project we're now doing the
hybrid approach and have a similar challenge of load a LOT of records on
initial login, then receive updates as things change over time.
--
Darryl L. Pierce <mcpierce@xxxxxxxxx>
"Le centre du monde est partout." - Blaise Pascal
"Let's try and find some point of transcendence and leap together." - Gord
Downie
--
Darryl L. Pierce <mcpierce@xxxxxxxxx>
"Le centre du monde est partout." - Blaise Pascal
"Let's try and find some point of transcendence and leap together." - Gord
Downie
