Mmmm....this is sounding familiar, I’ll need to revisit the open tickets to
refresh my memory.
Can you outline what happens when the library is being loaded (in laymen’s
terms)?
I have a rough idea, but want to double check before I say something stupid.
On 26/11/2019, at 05:06, Darryl L. Pierce <mcpierce@xxxxxxxxx> wrote:
With the code that I checked in yesterday, it should be a lot faster to load
since it's no longer going to try and load everything once you log in. It's
going to now start pulling strategic chunks of data as needed, only as much
as need be shown on the current page. The old code is still there as the
collections views still depend on it. But for the main library viewing it
only loads what it needs.
Regarding caching things, we had/have a ticket for caching the cover image so
they can be more quickly retrieved. I'm not sure if pulling them from a
database is going to be any faster than extracting them out of the comic
file, but my thought was to have CX maintain a thumbnails/caching directory
where it would story covers hierarchically by hash value (break up the hash
into 4-char pieces and each represents a subdirectory). Since file access is
fastest, it can get the hash, look for the file and, if it's not there, grab
it from the archive, cache it and return it. But I'm open to other ideas for
how to make it faster.
On Mon, Nov 25, 2019 at 5:32 AM bareheiny Alexander
<dmarc-noreply@xxxxxxxxxxxxx> wrote:
Over the years, I’ve been explored YAC, Ubooquity and ComicRack (something’s
happening here...the website is showing a new landing page).
As far as I can tell, all of the applications extract and store comic covers
– making the thumbnail generation a lot quicker. Is that something you’ve
considered for CR?
My (non-developer) thoughts are that cover display would be pulled from
pre-extracted images...statistics and filter fields (publisher, characters
etc.) would be pulled from the database via SQL (I’m a BI developer, so I
see SQL everywhere – if all you have is a hammer, all you see are nails as
they say) as the user navigates to the relevant pages – rather than having
to wait for the entire library to load and get everything populated.
As it stands, with CX trying to load the entire library and all
metadata....it takes far too long to load a large library (as I’ve mentioned
on GitHub, my session times out before the library fully loads). If there
is no metadata, the library loads quick smart (to be fully confirmed).
Not being an application developer, I’m likely missing vital information /
experience...I’m more than happy to be enlightened. Also happy to be told
that this isn’t the appropriate list for my input :)
From: Darryl L. Pierce
Sent: Monday, 25 November 2019 2:43 PM
To: comixed-dev@xxxxxxxxxxxxx
Subject: [comixed-dev] New library code for fetching
I spent time yesterday and today writing a new set of actions for the
library to move away from loading the whole library into the browser. What
it does now is, when you go to the library view, it fetches just the comics
for the current page. And as you move about, changing sorting, switching
pages, changing the number of comics to show, etc, it makes a REST call to
fetch the comics to display and refreshes the view.
It's going to be a process to refactor other pages, but what I'm thinking is
this:
1. Have the main page load specific statistics (rather than extracting it
from the comics as they're loaded) with a single REST API.
2. Have each of the collections page use a similar request (or enhancements
to the new request) to get comics in pages.
3. Change the current continuous update to do some other, simpler actions to
get statistics on the library rather than loading everything. Or just dump
that continuous update entirely.
And these are all things I think we can get done by the end of the year when
I want to put the 0.5 release out for RC.
I merged this one, but would like to start putting things up for PRs to get
feedback if you guys are comfortable with getting more hands-on now. :D
--
Darryl L. Pierce <mcpierce@xxxxxxxxx>
"Le centre du monde est partout." - Blaise Pascal
"Let's try and find some point of transcendence and leap together." - Gord
Downie
--
Darryl L. Pierce <mcpierce@xxxxxxxxx>
"Le centre du monde est partout." - Blaise Pascal
"Let's try and find some point of transcendence and leap together." - Gord
Downie
