Created a new library - seeing same issue.
Steps I took:
1. Start CX with no existing library and log in.
2. Search for comics to import, select 50 and begin the import.
3. Wait for all comics to be fully imported.
4. Create reading list and add 5 comics to it.
5. Open reading list - all covers displayed with spinning circle overlay, and
“scheduled for processing” pop out text is displayed.
This is on the 0.5 pre release - although I see the same issues in 0.6 with
lists created in 0.5 and via SQL.
On 1/03/2020, at 08:31, bareheiny <bareheiny@xxxxxxxxx> wrote:
Yeah, sorry - that’s the log text I’m seeing, so from that perspective
everything is working as expected.
I’m going to create a clean database and load a sample of comics in to see if
anything I’ve done has broken my reading lists.
On 1/03/2020, at 08:05, Darryl L. Pierce <mcpierce@xxxxxxxxx> wrote:
Can you paste a sample of what you're seeing in the logs when you say
"searching for new comics"? That text doesn't show up in the code base
anywhere, so I'm not sure what you're seeing.
Is it something like the following:
2020-02-29 14:00:10.843 DEBUG 19118 --- [ Jarvis-ComiXed]
org.comixed.task.runner.Worker : Woke up: state=IDLE
2020-02-29 14:00:10.843 DEBUG 19118 --- [ Jarvis-ComiXed]
org.comixed.task.adaptors.TaskAdaptor : Getting the next task to run
2020-02-29 14:00:10.843 WARN 19118 --- [ Jarvis-ComiXed]
o.h.h.internal.ast.QueryTranslatorImpl : HHH000104: firstResult/maxResults
specified with collection fetch; applying in memory!
2020-02-29 14:00:10.844 DEBUG 19118 --- [ Jarvis-ComiXed]
org.comixed.task.runner.Worker : Waiting for task or notification
2020-02-29 14:00:10.902 DEBUG 19118 --- [nio-7171-exec-7]
o.c.service.library.LibraryService : Finding up to 101 comics updated
since Sun Feb 23 18:31:04 EST 2020 for comixedadmin@xxxxxxxxxxxxx
2020-02-29 14:00:10.903 DEBUG 19118 --- [nio-7171-exec-7]
o.c.service.library.LibraryService : Checking comic: timestamp:
1582500664121 >= 1582500664121 id: 101 > 101
2020-02-29 14:00:10.903 DEBUG 19118 --- [nio-7171-exec-7]
o.c.service.library.LibraryService : Returning 0 updated comics
2020-02-29 14:00:10.903 DEBUG 19118 --- [nio-7171-exec-7]
o.c.c.library.LibraryController : Getting processing count
2020-02-29 14:00:10.903 DEBUG 19118 --- [nio-7171-exec-7]
o.c.service.library.LibraryService : Getting processing count
2020-02-29 14:00:10.904 DEBUG 19118 --- [nio-7171-exec-7]
org.comixed.service.task.TaskService : Found 0 instances of ProcessComic
2020-02-29 14:00:10.904 DEBUG 19118 --- [nio-7171-exec-7]
o.c.c.library.LibraryController : Sleeping 1000ms
If so, that's expected behavior. Once a user logs in, the frontend sends a
request to the backend for comic updates. The backend sits on that request,
sleeping for 1000ms and then waking up to check for any comics that have
been updated since it went to sleep. If anything has changed, it returns
with the updated comics. Otherwise, if it times out, it returns that no
comics have been updated.
On Sat, Feb 29, 2020 at 2:53 AM Guy Incognito <dmarc-noreply@xxxxxxxxxxxxx>
wrote:
Apologies, I must have mis-read the log – I’m not seeing any looping.
What I am seeing is that the covers are being returned for the reading
lists, then the log goes back to the “searching for new comics” process.
The wheels keep spinning on the covers, but nothing else really happens ☹
From: bareheiny
Sent: Saturday, 29 February 2020 9:02 AM
To: comixed-dev@xxxxxxxxxxxxx
Subject: [comixed-dev] Re: Reading lists
I’ll grab a log when my surface is charged and I can use it again.
For now now, my library (now only 1,200 comics) is fully loaded and
processed.
I can happily navigate the issues, scraping as I go....but the reading
lists appear to be having differently.
I’m assuming I’m correct in thinking the reading lists just pull the
existing comic information and don’t trigger a re-process of each issue on
the list?
On 29/02/2020, at 02:28, Darryl L. Pierce <mcpierce@xxxxxxxxx> wrote:
The spinner means the comic itself hasn't been fully imported yet and is
queued up for processing. The flow for importing comics is:
1. The list of fully qualified filenames is submitted to the backend,
2. a QueueComicsWorkerTask instance is created, given the filenames, and
pushed into the Worker's queue, and
3. Worker runs this task, which creates a record in the tasks database for
each comic being imported.
Worker monitors the tasks table (it's how the backend keeps track of what
needs to be done as far as managing the library) and pops records out in
the order they were created in order to perform tasks. So after the above
steps complete, the tasks table has a bunch of records for comics to be
imported into the database.
1. When an add comic task record is loaded, an instance of
AddComicWorkerTask is created and pushed into the Worker's queue,
2. that task creates a record in the comics table, then
3. inserts a record into the tasks table to process the newly added comic.
At *this* point, the comic is visible from your browser as a library entry,
but it's not been processed yet. Each Comic object has a child object,
ComicFileDetails (stored in the comic_file_details table), which holds the
details of the physical file itself (currently the MD5 hash for the file).
When the comic is added in the above steps, this child object isn't
created. The system hasn't cracked open the comic file yet, it's only stuck
a record in the comics table. And, when the frontend receives a comic
without it's file details child object, it puts a spinner over the cover to
indicate the comic isn't processed yet.
The finally:
1. When the process comic task record is loaded, an instance of
ProcessComicTask is created an pushed into the Worker's queue, and
2. that task gets the MD5 hash for the comic's file, creates the
ComicFileDetails object for the comic and updates the database.
At this point an update is sent to the frontend with the details object and
the browser removes the spinner.
On Thu, Feb 27, 2020 at 7:53 PM Guy Incognito <dmarc-noreply@xxxxxxxxxxxxx>
wrote:
I used the 0.5 release to add comics to a reading list (I’ve also added
comics via SQL to another list).
Both lists take some time to display the title and description, and even
longer to load the thumbnails.
It also looks like the comics metdata isn’t being returned in a timely
manner – I’m seeing the spinning circles for some time, and when I opened
one comic it only had “unknown” type values against the series and volume
etc.
Made that particular reading list useless as it contained all the comics
that required scraping >_<
From: Darryl L. Pierce
Sent: Friday, 28 February 2020 8:52 AM
To: comixed-dev@xxxxxxxxxxxxx
Subject: [comixed-dev] Re: Reading lists
I'm not aware of any issues around loading reading lists. The only issue I
know ATM is that the option to add comics to them is gone in the develop
branch while I refactor the context menu system. It should be added back in
sometime in the next week or so.
What are the symptoms you're seeing? They're just not loading at all, or
they load but without any details?
On Wed, Feb 26, 2020 at 6:26 PM Guy Incognito <dmarc-noreply@xxxxxxxxxxxxx>
wrote:
Is there still work to be done on reading lists?
I ask as I created one with 1,200 unscraped comics....it takes ages to
load, and doesn’t display the series and volume information.
I did add the comics to the list via SQL, but I don’t see how that cause
any issues.
--
Darryl L. Pierce <mcpierce@xxxxxxxxx>
"Le centre du monde est partout." - Blaise Pascal
"Let's try and find some point of transcendence and leap together." - Gord
Downie
--
Darryl L. Pierce <mcpierce@xxxxxxxxx>
"Le centre du monde est partout." - Blaise Pascal
"Let's try and find some point of transcendence and leap together." - Gord
Downie
--
Darryl L. Pierce <mcpierce@xxxxxxxxx>
"Le centre du monde est partout." - Blaise Pascal
"Let's try and find some point of transcendence and leap together." - Gord
Downie
