I'm not seeing that in the 0.6 branch. I just ran a test importing 5 comic
files and they all imported with the spinner's disappearing as the
ProcessComicTask ended.
Doing the same thing on 0.5 for the same comics, the behavior is the same
as for 0.6 with the spinners disappearing as ProcessComicTask finishes
processing each comic.
In both cases the overall processing time for five comics is about 45
seconds.
On Sat, Feb 29, 2020 at 3:36 PM bareheiny <dmarc-noreply@xxxxxxxxxxxxx>
wrote:
The steps I mentioned were all done in 0.5
I had the comic view set to 100 items, so everything was displayed. No
sign of any of the spinners stopping, and noting in the logs indicating
anything was happening.
Will have another look later today, and will open a bug if needed.
I’m assuming nobody else has experienced this behaviour? My initial
thought about this was “what did I do to cause it?”...
On 1/03/2020, at 09:06, Darryl L. Pierce <mcpierce@xxxxxxxxx> wrote:
Between 0.5 and 0.6 the process comic task has changed, so I wouldn't run
0.6 until 0.5 finishes processing the comics. So I'd ask you to redo the
above steps but don't bring 0.6 into the mix.
After step #5 do you see the spinners going away one by one as each comic
is processed? Expand the # of comics displayed to show everything on one
page and see if spinner's go away one at a time. Is the log showing the
comics being processed?
If none of those things are happening, please open a bug against 0.5 and
we'll start putting together an 0.5.1 release. This would be a blocker that
should have stopped 0.5.0 from being released if it's not working.
On Sat, Feb 29, 2020 at 2:58 PM bareheiny <dmarc-noreply@xxxxxxxxxxxxx>
wrote:
Created a new library - seeing same issue.
Steps I took:
1. Start CX with no existing library and log in.
2. Search for comics to import, select 50 and begin the import.
3. Wait for all comics to be fully imported.
4. Create reading list and add 5 comics to it.
5. Open reading list - all covers displayed with spinning circle
overlay, and “scheduled for processing” pop out text is displayed.
This is on the 0.5 pre release - although I see the same issues in 0.6
with lists created in 0.5 and via SQL.
On 1/03/2020, at 08:31, bareheiny <bareheiny@xxxxxxxxx> wrote:
Yeah, sorry - that’s the log text I’m seeing, so from that perspective
everything is working as expected.
I’m going to create a clean database and load a sample of comics in to
see if anything I’ve done has broken my reading lists.
On 1/03/2020, at 08:05, Darryl L. Pierce <mcpierce@xxxxxxxxx> wrote:
Can you paste a sample of what you're seeing in the logs when you say
"searching for new comics"? That text doesn't show up in the code base
anywhere, so I'm not sure what you're seeing.
Is it something like the following:
2020-02-29 14:00:10.843 DEBUG 19118 --- [ Jarvis-ComiXed]
org.comixed.task.runner.Worker : Woke up: state=IDLE
2020-02-29 14:00:10.843 DEBUG 19118 --- [ Jarvis-ComiXed]
org.comixed.task.adaptors.TaskAdaptor : Getting the next task to run
2020-02-29 14:00:10.843 WARN 19118 --- [ Jarvis-ComiXed]
o.h.h.internal.ast.QueryTranslatorImpl : HHH000104:
firstResult/maxResults specified with collection fetch; applying in memory!
2020-02-29 14:00:10.844 DEBUG 19118 --- [ Jarvis-ComiXed]
org.comixed.task.runner.Worker : Waiting for task or notification
2020-02-29 14:00:10.902 DEBUG 19118 --- [nio-7171-exec-7]
o.c.service.library.LibraryService : Finding up to 101 comics updated
since Sun Feb 23 18:31:04 EST 2020 for comixedadmin@xxxxxxxxxxxxx
2020-02-29 14:00:10.903 DEBUG 19118 --- [nio-7171-exec-7]
o.c.service.library.LibraryService : Checking comic: timestamp:
1582500664121 >= 1582500664121 id: 101 > 101
2020-02-29 14:00:10.903 DEBUG 19118 --- [nio-7171-exec-7]
o.c.service.library.LibraryService : Returning 0 updated comics
2020-02-29 14:00:10.903 DEBUG 19118 --- [nio-7171-exec-7]
o.c.c.library.LibraryController : Getting processing count
2020-02-29 14:00:10.903 DEBUG 19118 --- [nio-7171-exec-7]
o.c.service.library.LibraryService : Getting processing count
2020-02-29 14:00:10.904 DEBUG 19118 --- [nio-7171-exec-7]
org.comixed.service.task.TaskService : Found 0 instances of ProcessComic
2020-02-29 14:00:10.904 DEBUG 19118 --- [nio-7171-exec-7]
o.c.c.library.LibraryController : Sleeping 1000ms
If so, that's expected behavior. Once a user logs in, the frontend sends
a request to the backend for comic updates. The backend sits on that
request, sleeping for 1000ms and then waking up to check for any comics
that have been updated since it went to sleep. If anything has changed, it
returns with the updated comics. Otherwise, if it times out, it returns
that no comics have been updated.
On Sat, Feb 29, 2020 at 2:53 AM Guy Incognito <
dmarc-noreply@xxxxxxxxxxxxx> wrote:
Apologies, I must have mis-read the log – I’m not seeing any looping.
What I am seeing is that the covers are being returned for the reading
lists, then the log goes back to the “searching for new comics” process.
The wheels keep spinning on the covers, but nothing else really happens
☹
*From: *bareheiny <dmarc-noreply@xxxxxxxxxxxxx>
*Sent: *Saturday, 29 February 2020 9:02 AM
*To: *comixed-dev@xxxxxxxxxxxxx
*Subject: *[comixed-dev] Re: Reading lists
I’ll grab a log when my surface is charged and I can use it again.
For now now, my library (now only 1,200 comics) is fully loaded and
processed.
I can happily navigate the issues, scraping as I go....but the reading
lists appear to be having differently.
I’m assuming I’m correct in thinking the reading lists just pull the
existing comic information and don’t trigger a re-process of each issue on
the list?
On 29/02/2020, at 02:28, Darryl L. Pierce <mcpierce@xxxxxxxxx> wrote:
The spinner means the comic itself hasn't been fully imported yet and is
queued up for processing. The flow for importing comics is:
1. The list of fully qualified filenames is submitted to the backend,
2. a QueueComicsWorkerTask instance is created, given the filenames, and
pushed into the Worker's queue, and
3. Worker runs this task, which creates a record in the tasks database
for each comic being imported.
Worker monitors the tasks table (it's how the backend keeps track of
what needs to be done as far as managing the library) and pops records out
in the order they were created in order to perform tasks. So after the
above steps complete, the tasks table has a bunch of records for comics to
be imported into the database.
1. When an add comic task record is loaded, an instance of
AddComicWorkerTask is created and pushed into the Worker's queue,
2. that task creates a record in the comics table, then
3. inserts a record into the tasks table to process the newly added
comic.
At *this* point, the comic is visible from your browser as a library
entry, but it's not been processed yet. Each Comic object has a child
object, ComicFileDetails (stored in the comic_file_details table), which
holds the details of the physical file itself (currently the MD5 hash for
the file). When the comic is added in the above steps, this child object
isn't created. The system hasn't cracked open the comic file yet, it's only
stuck a record in the comics table. And, when the frontend receives a comic
without it's file details child object, it puts a spinner over the cover to
indicate the comic isn't processed yet.
The finally:
1. When the process comic task record is loaded, an instance of
ProcessComicTask is created an pushed into the Worker's queue, and
2. that task gets the MD5 hash for the comic's file, creates the
ComicFileDetails object for the comic and updates the database.
At this point an update is sent to the frontend with the details object
and the browser removes the spinner.
On Thu, Feb 27, 2020 at 7:53 PM Guy Incognito <
dmarc-noreply@xxxxxxxxxxxxx> wrote:
I used the 0.5 release to add comics to a reading list (I’ve also added
comics via SQL to another list).
Both lists take some time to display the title and description, and even
longer to load the thumbnails.
It also looks like the comics metdata isn’t being returned in a timely
manner – I’m seeing the spinning circles for some time, and when I opened
one comic it only had “unknown” type values against the series and volume
etc.
Made that particular reading list useless as it contained all the comics
that required scraping >_<
*From: *Darryl L. Pierce <mcpierce@xxxxxxxxx>
*Sent: *Friday, 28 February 2020 8:52 AM
*To: *comixed-dev@xxxxxxxxxxxxx
*Subject: *[comixed-dev] Re: Reading lists
I'm not aware of any issues around loading reading lists. The only issue
I know ATM is that the option to add comics to them is gone in the develop
branch while I refactor the context menu system. It should be added back in
sometime in the next week or so.
What are the symptoms you're seeing? They're just not loading at all, or
they load but without any details?
On Wed, Feb 26, 2020 at 6:26 PM Guy Incognito <
dmarc-noreply@xxxxxxxxxxxxx> wrote:
Is there still work to be done on reading lists?
I ask as I created one with 1,200 unscraped comics....it takes ages to
load, and doesn’t display the series and volume information.
I did add the comics to the list via SQL, but I don’t see how that cause
any issues.
--
Darryl L. Pierce <mcpierce@xxxxxxxxx>
"Le centre du monde est partout." - Blaise Pascal
"Let's try and find some point of transcendence and leap together." -
Gord Downie
--
Darryl L. Pierce <mcpierce@xxxxxxxxx>
"Le centre du monde est partout." - Blaise Pascal
"Let's try and find some point of transcendence and leap together." -
Gord Downie
--
Darryl L. Pierce <mcpierce@xxxxxxxxx>
"Le centre du monde est partout." - Blaise Pascal
"Let's try and find some point of transcendence and leap together." -
Gord Downie
--
Darryl L. Pierce <mcpierce@xxxxxxxxx>
"Le centre du monde est partout." - Blaise Pascal
"Let's try and find some point of transcendence and leap together." - Gord
Downie
