[contestms] Re: Error when exporting
- From: Stefano Maggiolo <s.maggiolo@xxxxxxxxx>
- To: contestms@xxxxxxxxxxxxx
- Date: Wed, 29 Oct 2014 20:56:44 +0000
Ids in the contest.json are not related to ids in the database (I forgot
about this earlier, sorry). You can find the mapping by adding this other
line after "obj = self.queue.pop(0)" (i.e., as line 200):
print(type(obj).__name__, obj.__dict__.get("id", "?"),
self.ids[obj.sa_identity_key])
(all in one line). If you rerun the export, it will print something like
Submission XXX 162
and XXX is the submission id in the database, so you can check the result
in AWS.
-- Stefano
On Wed Oct 29 2014 at 7:44:39 PM Nir <nir692007@xxxxxxxxx> wrote:
> Thank you for the prompt replies!
>
> Regarding the statement in Italian: yes you're right. I used the Yaml
> importer, and it failed when there was no statement file, so I made my
> script create an empty one (obviously this is not a preferred way, but
> that's what I did back then). I suppose the Yaml importer took it for
> Italian as default.
>
>
> Here's the relevant part of the json:
>
> "3692": {
> "_class": "File",
> "digest": "f2a4[..]",
> "filename": "rectangle_cutting.%l",
> "submission": "162"
> },
>
>
>
> Where rectangle_cutting is one of the tasks in the contest. I then
> searched for 162:
>
> "162": {
> "_class": "Submission",
> "files": {
> "rectangle_cutting.%l": "3692"
> },
> "language": "py",
> "results": [
> "3691"
> ],
> "task": "1",
> "timestamp": 1393716125.757333,
> "token": null,
> "user": "69"
> },
>
> Now, in the admin interface, I see that user 69 exists. However, I did not
> find submission 162 or task 1 anywhere (perhaps I overlooked, is there a
> convenient way to search?). Any ideas?
> Thanks,
> Nir
>
>
>
>
>
> On 10/29/2014 08:53 PM, Stefano Maggiolo wrote:
>
> da39[...] is the hash of the empty file, so it makes sense that the
> f2a4[...] is empty as well.
>
> The occurrence of da39[...] as the statement in Italian is probably a
> innocent mistake somewhere in the setting up of the contest (probably some
> small bug in the importer creating an empty italian statement? Are you
> using the YamlImporter?).
>
> As for f2a4, where do you see it in contest.json? It should be
> relatively easy to find out which file has been lost backtracking the
> references in the json.
>
> On Wed Oct 29 2014 at 6:03:36 PM Nir <nir692007@xxxxxxxxx> wrote:
>
>> Thank you. The export has completed, and I hope you can help with the
>> following information. These are all occurrences of both hashes that exist
>> in the export:
>>
>> - Both f2a4[..] and da39[..] exist under "files". They are both empty.
>> - The file da39[..] exists under "descriptions". It contains only the
>> string "Statement for task argazoids (lang: it)". (Note argazoids is indeed
>> the name of a task in the contest).
>> - The string "f2a4[..]" appears once in the file contest.json once, and
>> "da39[..]" appears 115 times.
>>
>> If there is specific information that may help please let me know.
>>
>> Thanks again,
>> Nir
>>
>>
>>
>>
>>
>> On Wed, Oct 29, 2014 at 11:51 AM, Stefano Maggiolo <s.maggiolo@xxxxxxxxx>
>> wrote:
>>
>>> Hello Nir,
>>>
>>> the quick way is to change line #345 of cmscontrib/ContestExporter.py
>>> from return False to return True. It will still log that the hash is wrong
>>> (corruption somewhere?) but it will continue the export *including* the
>>> file(s) with the wrong hash. I suggest you to later check the content of
>>> the file named f2a4[...] to see what happened.
>>>
>>> Cheers,
>>> Stefano
>>>
>>> On Tue Oct 28 2014 at 7:10:29 PM Nir Lavee <nir692007@xxxxxxxxx> wrote:
>>>
>>>> Hello,
>>>>
>>>> I am trying to export a contest. cmsContestExporter fails with this
>>>> error (I snipped the hashes):
>>>>
>>>> CRITICAL [None] File f2a4[snip]53f has wrong hash da39[snip]709.
>>>>
>>>> How can I resolve the issue and prevent it from recurring?
>>>>
>>>> Note: the contest is rather large and contains a lot of information
>>>> that I must backup. If all else fails, is it possible to remove the faulty
>>>> file or somehow create a backup of all the uncorrupted data?
>>>>
>>>> Thanks,
>>>> Nir
>>>>
>>>>
>>
>
Other related posts:
