[contestms-dev] outsourcing cmsWorkerService

From: Motiejus Jakštys <desired.mta@xxxxxxxxx>
To: contestms-dev@xxxxxxxxxxxxx
Date: Tue, 28 Oct 2014 15:24:12 +0100

Dear CMS users/developers,

I am interested in the following to make the tech setup easier by not
having dedicated machines for workers:

1. Running workers on subset of contestant machines.
2. Running workers on a remotely located "cloud".

1 could work by assigning half o the CPUs for the cmsWorkerService
(cgroup), and the other half to the contestant (if there is only 1
CPU, do not run the worker). I believe I could isolate this well
enough in security perspective so they don't see each other. Since we
control the Linux distribution contestants are running, this sounds
like a really compelling solution. The machines in contests are mostly
homogeneous. Has anyone done this before? Interestingly, there is a
hint on CMS website[1] that this is not unheard of.

2. Amount of traffic between Worker and everything else. While
skimming through EvaluationService, I understand each worker on
startup downloads all contest-related files, but it is quite hard to
see a bigger picture. In particular, I would appreciate answers to the
following:

* What/how much is transferred to/from the worker for every test, test suite?
* Program compilation is done in the worker. Is the compiled
executable left on the worker, or sent back to the EvaluationService?
* Do the test artifacts stay on the Worker for the evaluation, or are
they sent back?
* How can contest organizers make ops life most difficult? I.e. what
is the worst-case scenario of network requirement between Workers and
"the rest"?
* Under which circumstances do the files on the worker have to be
re-downloaded? If worker restarts, does this happen?

Thank you!

--
Regards,
Motiejus Jakštys

[1]: https://cms.readthedocs.org/en/latest/Internals.html#backdoor

[contestms-dev] outsourcing cmsWorkerService

Other related posts: