2017-01-11 21:31 GMT+01:00 Agustín Santiago Gutiérrez <
elsantodel90@xxxxxxxxx>:
That is really good to know, our default servers are created withFor a while you should have no problems, but at some point the number of
relatively low hard disk space. Is the vast majority of space (apart from
say, problem statements and test cases) used in /var/local/cache/cms, and
simply erasing it periodically (daily?) should bring no problems?
Try using sqlalchemy version 0.9 (this is a know issue)
I have created a contest and loaded it in practiceWebServer, it would seem
to work and the problems added show up in the problem list, unfortunately I
am getting an error when opening it on the judge as a user:
2017-01-11 16:47:20,679 - ERROR [Practice,0] Traceback (most
recent call last):
File "build/bdist.linux-i686/egg/cmsocial/server/pws.py", line
117, in __call__
return self.wsgi_app(environ, start_response)
File "build/bdist.linux-i686/egg/cmsocial/server/pws.py", line
206, in wsgi_app
ans = getattr(self, args['target'] + '_handler')()
File "build/bdist.linux-i686/egg/cmsocial/server/pws.py", line
965, in task_handler
dict([(l, s.digest) for l, s in t.statements.iteritems()])
File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py",
line 237, in __get__
return self.impl.get(instance_state(instance), dict_)
File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py",
line 599, in get
return self.set_committed_value(state, dict_, value)
File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py",
line 1074, in set_committed_value
collection, user_data = self._initialize_collection(state)
File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py",
line 986, in _initialize_collection
self.dispatch.init_collection(state, collection, adapter)
File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/event/attr.py",
line 256, in __call__
fn(*args, **kw)
File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/events.py",
line 1644, in wrap
fn(target, value, *arg)
File "/usr/local/lib/python2.7/dist-packages/cms-1.3.dev0-
py2.7.egg/cms/db/smartmappedcollection.py", line 46, in cb_on_init
coll.on_init()
File "/usr/local/lib/python2.7/dist-packages/cms-1.3.dev0-
py2.7.egg/cms/db/smartmappedcollection.py", line 146, in on_init
assert not self.linked
AssertionError
The user is shown "ERROR 500 Make sure your internet connection is working
and, if this error occurs again, contact an administrator." and nothing
happens. No statement is shown and no submission options are shown. As far
as I can tell, I have filled all the relevant problem and contest data,
just as if it were a normal CMS contest. If things do not progress, I might
try later reinstalling the newest version of CMS instead of the one linked
in cmsocial (not sure if using that specific linked commit is important or
not).
Every contest has a different set of users (but you can enable credential
On other notes:
If I want to actively use the multicontest feature, I would need to have
one cms.conf and one different set of services for each, right? I believe
cmsEvaluationService is the only server that is not able to handle multiple
contests, but that nevertheless requires creating multiple servers. Also,
different contests are supposed to operate as completely different online
judges right? Each with its own users, ranking, etc.
No, it should be as simple as creating a new contest and using the new URL.
Also, tags seem to be added and removed by users. Are there different userYeah, there are different roles (that are chosen by setting
roles for the online judge, so that only "admins" can modify tags? I guess
that this is not controlled from the standard AdminWebServer since I do not
see such an option. I also see no way of creating new tags. Similarly, I
don't know if the "Lessons" and "Quizzes" fit somewhere standard in CMS or
not.
I don't think so.
Also, are teams used somewhere on cmsocial? I believe that they are only
used in cmsRankingWebServer, but I might be wrong about that.
Agustín
2017-01-10 23:10 GMT-03:00 Luca Versari <veluca93@xxxxxxxxx>:
I have been running this system without major issues on a virtual machine
with ~6gb RAM, a single Xeon E5-2630 core, and ~200gb hard drive size.
Actually, the system also runs two nodebb and a discourse installations,
so you probably could be fine with less memory.
I immagine the requirements should be similar to cms, but as a rule of
thumb I would go with ~256mb of memory per worker and 1 to 4 workers per
core (depending on how accurate you want the evaluations to be).
Disk space does get filled up quite quickly, so you should either have a
very large hard drive or routinely clean up the cms cache folder
(/var/local/cache/cms).
2017-01-11 2:53 GMT+01:00 Agustín Santiago Gutiérrez <
elsantodel90@xxxxxxxxx>:
That worked! Thank you very much. What a simple detail.
I'll look more into the system tomorrow but it is very nice to have it
running.
I'll just leave a quick question: Do you have any idea of what are some
reasonable hardware requirements for hosting this system? That is, assuming
for example that the general user load is very low, say like only a few
frequent users.
Agustín
2017-01-10 22:37 GMT-03:00 Luca Versari <veluca93@xxxxxxxxx>:
Ok, quick check: have you tried accessing http://localhost:18888/xyz/
instead of http://localhost:18888/xyz ;?
Anyway, there is no need to run the multicontest update script for
fresh installs.
2017-01-11 2:30 GMT+01:00 Agustín Santiago Gutiérrez <
elsantodel90@xxxxxxxxx>:
Thank you once again!
I retook work on this issue today. I have explored all of this in a
fresh xubuntu 16 vm, and found out that my cms installation was probably
incomplete.
I was used to the installation instructions published for stable
version 1.2, but I just noticed that there are some changes, in
http://cms.readthedocs.io/en/latest/Installation.html. Particularly,
changes in dependencies and the new script "prerequisites.py", so my CMS
installation was probably faulty to begin with. However the results so far
after doing all of this are the same.
When trying to run that prerequisites.py, I was getting:
Traceback (most recent call last):
File "./prerequisites.py", line 439, in <module>
CLI()
File "./prerequisites.py", line 212, in __init__
getattr(self, args.command)()
File "./prerequisites.py", line 335, in install
self.install_conf()
File "./prerequisites.py", line 281, in install_conf
makedir(os.path.join(USR_ROOT, "etc"), root, 0755)
NameError: global name 'root' is not defined
I manually added the following lines to function install_conf to
apparently solve it:
root = pwd.getpwnam("root")
cmsuser = pwd.getpwnam("cmsuser")
I believe that the last master version of CMS has this fixed, but the
specific commit that is downloaded and targeted in cmsocial's
requirements.txt in the develop branch presents this "bug" and so the
script fails. This is the specific commit targeted in cmsocial's
requirements.txt:
https://github.com/algorithm-ninja/cms/blob/1681ee4841004190
a293b290d93525a09acba9f3/prerequisites.py
Right now, I am getting an error at the very last step of running the
multicontest update (I guess it has to be done also for a new database?)
At the end of this email follows the full database initialization with
the final error. Without completing this final "multicontest update" step,
the situation is still the same as it used to be, getting always a very
bare 404 message whose full html is:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>404 Not Found</title>
<h1>Not Found</h1>
<p>The requested URL was not found on the server. If you entered the
URL manually please check your spelling and try again.</p>
This might indicate some misconfiguration of the practiseWebServer,
since the previous oii-web project at least always showed some kind of
homepage screen when up. I am using the default cmsocial.ini file exactly
as is, accessing http://localhost:18888/xyz , with xyz being the name
I gave to a single contest created in the database.
I might try everything again later/tomorrow without downloading any
CMS 1.2 dependency, and using the latest master commit in CMS instead of
the version automatically downloaded by cmsocial's requirements.txt. At
the
very end of the email after the database creation error message, I list
all
the steps that I carried out from a fresh xubuntu 16.04 install, in case
that is helpful. I had to manually add and discover a few ubuntu
dependencies like sudo apt-get install npm, I do not know if there is a
list of such for cmsocial, as there is for CMS in the docs. Maybe
something
like that is missing.
Once again, thank you very much for all your collaboration.
Agustín
cms@cms-VirtualBox:~/repos/cmsocial$ cmsDropDB
2017-01-10 22:01:59,351 - INFO [<unknown>] Using configuration file
/usr/local/etc/cms.conf.
Are you sure you want to DROP the database? [y/N] y
Dropping database.
cms@cms-VirtualBox:~/repos/cmsocial$ cmsInitDB
2017-01-10 22:02:04,733 - INFO [<unknown>] Using configuration file
/usr/local/etc/cms.conf.
cms@cms-VirtualBox:~/repos/cmsocial$ python2 -c "import cmsocial.db;
from cms.db import metadata; metadata.create_all()"
2017-01-10 22:02:09,711 - INFO [<unknown>] Using configuration file
/usr/local/etc/cms.conf.
cms@cms-VirtualBox:~/repos/cmsocial$ psql oiajuez -h 127.0.0.1 -d
database < sql_scripts/create_triggers.sql
Password for user oiajuez:
BEGIN
CREATE FUNCTION
NOTICE: trigger "submission_insert" for relation "submissions" does
not exist, skipping
DROP TRIGGER
CREATE TRIGGER
CREATE FUNCTION
NOTICE: trigger "submission_scored" for relation "submission_results"
does not exist, skipping
DROP TRIGGER
CREATE TRIGGER
CREATE FUNCTION
NOTICE: trigger "user_insert" for relation "users" does not exist,
skipping
DROP TRIGGER
CREATE TRIGGER
CREATE FUNCTION
NOTICE: trigger "task_insert" for relation "tasks" does not exist,
skipping
DROP TRIGGER
CREATE TRIGGER
CREATE FUNCTION
NOTICE: trigger "contest_insert" for relation "contests" does not
exist, skipping
DROP TRIGGER
CREATE TRIGGER
CREATE FUNCTION
NOTICE: trigger "participation_insert" for relation "participations"
does not exist, skipping
DROP TRIGGER
CREATE TRIGGER
COMMIT
cms@cms-VirtualBox:~/repos/cmsocial$ python2 -c "import cmsocial.db;
from cms.db import metadata; metadata.create_all()"
2017-01-10 22:02:32,948 - INFO [<unknown>] Using configuration file
/usr/local/etc/cms.conf.
cms@cms-VirtualBox:~/repos/cmsocial$ psql oiajuez -h 127.0.0.1 -d
database < sql_updaters/multicontest.sql
Password for user oiajuez:
BEGIN
INSERT 0 0
INSERT 0 0
ERROR: column "score" of relation "social_users" does not exist
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ROLLBACK
LOG OF STEPS:
-> Installed xubuntu 16.04 32 bits version
-> sudo apt-get update
-> sudo apt-get dist-upgrade
-> sudo apt-get install git
-> git clone https://github.com/algorithm-ninja/cmsocial.git
-> git checkout develop
-> Download cms dependencies (I started with 1.2 version, in the
end I installed all the dependencies of both after realizing the 1.3
difference)
CMS: 1.2
sudo apt-get install build-essential fpc postgresql
postgresql-client \
gettext python2.7 python-setuptools python-tornado
python-psycopg2 \
python-sqlalchemy python-psutil python-netifaces
python-crypto \
python-tz python-six iso-codes shared-mime-info stl-manual \
python-beautifulsoup python-mechanize python-coverage
python-mock \
cgroup-lite python-requests python-werkzeug python-gevent
patool
CMS 1.3:
sudo apt-get install build-essential fpc postgresql
postgresql-client \
gettext python2.7 iso-codes shared-mime-info stl-manual
cgroup-lite
// I had to manually add these in order for some stuff to
compile without errors. But maybe that was before I realized I should
install 1.3 dependencies...
postgresql-server-dev-all libcups2-dev
-> Create postgres database
sudo su - postgres
createuser oiajuez -P [password oiapass]
createdb -O oiajuez database
psql database -c 'ALTER SCHEMA public OWNER TO oiajuez'
psql database -c 'GRANT SELECT ON pg_largeobject TO oiajuez'
exit
-> sudo apt-get install python-pip
-> sudo pip install --upgrade pip
-> sudo pip install --upgrade -r requirements.txt [Note that CMS
is downloaded into ./src]
-> sudo apt-get install npm
-> sudo apt-get install nodejs-legacy
-> make
-> sudo python2 ./setup.py install
-> cp config/cmsocial.ini.sample config/cmsocial.ini
-> sudo cp config/cmsocial.ini /usr/local/etc/cmsocial.ini
INSTALL THE AUTOMATICALLY DOWNLOADED VERSION OF CMS
-> sudo chown -R cms src
-> cp src/cms/config/cms.conf.sample src/cms/config/cms.conf
-> Edit src/cms/config/cms.conf:
"Add service PracticeWebServer to /usr/local/etc/cms.conf in
core_services section." [Actually replaced ContestWebServer]
Database connection line: postgresql+psycopg2://oiajuez:
oiapass@localhost/database
-> Install combo:
cd src/cms
sudo pip install --upgrade -r requirements.txt
sudo ./setup.py install
-> sudo cp src/cms/config/cms.conf /usr/local/etc/cms.conf
-> sudo usermod -a -G cmsuser cms
-> Log out and login
-> sudo mkdir -p /var/local/log/cms [trying to get rid of some
errors... probably because of not running prerequisites.py]
-> sudo chmod a+rwx /var/local/log/cms/ [trying to get rid of some
errors... probably because of not running prerequisites.py]
-> After all this, I discovered and run (after fixing!) the
prerequisites.py script.
-> cmsInitDB
-> python2 -c "import cmsocial.db; from cms.db import metadata;
metadata.create_all()"
-> psql oiajuez -h 127.0.0.1 -d database <
sql_scripts/create_triggers.sql
-> CAMBIAR el de sql_updaters multicontest a que termine en COMMIT
-> python2 -c "import cmsocial.db; from cms.db import metadata;
metadata.create_all()"
-> psql oiajuez -h 127.0.0.1 -d database <
sql_updaters/multicontest.sql [THIS STEPS FAILS!]
=> Create an admin user: cmsAddAdmin -p pass user
2017-01-08 20:03 GMT-03:00 Luca Versari <veluca93@xxxxxxxxx>:
Ok, I've had some time to look at the commits, and I think you should
use the develop branch, since it has some more features and is actually
more tested :)
The reason why you get a 404 page is that the master branch also has
a multicontest feature enabled (but I'm not sure how well it works), so
in
both cases you should create a contest with some name xyz and then visit
localhost:8888/xyz/.
That version of cmsocial, with slight modifications, is currently
running here http://dijkstra.di.unipi.it/#/overview ;(using nginx to
use subdomains instead of prefixes) so you can have a preview, if you
want.
2017-01-08 20:23 GMT+01:00 Agustín Santiago Gutiérrez <
elsantodel90@xxxxxxxxx>:
Hi Luca,
"make" would seem to be working without errors but everything
remains the same, after carrying out all the steps, PracticeWebServer
just
responds a generic 404 page.
I have now noticed that the https://github.com/algorithm-n
inja/cmsocial/tree/develop/cmsocial subfolder contains a read.md
and suggest that "grunt" should be run to build that. I'm not sure if
it is
supposed to be run apart from the "main project make" sequence or not.
Trying a simple "grunt" command in that folder fails with:
"A valid Gruntfile could not be found. Please see the getting
started guide for
more information on how to configure grunt:
http://gruntjs.com/getting-started
Fatal error: Unable to find Gruntfile."
So it seems that something is missing or it is not supposed to be
used like that.
Currently I am inclined to work with the original oii-web if I can
learn how to load data to the system (for example, to add regions) since
that web server at least started correctly, and also that one seems to
have
way less dependencies (I think that one was just python for the server
and
javascript for the client, while this other one seems to download lots
of
nodejs stuff).
However it might well be that a simple yet key step is missing for
me to successfully start the new project's PracticeWebServer.
Thanks once again for all the help that you've already provided.
Agustín
2017-01-08 4:05 GMT-03:00 Luca Versari <veluca93@xxxxxxxxx>:
Now that I think about it, the instructions in the master branch
forget to mention to run make before running setup.py (as in the
develop
branch)...
On Jan 8, 2017 3:08 AM, "Agustín Santiago Gutiérrez" <
elsantodel90@xxxxxxxxx> wrote:
Thank you very much for your fast respose Luca!
Unfortunately, after trying for a while, this other project seems
harder to get running than the previous one oii-web.
I am using the master branch for now. All the steps that I have
carried out so far are:
pip install --upgrade pip
pip install -r requirements.txt [Both for the
https://github.com/algorithm-ninja/cmsocial project and the cms
project that it downloads from git when installing requirements|
python setup.py build
python setup.py install [This adds the cmsPracticeWebServer
command]
Create a database as explained in http://cms.readthedocs.io/en/v
1.2/Running%20CMS.html:
sudo su - postgres
createuser oia-juez-user -P
createdb -O oia-juez-user oia-juez-db
psql oia-juez-db -c 'ALTER SCHEMA public OWNER TO
"oia-juez-user"'
psql oia-juez-db -c 'GRANT SELECT ON pg_largeobject TO
"oia-juez-user"'
Initialize said database:
cmsInitDB
python2 -c "import cmsocial.db; from cms.db import metadata;
metadata.create_all()"
psql oia-juez-user -h 127.0.0.1 -d oia-juez-db <
create_triggers.sql
Create an admin:
cmsAddAdmin -p password name
So far, it would seem like all of these steps worked.
All cms services seem to start allright when started individually,
but the cmsPracticeWebServer gives a completely generic 404 page,
unlike
when using oii-web, which put the site up on the first try. Maybe it
is not
supposed to just work like that like the previous version did?
Again, thanks a lot for your fast response.
Agustín
2017-01-07 20:38 GMT-03:00 Luca Versari <veluca93@xxxxxxxxx>:
2017-01-08 0:29 GMT+01:00 Agustín Santiago Gutiérrez <
elsantodel90@xxxxxxxxx>:
Hi,I wouldn't say this is the right mailing list, but as far as I
I am not writing about the main CMS system per se, but instead I
am interested in the "oii-web" variant, as used in
https://cms.di.unipi.it. In the event that this is not the
right mailing list, please let me know.
know there isn't a right one - I think the best way would be to
create an
issue on github so far...
Anyway, "oii-web" is the legacy version of that project - you can
find the latest one here https://github.com/algorithm-n
inja/cmsocial. There is some (really minor) documentation in the
readme file. In the branch "develop" you can find the latest
changes, with
some extra documentation.
That's really good :)
We are planning to use such a system for our national olympiad,
since it seems to already implement most of the features we are
interested
in and relies heavily on cms, which is already our preferred contest
manager. We will probably translate to Spanish and possibly make
very minor
tweaks but would otherwise use most of the project just as it is.
Is there by any chance any installation and use documentation asYou should use the new version, if I remember well it won't ask
there is for the main cms at https://cms.readthedocs.io/en/
v1.2/index.html? I have not found any so far, apart from the
paper which describes the system from a fairly general perspective.
I have successfully created an empty database and started
cmsPracticeWebServer (it seems to use the ContestWebServer port
from config
file, since that service is not used in an online judge), which
seems to be
the service that implements this new nice online-judge
functionality.
Operating on this empty database, things seem to work without
errors, but I
am not able to sign up with a new user because the "Region" field
cannot be
left blank, but the drop-down list is empty. I guess region data
must be
entered somehow into the database by an administrator. How is that
done?
for a region anymore.
In the master branch, all tasks from contest with id 1 will be
That leads me to another question. Will contests (and problems)
in the database be used automatically and shown by
cmsPracticeWebServer?
For example if I create new contests with adminWebServer, or if I
import
them with importers.
used. Branch develop supports multiple contests running on a
/contest_name/
prefix.
Should be about right. I don't remember ever using
Also: Which cms services must be up for the online judge to work
properly? I don't really know right now but I'm imagining they will
be:
"Optionals"
LogService [just for logging]
ResourceService [just to start and keep services up, as usual]
Checker [just to ping services for resource service to restart
them]
AdminWebServer [for an admin to operate directly on the database]
TestFileCacher [not sure exactly if optional or very important]
"Essentials"
ScoringService
EvaluationService
Worker
PracticeWebServer
Not necessary at all:
ProxyService
PrintingService
RankingWebServer
TestFileCacher, so I don't know what it should do - it is surely not
really
important :)
Localization should be entirely doable in the cmsocial
Please correct me if this understanding is mistaken.
Also, while it is our last priority right now, any useful
information or tip on localization of this system will eventually
be useful
at a later point.
<https://github.com/algorithm-ninja/cmsocial/tree/develop>/
cmsocial-web
<https://github.com/algorithm-ninja/cmsocial/tree/develop/cmsocial-web>
/scripts
<https://github.com/algorithm-ninja/cmsocial/tree/develop/cmsocial-web/scripts>
/*l10n.js* file.
Thank you very much for providing all these useful tools to the
community.
Agustín
Luca
--
"The time you enjoy wasting is not wasted time"
"Valora a quien te dedica su tiempo: Te está dando algo que nunca
recuperará."
"Cuando solo hay un candidato, solo hay una elección."
