About the standard architecture, I am going split it into two tasks, one about
the Zoe components and dependencies.
Another one about the standard set of Zoe applications that showcase its
usefulness.
In my mind they are two different tasks, unrelated.
About the Zoe components, I think that we should propose two/three sets. Like
we already did on Task 1.2.1, we need to define at least a “Beginner” Zoe
architecture and an “Advanced” Zoe architecture.
Also for Zoe apps: we can provide several sets targeted to certain classes of
problems/users.
On the questions put forward in the new “microservices platform” task: refer
also to the proposed internal software architecture
(http://docs.zoe-analytics.eu/en/latest/vision.html#architecture).
Periodic jobs or processing pipelines would be handled by the workflow Zoe
module.
Long lived apps (web servers, machine learning servers, etc) can be handled by
Zoe, with the addition of the HA Zoe module, that instructs the backend on
amount of replicas, load balancing, etc.
So I propose to add more tasks for each Zoe module in WP2 (or perhaps in their
own WP?).
Why do this through Zoe and not tell the admins to just speak directly to
kubernetes? Two reasons:
1. We provide a more integrated and general solution, with only one set
of commands (user point of view)
2. By running everything through our scheduler we can be smarter on
deciding priorities, for example we can temporarily reduce the number of
under-used web server replicas to spawn more spark workers for a certain job
(efficiency point of view)
--
Daniele Venzano
Distributed Systems Group – Eurecom
http://distsysgroup.wordpress.com/
