Daniele,
I see your point and — putting a user hat on — I think it is very important to
have a coherent and well defined method to deal with logs.
Technically, I find your approach pretty clean, so you have an +1 from me.
I would add that logging is just one side of the coin: we also need a unified
and coherent method to search and display logging information. But I guess this
will be part of another discussion.
Thanks,
P
On 12 Jan 2017, at 16:45, Daniele Venzano <Daniele.Venzano@xxxxxxxxxx> wrote:
While working on a generalized backend interface that would let Zoe abstract
away the details of which backend is used to manage the containers I hit
again the problem of how to manage the logs produced by containers.
We discussed this already several times and at Eurecom we tried various
solutions (kibana, graylog, Zoe custom implementation), without finding a
good one. The problem is exacerbated by having backends with different APIs
that could give or give not access to container standard output and standard
error.
Moreover we have services, like spark-workers, that produce a (mostly
useless) standard output and save a much more useful log in some location
inside the container (that currently is lost forever).
My idea is simple: let’s shift the burden of deciding what is important to
log and save to the ZApp creator.
Zoe will mount a volume inside each container with a well-known path. It is
up to the creator of the ZApp to decide what to save in this volume during
the lifetime of the container. He could just redirect the standard output or
configure the service to save some log files.
This volume is part of a hierarchy that is exposed by Zoe’s web interfaces
(and APIs), so the user can access the log data.
I have added an initial description of this idea in the wiki, for task 5.2.3.
--
Daniele Venzano
Distributed Systems Group – Eurecom
http://distsysgroup.wordpress.com/
