Re: rumpbake support for baked-in rootfs

  • From: Antti Kantee <pooka@xxxxxx>
  • To: rumpkernel-users@xxxxxxxxxxxxx
  • Date: Tue, 16 Jun 2015 10:35:59 +0000

On 16/06/15 09:36, Martin Lucina wrote:

Further, I'd like to enable Andrew to get a useful unikernel working on AWS
*now*, the same for the Mirage folks who want to boot a demo Mirump
unikernel on eg. Virtualbox because they are using Macs for their
workstations.

That can be done by adding the appropriate block device drivers (which I was planning to do last week and yesterday) without introducing new usage.

1) adds a default set of files which libc internally expects

My proposal provides a mechanism to enable adding a default /etc (or
anything else), but defining what goes in there is a theme for a separate
discussion. Having said that, at least mathopd, nginx and MySQL *require* a
minimal default /etc and one non-root user in order to work at all, so I
think adding the "stubetc" as a default /etc is not controversial.

yes

"2" might be nice, though it might be nicer if it actually worked
like every other file system driver instead of just being randomly
hacked somewhere.

As was discussed in [1], NetBSD does not currently provide a suitable
filesystem driver. Hence my use of a Linux initrd inspired approach of
extracting a tarball to /. Of course the current implementation leaves much
to be desired (at the very least, it wastes memory since your data is in
there twice), however it is also minimal and works today.

I have no attachment to this particular *implementation*, in fact I'd be
the first to throw it out and replace with something like SquashFS.

I read between the lines that you don't really even want the tarball method (correct me if I'm interpreting too much). Wouldn't a zero-effort driver implementation have been easier than an adhoc tarball implementation, i.e. use MFS to mount an in-memory FFS image? We'd need to bundle newfs and fs-utils, but we want to do that *anyway* for other purposes. It seems like a double win over adding something you don't really want.

"3" is, AFAICT, useful only when you can't have a block device.
What are such cases? I know Andrew used it for platform bootstrap,
but adding an entire mechanism to support a small phase in platform
bringup is out of the question.

Can't or don't want to. I can think of several scenarios where I don't want
a block device:

1) Cloud providers (AWS, etc.) only provide block storage in gigabytes, and
each cloud provider has their own specific mechanism for provisioning such
storage. I don't expect the former to change any time soon, and support for
the latter in rumprun is going to take some time to materialise, if at all
(worth a separate discussion about the scope of rumprun, not here).

Therefore, if I want to run a minimal and small[*] service as a rumprun
unikernel in the cloud *today*, and that service does not need any
persistent data, why should I bother with an (external) block device at all?

Ok, good reason. Can you point me to a resource giving an overview? I clearly need to read up on the subject.

2) Expanding on Andrew's point about "ease of deployment" from a later
reply in this thread, the ability to have a single self-contained
image containing all code, data and configuration needed to run a service
is extremely appealing to many people. For example, the Mirage folks use a
deployment workflow based on exactly this [3].

Another important usability point for such self-contained kernels is that
they are extremely easy to migrate from "development and testing on local
laptop with KVM" to "deployment on $CLOUD". In fact, the only thing missing
for this to work right now is network device autoconfiguration by default
(separate discussion again).

You too are confusing a particular way of bundling the binary+fs with the concept of it.

I absolutely think that we should be able to distribute a single file image. The problem is that to launch it, you must somehow distribute the rumprun parameters too. So I'm a bit confused as to how including the data but not the configuration in the image gives you what you actually want. That's the whole thing that bugs me. I can't wrap my head around the different configuration spaces, which *all* contribute to what happens at runtime:

1: binary code
2: block device data
3: rumprun launch parameters, including application command line
(4: now-proposed alternative way to supply data)

I get the feeling that we aren't thinking hard enough outside of the context of the toolchain that we already have to obtain the solution that we really want.

3) For my own deployments, I want to use Xen as provided and supported by
the Linux distribution I run on my dom0, and I'd like my unikernels to boot
fast. Currently that rules out using image files for block devices as
booting a unikernel using such a block device takes seconds due to spawning
a new qemu instance (which I don't want either, even if it was fast!). I'm
sure this will get fixed eventually, but I can't wait months until such
fixes trickle down to production deployments of Xen. I can work around this
today by using LVM volumes, however a baked-in filesystem just makes it
much easier.

Doesn't spawning a qemu for networking take time? Or do you want to run a kernel without any I/O capabilities?

"4" leaves much to be desired. Why can you only add files but not
remove them? Why is the backing file system type hardcoded to
"tarball"?

I can make the implementation even more minimal and allow for including
only a single directory tree if it bothers you. The reason I implemented
multiple -R's was to provide a (granted, poor/minimal but working) ability
to accomplish:

rumpbake -R ...path/to/default/root -R data/ ...

ie. Give us the ability to provide a default /etc in the rumprun
repository.

So with a single -R parameter, is the user is responsible for figuring out where to get /etc from?

Is rumpbake really the best tool for handling the complexity? How about a builtin file system tool which supports something like "prepopulate", <user uses normal shell commands here>, "slurp"? (throwing out an unchewed idea, not saying it's a good one)

I'm not sure what you mean by the second point about the backing filesystem
type being hardcoded to "tarball"? That seems like an implementation
detail.

It seemed appropriate that you could tell the tool what kind of file system characteristics to optimize for, e.g. "very small fs", "slightly larger fs". But maybe the tool can internally figure it out.

How do you justify "2" and "3", and how do you plan to fix "4"?

Again, my main justification is responding to user demand and enabling
people to get things running *now* with a minimal amount of implementation
work on our side.

The only reason you gave why you need it *now* was due to lack of proper block device drivers. That will be fixed ASAP, and we don't need to rush this feature in. Immediate satisfaction shouldn't get in the way of the long-term goal. I'll repeat what I wrote in the middle of the mail, because I think it captures the issue:

I get the feeling that we aren't thinking hard enough outside of the context of the toolchain that we already have to obtain the solution that we really want.

Other related posts:

  1. Home
  2. rumpkernel-users
  3. Archive
  4. 06-2015