Any feedback from anyone?
If not, what should I do from here - add it to the wiki, put it somewhere on
github? There’s been a few updates but largely its intact.
as
On 26 Jul 2015, at 6:02 pm, Andrew Stuart <andrew.stuart@xxxxxxxxxxxxxxxxxx>
wrote:
I’m interested to get feedback please on this. Any thoughts and opinions
valued. I’m not precious about it so no need to tiptoe around - rip it apart if
you want - it’s my first shell script larger than 3 lines so any improvement
ideas welcome.
This is a script for getting a rump kernel to work on EC2. Start point was the
MiraegOS script for doing a similar thing but this is substantially modified.
You must modify rumprun/platform/xen/callmain.c before building rumprun (which
is done with ./build-rr.sh xen) and include hardcoded_jsoncfg. Here is an
example:
static char hardcoded_jsoncfg[] = "{\"cmdline\": \"hiawatha_baked.xen -d -c
/data/conf\", \"net\": {\"cloner\": \"true\", \"type\": \"inet\", \"method\":
\"dhcp\", \"if\": \"xenif0\"}, \"blk\": {\"fstype\": \"ext2fs\", \"source\":
\"etfs\", \"path\": \"blk0\", \"mountpoint\": \"/\"}}”;
After rumprun is built, you must then do all the normal steps for building a
rump kernel and bake it with something like:
rumpbake xen_pv bin/hiawatha_baked.xen bin/hiawatha
(Incidentally that syntax seems reversed where output target filename comes
first?)
To run the script below you must
1: launch an EC2 instance
2: copy the script onto that instance
3: ensure you have set up the ec2 cli
http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/set-up-ec2-cli-linux.html
On Ubuntu its pretty easy:
sudo apt-add-repository ppa:awstools-dev/awstools
sudo apt-get update
sudo apt-get install ec2-api-tools
ensure your environment has the AWS keys set up
export AWS_ACCESS_KEY=XXXXXXXXXXX
export AWS_SECRET_KEY=YYYYYYYYYYYYY
In addition to creating an EC2 AMI for the unikernel, this script creates and
attaches an ext2 formatted block device. The script copies files onto that
block device so that your rump kernel is able to access them. You tell the
script which directory to copy onto the block device via the -p /filesystempath
parameter.
The script must be run from an EC2 instance and cannot run outside EC2.
For now, it creates a unikernel in the EC2 availability zone that the instance
is on that executes the script. I’ll change that at some point sp it can built
for other availability zones.
A final important note is that right now the launched unikernel doesn’t work as
there’s an issues currently being resolved I suspect with vbd mapping but
waiting to hear ideas on how to resolve it from this list.
#!/bin/bash
# todo need to give snapshots a name
# todo need to finalise block mapping for AMI when its working
# this script is copied from the Mirage-OS project and modified for rump
kernels.
# IMPORTANT: script must be run from an EC2 instance - it won't work outside EC2
#set -x
SUDO=sudo
UNIKERNELMOUNTPOINT=/mnt/unikernel
FILESYSTEMMOUNTPOINT=/mnt/filesystem
# the commands in this script must be run in the same availability zone and
region that the EC2 instance is running on
THISINSTANCEID=`wget -q -O - http://instance-data/latest/meta-data/instance-id`
THISREGION=`wget -q -O -
http://instance-data/latest/dynamic/instance-identity/document | awk '/region/
{gsub(/[",]/, "", $3); print $3}'`
THISAVAILABILITYZONE=`wget -q -O -
http://instance-data/latest/dynamic/instance-identity/document | awk
'/availabilityZone/ {gsub(/[",]/, "", $3); print $3}'`
# http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/UserProvidedKernels.html
case "${THISREGION}" in
ap-northeast-1)
KERNELID=aki-176bf516
;;
ap-southeast-1)
KERNELID=aki-503e7402
;;
ap-southeast-2)
KERNELID=aki-c362fff9
;;
eu-central-1)
KERNELID=aki-184c7a05
;;
eu-west-1)
KERNELID=aki-52a34525
;;
sa-east-1)
KERNELID=aki-5553f448
;;
us-east-1)
KERNELID=aki-919dcaf8
;;
us-gov-west-1)
KERNELID=aki-1de98d3e
;;
us-west-1)
KERNELID=aki-880531cd
;;
us-west-2)
KERNELID=aki-fc8f11cc
;;
*)
echo $"Error selecting pvgrub kernel for region"
exit 1
esac
## TODO - how to handle when the target region is not the current region? Needs
thinking.
NAME=unikernelfilename
while getopts "h:p:n:r:k:" arg; do
case $arg in
h)
echo "usage: $0 [-h] [-n <name>] [-r <region>] -k <unikernel> -p
<filesystempath> "
echo "usage: $0 [-h] [-n <name>] [-r <region>] -k <unikernel> -p
<filesystempath> "
echo ""
echo "<filesystempath>: Directory path to copy to block file system and
attach to unikernel"
echo "<unikernel>: Name of the kernel file (e.g. rump.xen)"
echo "<name>: the application name to use (default: ${NAME})"
echo "<region>: the EC2 region to register AMI in (default: ${THISREGION})"
echo To run this script you will need the Amazon command line tools installed
from here:
echo
http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/set-up-ec2-cli-linux.html
echo ""
echo Remember to set each of the following environment variables in your
echo environment before running this script:
echo AWS_ACCESS_KEY
echo AWS_SECRET_KEY
exit 1 ;;
p) FILESYSTEMPATH=$OPTARG ;;
n) NAME=$OPTARG ;;
r) THISREGION=$OPTARG ;;
k) UNIKERNELFILE=$OPTARG ;;
esac
done
if [ ! -e "${FILESYSTEMPATH}" ]; then
echo "Must specify a file system path file with the [-p] flag."
echo "Run '$0 -h' for full option list."
exit 1
fi
if [ ! -e "${UNIKERNELFILE}" ]; then
echo "Must specify a unikernel file with the [-k] flag."
echo "Run '$0 -h' for full option list."
exit 1
fi
# Make name unique to avoid registration clashes
NAME=${NAME}-`date +"%d-%b-%Y-%s"`
echo Name : ${NAME}
echo THISREGION: ${THISREGION}
echo THISINSTANCEID: ${THISINSTANCEID}
echo THISAVAILABILITYZONE: ${THISAVAILABILITYZONE}
echo UNIKERNELMOUNTPOINT: ${UNIKERNELMOUNTPOINT}
echo FILESYSTEMMOUNTPOINT: ${FILESYSTEMMOUNTPOINT}
echo UNIKERNELFILE: ${UNIKERNELFILE}
echo FILESYSTEMPATH: ${FILESYSTEMPATH}
##########################################################################################
##########################################################################################
###### ask EC2 what block devices are attached to this instance.
###### unmount and detach the block devices in case they are still attached for
some reason
##########################################################################################
##########################################################################################
BLKDEVICEXVDK=`ec2-describe-instance-attribute ${THISINSTANCEID}
--block-device-mapping --region ${THISREGION} | awk '/xvdk/ {print $3}'`
if [ -e "${BLKDEVICEXVDK}" ]; then
echo BLKDEVICEXVDK: ${BLKDEVICEXVDK}
${SUDO} umount ${UNIKERNELMOUNTPOINT}
echo Waiting for unmount to complete......
sleep 5
ec2-detach-volume ${BLKDEVICEXVDK} --region ${THISREGION}
fi
BLKDEVICEXVDK=`ec2-describe-instance-attribute ${THISINSTANCEID}
--block-device-mapping --region ${THISREGION} | awk '/xvdl/ {print $3}'`
if [ -e "${BLKDEVICEXVDL}" ]; then
echo BLKDEVICEXVDK: ${BLKDEVICEXVDL}
${SUDO} umount ${UNIKERNELMOUNTPOINT}
echo Waiting for unmount to complete......
sleep 5
ec2-detach-volume ${BLKDEVICEXVDL} --region ${THISREGION}
fi
##########################################################################################
##########################################################################################
###### prepare block device
###### this will be the root file system for the unikernel
##########################################################################################
##########################################################################################
# create a 1 GB EBS volume using the AWS console
#
http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-CreateVolume.html
UNIKERNELVOLUMEID=`ec2-create-volume --availability-zone
${THISAVAILABILITYZONE} --region ${THISREGION} -s 1 | awk '{print $2}'`
# wait for EC2 to get its act together
echo Waiting for create volume to complete......
sleep 10
# attach the EBS volume
#
http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-AttachVolume.html
ec2-attach-volume ${UNIKERNELVOLUMEID} --region ${THISREGION} --instance
${THISINSTANCEID} --device /dev/xvdk
echo Waiting for attach volume to complete......
sleep 10
# unmount any existing volume at the UNIKERNELMOUNTPOINT
${SUDO} umount ${UNIKERNELMOUNTPOINT}
# create the UNIKERNELMOUNTPOINT
${SUDO} mkdir -p ${UNIKERNELMOUNTPOINT}
# format the EBS volume as ext2
${SUDO} mkfs -t ext4 /dev/xvdk
#Label the disk. AWS has an unofficial tutorial that does not include this step.
${SUDO} tune2fs -L '/' /dev/xvdk
# mount the device
${SUDO} mount /dev/xvdk ${UNIKERNELMOUNTPOINT}
# set permissions
${SUDO} chmod -R ug+rwx ${UNIKERNELMOUNTPOINT}
${SUDO} mkdir -p ${UNIKERNELMOUNTPOINT}/boot/grub
echo default 0 > menu.lst
echo timeout 1 >> menu.lst
echo title Rump >> menu.lst
echo " root (hd0)" >> menu.lst
echo " kernel /boot/rump-os.gz" >> menu.lst
${SUDO} mv menu.lst ${UNIKERNELMOUNTPOINT}/boot/grub/menu.lst
${SUDO} gzip -c ${UNIKERNELFILE} > ./rump-os.gz
${SUDO} mv ./rump-os.gz ${UNIKERNELMOUNTPOINT}/boot/.
# show what is in the target
${SUDO} find ${UNIKERNELMOUNTPOINT}
# unmount any existing volume at the UNIKERNELMOUNTPOINT
${SUDO} umount ${UNIKERNELMOUNTPOINT}
# detach the EBS volume
#
http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-DetachVolume.html
ec2-detach-volume ${UNIKERNELVOLUMEID} --region ${THISREGION}
##########################################################################################
##########################################################################################
###### prepare block device
###### this block device will contain the unikernel code, and is booted by EC2
##########################################################################################
##########################################################################################
# create a 1 GB EBS volume using the AWS console
#
http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-CreateVolume.html
FILESYSTEMVOLUMEID=`ec2-create-volume --availability-zone
${THISAVAILABILITYZONE} \
--region ${THISREGION} -s 1 | awk '{print
$2}'`
echo Waiting for create volume to complete......
sleep 5
# attach the EBS volume
#
http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-AttachVolume.html
ec2-attach-volume ${FILESYSTEMVOLUMEID} --region ${THISREGION} --instance
${THISINSTANCEID} --device /dev/xvdl
echo Waiting for attach volume to complete......
sleep 5
# unmount any existing volume at the UNIKERNELMOUNTPOINT
${SUDO} umount ${FILESYSTEMMOUNTPOINT}
# create the UNIKERNELMOUNTPOINT
${SUDO} mkdir -p ${FILESYSTEMMOUNTPOINT}
# format the EBS volume as ext2
${SUDO} mke2fs /dev/xvdl
# mount the device
${SUDO} mount /dev/xvdl ${FILESYSTEMMOUNTPOINT}
# set permissions
${SUDO} chmod -R ugo+rwx ${FILESYSTEMMOUNTPOINT}
# copy into the EBS volume the files that will live on the block device
${SUDO} cp -r ${FILESYSTEMPATH} ${FILESYSTEMMOUNTPOINT}
# show what is in the target
${SUDO} find ${FILESYSTEMMOUNTPOINT}
# unmount any existing volume at the mount point
${SUDO} umount ${FILESYSTEMMOUNTPOINT}
# detach the EBS volume
#
http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-DetachVolume.html
ec2-detach-volume ${FILESYSTEMVOLUMEID} --region ${THISREGION}
# make a snapshot of the filesystem block volume
# -- block devices must be attached to AMI’s from snapshots
FILESYSTEMSNAPSHOTID=`ec2-create-snapshot --description 'unikernel filesystem
volume' \
--region ${THISREGION} ${FILESYSTEMVOLUMEID} |
awk '{print $2}'`
##########################################################################################
###### prepare the unikernel for booting on EC2
##########################################################################################
# make a snapshot of the unikernel root block volume
# -- AMI’s cannot be created from volumes, only from snapshots
UNIKERNELSNAPSHOTID=`ec2-create-snapshot --description 'unikernel boot volume' \
--region ${THISREGION} ${UNIKERNELVOLUMEID} |
awk '{print $2}'`
# Create image/AMI from the snapshot
#
http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-CreateImage.html
## HAVING TROUBLE? COULD IT BE [--root-device-name name]
echo Waiting for snapshot to complete......
sleep 20
# -b "/dev/sdf=${FILESYSTEMSNAPSHOTID}" \
AMIID=`ec2-register --name "${NAME}" \
--description "${NAME}" \
-a x86_64 \
-s ${UNIKERNELSNAPSHOTID} \
--region ${THISREGION} \
--kernel ${KERNELID} \
--virtualization-type paravirtual \
| awk '{print $2}'`
##########################################################################################
###### unmount and detach the block devices in case they are still attached for
some reason
##########################################################################################
BLKDEVICEXVDK=`ec2-describe-instance-attribute ${THISINSTANCEID}
--block-device-mapping --region ${THISREGION} | awk '/xvdk/ {print $3}'`
if [ -e "${BLKDEVICEXVDK}" ]; then
echo BLKDEVICEXVDK: ${BLKDEVICEXVDK}
${SUDO} umount ${UNIKERNELMOUNTPOINT}
echo Waiting for unmount to complete......
sleep 5
ec2-detach-volume ${BLKDEVICEXVDK} --region ${THISREGION}
fi
BLKDEVICEXVDK=`ec2-describe-instance-attribute ${THISINSTANCEID}
--block-device-mapping --region ${THISREGION} | awk '/xvdl/ {print $3}'`
if [ -e "${BLKDEVICEXVDL}" ]; then
echo BLKDEVICEXVDK: ${BLKDEVICEXVDL}
${SUDO} umount ${UNIKERNELMOUNTPOINT}
echo Waiting for unmount to complete......
sleep 5
ec2-detach-volume ${BLKDEVICEXVDL} --region ${THISREGION}
fi
##########################################################################################
###### finish
##########################################################################################
echo You can now start this instance via:
echo ec2-run-instances --region ${THISREGION} ${AMIID}
echo ""
echo Don\'t forget to customise this with a security group, as the
echo default one won\'t let any inbound traffic in.
