[openbeos] Re: Installer Proposal

  • From: Scott Mansfield <thephantom@xxxxxxx>
  • To: openbeos@xxxxxxxxxxxxx
  • Date: Sat, 30 Nov 2002 14:05:33 -0800

Neils,

As the victim of more bad installations than I have digits on my carcass (21, in fact); my hat's off to you -- well done mate! :-D

WARNING: Lots of engineer-speak in this missive.

This might be a little deeper than the scope of your initial proposal, but here's some make-or-break points to ponder I consider tantamount to a successful installation, my purpose being to stimulate discussion in this area.

Fpr openers my definition of "success" in this case does not necessarily equal a fully-installed package. Rather, (dear reader, please pardon whatever overstatements of the obvious follow), my defintion of success includes at the very least:
1. Self-validation. Before installation can commence an install session should, for lack of a better description, validate itself. Simplistically this would include checksums on the contents of the package as well as administrative authorization (beyond R1 when we hopefully go multi-user).
2. In-session validation. During the course of installation the client system is going to be modified in some way. This should, at minimum, include source-to-destination validation of whatever action is to be carried out on the target (such as validating that a file was copied WITHOUT ERROR, ad nauseum).
3. Rollback. Should an installation fail for whatever reason, there must be a way to roll back the target system to its pre-session state thereby preserving the integrity of the target system in the event of catastrophic failure.
4. The post-install state must be checked somehow. I'm thinking that a post-install manifest of some kind of the package contents against what was deployed on the client would be ideal.
5. Installation must be idempotent. Probably implied by points one through four above but nonetheless deserves mention for consideration.


The main point of the above dissertation is that we should be able to guarantee that:
1. If installation is successful, the contents of the package are placed on the client system without corruption, as the author intended.
2. If installation cannot be carried to completion, the integrity of a client system must not, under any circumstance, be compromised. There must be no such thing as a "partial installation" which would subject a client system to breakage. This implies that (again, re-stating the obvious):
a) Any files or data that may be replaced during the installation of a package are returned to their pre-session state.
b) Any files or data that are to be removed during the installation of a package are returned to their un-deleted state.
3. There are enough consistency checks in place that the install session should proceed as the author intended.


For discussion, other things we could consider but may be outside the R1 sandbox:
1. Trojans, scumware, and malaware -- probably REALLY tough to detect (witness the recent TCPDUMP/LIBPCAP trojan).
2. Viruses (virii?) -- though a checksum on the package should catch this.


Now, to context-shift a bit: In short, there ain't no such thing as a fuzzy installation. If what I wanted to install didn't get jacked up during installation, I want to be sure that what I download from dub-dub-dub colon slash slash ID 10 T dot com got installed on my system is what the package author intended, period. Conversely, if the installation got whacked somehow, I don' want my system left in some unknown, half baked jacked up state. Finally, if I don't like your waReZ I want to remove it from my system COMPLETELY and UNEQUIVOCALLY without any chaff left behind whatsoever. As an end-user I don't even want to ponder reinstalling my operating system to fully undo some bizarre proggiez install to dump yer trash appz ::cough:: Windoze ::cough:: ;-)

Cheers,
Scott

On Saturday, Nov 30, 2002, at 11:47 America/Los_Angeles, Niels S. Reedijk wrote:

Hello all,

Attached is the first part of my installer proposal. I am willing to
code it if the idea is
supported by the rest of you lot. Here's a summary of the goals and
features of the
installer:
- Provide a standard interface for customized installers
- Create perfect uninstallers instantly
- Support for dependencies
- Centralized database

It is a replacement for SoftwareValet. This means that any package
compiled for
OBOS will install on all the different distributions derived from it.
Please read the
first two paragraphs, as the third one already goes into somewhat more
detail.
I'd like to get started on the library ASAP, however, I am eager for
your
comments.

Niels S. ReedijkOBOS INSTALLER PROPOSAL
=================
This document describes the Beatrice project, a project that tries to create
the perfect installer software. All rights reserved.
Copyright (c) 2002, Niels S. Reedijk


1 INTRODUCTION
---------------------------
SoftwareValet sucks. When I first started to look through the comments on the
internet about the current BeOS installation system, I was shocked by the tone
of the comments on it. For OpenBeOS we need an open installation standard, if
we're ever going to get a standard distribution. Thus having a public API defining
an installation standard is absolutely required. This document tries to describe
what my ideas and plans for 'Project Beatrice' are.


2 INFRASTRUCTURE
-------------------------------
The initial plans for 'Project Beatrice' consist of two plans. The core part is
the libinstaller.so library, which will contain some classes dealing with installation.
The second part is an application that manages the installation, and that also
is enabled to fetch data from the internet that contains information on updates.
Both parts will be discussed in further parts of this document.
A comment I am expecting to hear is that we will not allow any new API's
for the OBOS R1 release. While it is true that this will break that convention, I
would like to remind everyone that having a new kit like this one is required in
order to properly maintain installation tools. One part where Windows has
graciously failed is in fact the installation part. A common tool throughout
the distributions will make sure that 3rd party apps will never show this
behaviour. Furthermore, it sort of demands a quality standard required for
proper 3rd party apps.
It is not entirely fair to reject this proposal because it enacts a new
kit. Because it really isn't a new kit. The current API isn't changed, so no real
harm is done. Also, the current softwarevalet system can be maintained
using a compatibility layer. So it only adds necessary functionality.
Another argument against this approach is that it is rarely
needed to have custom installers. We'd better write the perfect
installer. The perfect argument against that is when we look at distributors.
They probably have their own means of getting online updates and how
users can install software. However, third party software will probably
have a different installer. Or think about an application that fetches
additional modules from the internet and installs these. In all these cases
there is no separate installer needed, it can just be done via the installation
API.


3 INSTALLATION LIBRARY
----------------------------------------
The core of the 'Beatrice Project' is the installation library. This library contains
all the functionality required to install a package, as well as a frontend to the
current database of installed packages and it also contains some utility
classes needed to perform the actions properly.


Below is a list of classes, and how they relate.
- BInstallerRoster -- frontend to the database of installed applications
- BInstallerScript -- a container and a parser for an installer script
- BInstallerExec -- executes the installation based on an installer script,
and also writes a restore script based on the installation
script.
- BVersion -- a container for versions. You can compare versions with
this class using the normal comparison operators
- BUUID -- a container for uuid's


A summary of the features of the library:
- Advanced dependencies
- Clean uninstallation
- Accepts patches

3.1 INSTALLING
In order to show how I plan the API, I have written an imaginary block of code
which illustrates best how the system works.


BInstallerScript *m_script;
m_script = BInstallerRoster::newInstallerScript();
m_script << "INIT" << "1" << end;
m_script << "DELIVERS" << "8710400282662" << "FooApp" << "1.0.22" << end;
m_script << "FRESH" << end;


status_t error = m_script->Init();
if ( error != B_OK )
  ...

m_script << "SYSTEM" << "1.0.0" << "1.2.0" << end;
m_script << "REQUIRES << "BarApp" << "22610499344664" << "0.9.44" << END;
error = m_script->ResolveDependencies();


if ( error != B_OK )
  ...

m_script << "INSTALL" << "barfile.txt" << "B_LOCATION" << end;

BInstallerExec *exec = m_script->Exec();
exec->Run();

What you will probably notice first is that the BInstallerScript does not use
member methods to input actions. This has two reasons. The first one is that
the installer API will probably change. Over a large period of BC time you
will probably get a problem with millions of methods overcrowding the class.
This way the parser just keeps the installer version in account and parses
the things according to it.
A second thing is that the BInstallerExec object can create a reverse
script based on the input script. This way there doesn't need to be a difficult
parser for the uninstall program, but it can be directly inserted into the
BInstallerScript object.
The BInstallerRoster can be used to query the installed programs
database. It saves the versions and also the dependencies.


3.2 THE SCRIPT SYNTAXIS
Beatrice Installation Scripts are simple and efficient. Scripts are simple lists
of actions that the BInstallerScript and the BInstallerExec does. There isn't
something like flow control statements, or loops, but only a simple list of
commands. This is easy, though perhaps a little less powerful. The idea behind
it is that when installing an application that has sophisticated requirements,
it can use an installation tool that guides the installation, rather than
reading from a complex script.
Thus the script can be inserted using an application, as well as via a
text file that is read by the Beatrice application. The Beatrice application
then feeds the script to the installerobject and, if needed, displays error
messages.
Writing the script is easy, though it would be a nice thing to create
one using a graphical program. Naturally it is possible to create such a
tool, in the idea of PackageBuilder.


The script uses a very simple syntaxis. It is basically of the following type:
COMMAND (required argument) [optional argument]; #comment


An installation has a few stages. The first stage is the initialisation stage. This
stage is to set the paramaters of the installation, and to check if the installation
can be done in general.


All of the following commands are required to be at the top of the file:
- The init command initialises the installer and checks if the installer is capable
of handling the commands in this script. The installer batch is a unique identifier
that refers to a revision of the syntaxis.
INIT (installer batch) ;
- The next command states the UID of the application. An UID is a generated
id that is unique for every application. These UID's can be generated with a
tool.
DELIVERS (package_uid) (packagename) (version);
- One of the three following statements are required. You can safely combine
FRESH and UPGRADE, but PATCH needs to be separate.
FRESH;
UPGRADE [version];
PATCH (version1) [version2] [... etcetera ...];


The second phase is the requirements phase. Only the SYSTEM command is
obligatory.
- With the system command you can specify a floor version of OBOS and the
maximum version supported by your package.
SYSTEM (min_version) [max_version];
- With the next statement you can check if a certain package is installed on the
system. You can also specify the version you'd like.
REQUIRES (packagename) (uid) (min_ver) [max_ver];


From the next phase on, the actual BInstallerExec object takes over. Thus the
next phase actually is just cached until it is executed. This phase is the pre-install
phase, so all the things that need to be done before installing.
- If you need to execute a shell script/command before the actual copying of files,
you can use this command.
PREEXEC (command) [argument1] [argument 2] [...];
- If you are going to overwrite files that you want to restore on an uninstall, use
BACKUP (filename);


The installation phase is just a sum of all the files that need to be copied.
- Install a file
INSTALL (filename) (location);
- Patch a file
PATCHFILE (filename) (patchname) [version1] [version2] [...];


The later phase is the postexec phase
- Execute something after the files are copied
        POSTEXEC (command) [argument1] [...];
- Restore a backed up file
        RESTORE (filename);

You can also include some uninstall commands.
- Remove a file/directory on uninstall
        UNINSTALLREMOVE (filename);
- Execute after uninstalling
        UNINSTALLEXEC (filename) [arguments...] [...];

3.3 PACKAGE DATABASE
The BInstallerRoster is a frontend to a database that maintains an extensive
package list. The package list stores the UID's, installed files, and dependencies.
The file is located somewhere in the system folder. Every installed package is
stored as a BMessage in the installation database.


The following things are stored in the message:
Name: Type: Description
uuid BUUID The uuid of the application
name String The string describing the package
version BVersion The version of the installed package
filesize int The size of the file list
files list<char> A list of installed files (and their location)
dependsize int The size of the depend list
depends list<Buuid> A list of apps that this one depends on<BeOS Attributes>


Other related posts: