[cldp] Re: libraries wrapping incompatible interfaces

  • From: "Benjamin Rich" <benjamin.rich@xxxxxxxxx>
  • To: cldp@xxxxxxxxxxxxx
  • Date: Tue, 18 Apr 2006 09:51:41 +1000

Karol,

On 4/18/06, Karol Pietrzak <kap4020@xxxxxxx> wrote:
> On Saturday 15 April 2006 02:25, Benjamin Rich wrote:
>
> > I think CUPS is probably the most promising, but if we need to support
> > other spoolers, then certainly I agree we should. I'll leave the way
> > multiple spoolers are supported up to you, though I think we should
> > have default printing functions accessing CUPS.
>
> One thing I thought about is whether uses would ever have multiple spoolers
> installed and running on a machine.  Like, CUPS _and_ LPRng.
>
> Also, do you know, offhand, of recent distribution which doesn't use CUPS?
> This is part of the research I'll have to conduct to make the Printing Layer
> usable.

I don't know offhand, but I think if you went to
http://distrowatch.com/ and just checked out the spoolers used in the
top 15 distros, that would give you what about 95% of linux users
have.

> Apart from keeping it in a file somewhere on my computer, is there a
> Wiki page or something I could put it on?

Haven't setup a wiki yet, but I might, what did you mean?

> > Here is the udev/hal/dbus/gnome-volume-manager stack of functionality
> > as I understand it:
> >
> > udev is a userpspace utility which interfaces with the kernel, and
> > dynamically provides the /dev tree of device nodes, eg. from the
> > bios/kernel to userspace apps. it is a replacement for the old static
> > /dev node tree. it has something to do with sysfs, which is also
> > implemented by the kernel and provides device information in /sys -
> > though I don't know if udev provides the info or the kernel does.
> >
> > hal is the linux 'hardware abstraction layer', which picks up devices
> > from the system, and allows abstracted access to them. it runs as a
> > daemon (and is also presumably a set of libraries for abstracted
> > hardware access)
> >
> > dbus is a messaging system, which allows applications to send messages
> > to other applications. it's like corba or dcom. it's used with hal to
> > inform various applications that such-and-such a device has been
> > loaded.
> >
> > gnome-volume-manager (along with a few other volume managers out
> > there) is a 'policy agent' which sits on top of the udev/hal/dbus
> > stack, and implements user-specified policies on loaded devices - eg.
> > 'when someone inserts a dvd, start playing it with X dvd move player
> > program'.
>
> I see... I don't know what the KDE equivalent of gnome-volume-manager is, but
> the Volume Layer will have to wrap them.  For example, if an ISV specifies
> that RealPlayer can now play DVDs, both GNOME and KDE should know about it.

I'm still against too much wrapping in general. This is a job that I
think should be done by an intermediatory component, like the xdg
tools - ie. gnome, kde, and the cldp use these tools to register
associations or default actions for the volume managers; not, cldp
wraps an ever-growing collection of incompatible desktop systems to
provide a small extra bit of efficiency. Then again we're at the mercy
of the distros and the desktop makers.

Maybe there's a tool which already does this? Or we should suggest
this to the xdg list?

> Any other window managers or desktop environments have something of this kind?
> FVWM?  If our library actually does all this, I can imagine lots of window
> managers using it. :)

Agreed, it would be neat, but we need to find out how these things are
registered in KDE and GNOME first.

> > Since it's stable, GTK, and doesn't have any crazy QT/KDE
> > meta-compiled event-driven thread crap in it, I was thinking it might
> > be a nice codebase to start from in designing, effectively, a stripped
> > down version of that very thing. Since looking into it though, it
> > doesn't seem useful since we don't want to enforce policies on devices
> > (we don't want to mount anything, we just want access to devices
> > through hal/dbus).
> >
> > But gnome-volume-manager contains (or uses, or is hard-coded with,
> > er...) something called pmount which seems to be a core component.
> > Maybe this is worth a look?
>
> Yes, definitely.  I've done little research on the Volume Layer section, so
> I'm keeping all my options open.
>
> Here's some interesting links:
>
> * pmount
> - http://packages.debian.org/unstable/utils/pmount
> * setting up KDE, HAL, etc.
> - http://gentoo-wiki.com/HOWTO_D-BUS,_HAL,_KDE_media:/
> * ivman: generic handler for HAL events
> - http://ivman.sourceforge.net/

Ah yes, that was the competitor to gnome-volume-manager for a while,
it runs in the command line so it serves everyone.

> * gnome-volume-manager
> - http://www.togaware.com/linux/survivor/Using_Gnome_Volume_Manager.shtml
> - http://gentoo-wiki.com/HOWTO_gnome-volume-manager
> -
> http://cvs.gnome.org/viewcvs/gnome-volume-manager/COPYING?rev=1.1.1.1&view=markup
> - the license of gnome-volume-manager is GPL, which is unfortunate because we
> will not be able to include in the library as a fallback if we keep a zlib
> style license.
>
> Initially, I'm hoping to wrap gnome-volume-manager and whatever KDE's
> equivalent is.  That will cover the vast majority of desktop Linux users.
>
> Hmm...

Are you making a separate component which wraps some of
gnome-volume-manager to add extra functionality, or do you want to
wrap g-v-m to provide all functionality? I think we should make our
own simple component.

> > The whole hal/volume-manager/etc. thing is part of Project Utopia, I
> > couldn't find the actual authors (other than that Robert Love appears
> > to written the original). I found this CVS link but I don't know if
> > it's current or they've moved the current sources to somewhere else:
> >
> > http://cvs.gnome.org/viewcvs/gnome-volume-manager/
> >
> > You can get the latest stable source tarball from the gnome site:
> >
> > http://ftp.gnome.org/pub/GNOME/desktop/2.14/2.14.0/sources/
> >
> > Take a look, tell me what you find =)
>
> I must admit, I was not familiar with Project Utopia, and I've read just a
> little bit.  The Linux Journal article was helpful to me:
> http://www.linuxjournal.com/article/7745.
>
> It appears Project Utopia is divided into a platform-agnostic, system-level
> and a GNOME-based user level.
>
> I imagine it would be great if we could somehow take advantage of the
> system-level part somehow.  Licensing issues will likely exist.  A quick
> search did not yield any answers.  Do you know, off chance?

I'd say off the top of my head that it would all be GPL. AFAIK,
project utopia is hal/udev, driver-on-demand (now defunct), and the
love patches for the kernel (which everyone's stopped using because he
pissed off all the kernel and distro people a while ago for being
outspoken).

> > > > The LDPL (which we can codename Sally for convenience)
> > > > would not allow a user, or an app, to search for printers or match
> > > > correct drivers; nor would the LDVL (which we'll call Norwood) ask the
> > > > user which was their default CD drive, etc.
> > >
> > > Sounds good!  We'll also have to avoid trampling on distribution makers
> > > toes: SuSE has its own dialogs for configuring CD/DVD drives, but the
> > > beauty of a library is that we can wrap that.
> >
> > Even so, I don't think we shouldn't wrap too much, it makes our
> > components too reliant on distro-specific things. My original idea for
> > the CLDP was to make it very general and open, so that distributions
> > could do their own integration of it with their systems.
>
> Yes, definitely.  I think the generic case would be the fallback.  For
> instance, SuSE has its own advanced CD-ROM/DVD-ROM drive detection scheme.
> It's probably way better than whatever I could come up with.
>
> The Volume Layer would check whether the system is a SuSE system, and if so,
> use SuSE's mechanism.  Otherwise, is this is a Linux from Scratch machine,
> then we're probably going to have to manually detect devices (or hopefully
> use someone else's, depending on the licensing).

This is a good idea, though again I would recommend a more generic
approach. For example, all of these detection systems no doubt rely on
HAL. If SuSE's doesn't, then we should write in compatibility for it's
proprietary system since it's a special case and SuSE has like 30% of
the linux desktp market; but in general, I think we should try and
rely on the most underlying element. This means we only have to worry
about changes to that component (eg. hal/dbus), not any of the
distro-specific ones. Otherwise, we will be constantly updating for
the shifting sands of the distros, and will once again be at their
mercy.

> > > > I might add these ones to the site soon, you've got me thinking now =)
> > > >  Of course, we will need someone to, um, write them - you're more than
> > > > welcome =P
> > >
> > > Gladly. :)  Like I said, I already have some code.  Here's a few more
> > > details on this library that I've thought up along the way:
> > > 1. I also want this library to gain a large user base in both the
> > > commercial and non-commercial spheres.  So, I would like the license to
> > > be something along the lines of the zlib license, which basically allows
> > > anything.
> > >
> > > I think this library would be staticly linked for convenience, at least
> > > in the beginning.
> >
> > What does the zlib license entail, is it like the berkley license?
>
> I'm not familiar with the Berkley license.  Are you referring to the BSD
> License?  http://www.opensource.org/licenses/bsd-license.php?

Probably that one - it basically says code can be used for any reason,
any time, no stipulations. eg commercially too - NT 4.0 had some BSD
code in it for the TCP/IP stack, and windows probably still does to
this day (it sure as hell has DOS 5.0 code in it =)

> FYI, here's a link to the zlib license:
> http://www.gzip.org/zlib/zlib_license.html

Looks like the BSD license. *Exactly* like the BSD license. Seems
fine, though I think for the sake of getting our stuff accepted into
Linux, and since there are no commercial, closed-source Linux distros,
we'll want to make virtually everything GPL.

> > Statically linked meaning, developers would compile their apps against
> > the source of the printing library? I think a more dyanmic model might
> > work better for making changes to the libraries - we want stable API's
> > with long release cycles, which can be distributed closed and without
> > the need for recompilation. I may be misunderstanding you though?
>
> No, I think we're on the same page.  I must have mis-spoken.
>
> Initially, our library will have limited visibility and little usage: the
> disadvantages of a shared library will outweight any advantages.
>
> In the beginning, I imagine developers and ISVs will just include the source
> in their projects and compile.  It's the simplest, easiest thing to do.
>
> In the future, as the library picks up speed and usage, then I feel the
> advantages of a shared library will outweigh the disadvantages.
>
> Perhaps one day (long shot here :-P),  both GNOME and KDE will use the
> library, in which case a shared library will be perfect.
>
> I completely agree with you in that stable APIs are of the utmost importance.

What are the disadvantages of a shared library?

Acceptance-wise I thought this approach would be good because it means
distros have to have versions of the cldp which complement their
systems, and so customisation gets done for us. BrickLayer, at least,
has to be installed for appfolder packages to work.

But I agree, things probably won't get picked up immediately, so no
doubt our code will be cut+pasted to begin with. If we *can't* zlib
license everything though, and I think not using the GPL might be
hard, then commercial developers won't take on our code unless it is
in a dyanmic library, because this is the only way they can link to it
without opensourcing their code.

> > > 2. Great names, IMO!  I think aesthetics  is very important, even for a
> > > purely developer-oriented library like this.  The names you've given also
> > > illustrate the different components that this library will consist of.
> >
> > Yeah I'm wondering whether to name everything individually, LD*L could
> > become confusing after we add another two or three projects =P
>
> Hmm, it's up to you.  Actually, I realized  that "CLDP" is actually a very
> common acronym:
> - Commercial Law Development Program
> - Chinese Linux Documentation Project
> - Collegiate Leadership Development Program
>
> Are you ok with that, or do you plan on changing the name?

For the moment it seems alright - it was originally ldp, but a friend
realised that was linux documentation project, so i added the c. All
these other examples are outside the field of linux/oss/software,
except for the chinese doc project, which probably won't affect us.
Unless there was something else in Linux with this name, I think it's
alright to keep.

The the LD** scheme could be getting a little ubiquitous though, I was
thinking we should name each component? Like with sally and norwood,
we could have names for the ldml etc. - I was only ever intending 2 or
3 components, so they seemed unique at the time.

> > > 3. Another component will be the Desktop Integration component, or some
> > > such. It will install menu items, register MIME types, and use handlers.
> > > Like you mentioned, the Portland project will do a lot of the work for
> > > us, but only for modern distributions.  The Autopackage folks have
> > > already done a lot of this, but even they have expressed interest in
> > > extracting the hard work they've done in wrapping distributions into an
> > > external library for ISVs to use.  It's not perfect, and hopefully we can
> > > help them out by having a complete library.
> > >
> > > This library will hopefully support older distributions that are still in
> > > heavy use, such as older releases Debian and RedHat Enterprise Linux
> > > releases.
> >
> > All install/uninstall/execution functionality will be part of
> > BrickLayer - check out the 'packaging' page on the site, I've updated
> > the spec a little more re these things.
>
> Ah, I see [http://www.linux-platform.org/bricklayer/index.php].  Actually, I
> think Klik has a working app folder implementation already.  See
> http://klik.atekon.de/.

Having a look now. Seems like something we could sponsor and use,
except for some things I've noticed:
"The klik server calculates dependencies either from entries in the
klik database or by using the debian package database.
...
This means that the server needs to assume a certain set of
applications and libraries to be present in the base system. klik here
assumes the "least common denominator" of multiple distributions to be
present in the base system. "

Great, except you can't assume anything. I think a local LDBK would be
much better here. You grab the app, it has meta-dependency info, you
get the distro to download and install it using it's local package
tree.

Though how they've implemented things inside an image and so forth is
clever - but then how does a user run the app? A wrapper could be put
in /usr/bin which mounts the image and runs the default app I
suppose...

This is just one more great system which I think still isn't
compatible with ours ideas. I'm going over all these though, I will be
updating the bricklayer page soon. Food for thought.

> Also, do you think it's a good idea to create another package in the Linux
> world?  We already have incompatible RPMs, DEBs, klik packages, and
> Autopackages hoping to be distribution neutral.

Yeah but we really only have debs, rpms and ebuilds. Klik and
autopackage have miniature userbases. Not until something can be
integrated into every distro, will one of these formats take over for
application installation - and that's just it, one set of package
standards is for system stuff, one is for desktop apps.

Autopackage we simply can't use, because it installs everything to
multiple paths like rpm/apt. Klik looks promising. Rox and zeroinstall
I'll take a closer look at.

> There's immense technical challenges with app folders, I believe.  For
> example, .desktop files must be installed into a directory in $XDG_DATA_DIRS.
> So, the appfolder runtime will have to install the .desktop file into a
> directory in order for the application to appear in the window manager menus.

Or, bricklayer will have to - which is fine. It can use the xdg tools
to do this.

> > A component for viewing installed apps, deinstalling them, and a
> > daemon for detecting appfolder apps and starting an install process
> > when you click on them or something will be components of bricklayer.
> >
> > As for abstracting a whole install/uninstall suite for ISV's, what
> > other functions do you think we might need to abstract for ISV's
> > concerning app integration? I don't really want to abstract the xdg
> > tools or anything.
>
> Oh, I feel we're going to have to, unfortunately.  Do FVWM support XDG?
> Actually, yes.  Does blackbox?  I don't think so.  What if a certain popular
> distribution simply packages things incorrectly and so _that_ particular
> version of that distribution doesn't fully support the XDG spec?
>
> That happens ALL THE TIME.
>
> e.g., http://plan99.net/~mike/blog/2006/04/03/icon-caches/

Good point, well made, will wrap xdg tools.

> > > > I agree with the commercial side of things, since this will be a way
> > > > ot distribute binary apps only. What sort of thing did you mean,
> > > > though?
> > >
> > > Well, I wanted to see whether we agree on the issue of binary-only,
> > > commercial applications.  I think we see eye-to-eye that commercial,
> > > binary applications should be given first-class status in whatever
> > > specifications, documents, and software we write.
> >
> > I agree - the LDML is based on this. Also, I want to bring in an
> > implementation of the driver-on-demand idea at a later stage when I
> > can start writing the LDML (it was another project utopia idea that
> > unfortunately dried up around the end of 2004).
>
> Question: do you think the LDML will be overwhelmed by the technical problems
> associated with creating a stable API for the Linux kernel?

The stability problems seem to come from the ABI being broken all the
time. The APIs themselves, I don't think actually change that much -
so if we abstract portions of them, that's all we need. Plus the
library can put code between the kernel and the ldml APIs, so we can
retrofit old functions too. In any case, I haven't looked into it
deeply yet, but I don't think it'll be impossible.

> > This platform should be for anyone who wants to create an application
> > and have it work on 'Linux', doesn't matter if they're commercial -
> > the more we can entreat commercial companies to port their stuff from
> > windows to Linux, and under some sort of sane standard instead of
> > getting a 1st year CS student to write it for them in motif, the
> > better =)
>
> Haha, yes, truly.
>
> Well, thanks for the reply!  I'm looking forward to hearing your response.
>
> --
> Karl Pietrzak
> kap4020@xxxxxxx
>
>
>

Other related posts: