[haiku-depot-web] HPKG/HPKR file format change(s)
- From: Ingo Weinhold <ingo_weinhold@xxxxxx>
- To: haiku-depot-web@xxxxxxxxxxxxx
- Date: Sun, 13 Jul 2014 00:02:27 +0200
Howdy,
just a heads up that the HPKG/HPKR file format has changed slightly,
which may be relevant for the web app's parser.
Formerly the heap compression type B_HPKG_COMPRESSION_NONE was never
used (only B_HPKG_COMPRESSION_ZLIB). I've changed the package kit and
the package tool can now create files with this compression type and
I've changed the file format for this case slightly: The chunk size
table at the end of the heap is redundant (all entries would be 65535)
and it is therefore omitted. [1]
This results in minor size savings, but more importantly now there's no
reason for the parser to read the table from the end of the file before
accessing the heap. This means recompressing an uncompressed package
file can be done from a sequential stream (the package tool still needs
to be adjusted for that), which is relevant for some intended
package/repository changes:
1. Since the compression we use is not very space efficient, it would be
nice to use a better algorithm for packages in the repository. The
package file heap wouldn't use compression, but the whole file would be
compressed e.g. via xz. On Haiku the downloaded package would then be
extracted and recompressed to be suitable for use ("xzcat foo.hpkg.xz |
package recompress - foo.hpkg").
2. To deal with the different compressions over the lifetime of a
pacakge, the checksumming of packages needs to be canonicalized. The
simplest way is to always checksum the uncompressed format ("package
recompress foo.hpkg - | sha256sum" or likely "package checksum foo.hpkg").
3. I'd also like to introduce support for a new compression algorithm,
LZ4 [2], since zlib is rather slow.
As usual, I can't say when I'll get to implementing these.
CU, Ingo
[1] https://dev.haiku-os.org/wiki/PackageManagement/FileFormat#Heap
[2] http://code.google.com/p/lz4/
Other related posts:
