[usf-devel] Re: embedded inline pictures
- From: unmei <unmei@xxxxxxxxxxxx>
- To: usf-devel@xxxxxxxxxxxxx
- Date: Sun, 14 Aug 2005 19:31:41 +0200
Luca Della Santina wrote:
I think is a good assumption to think that USF, as well as any of the
other textual subtitle format, is not going to contain a large amount of
pictures inside itself, as well as no very big picture must be included
in a single subtitle, otherwise it is obvious that playing will be choppy.
Cons: Since the picture is streamed along with the subtitle, the
processing time for that subtitle increases a lot, and this brings
higher probability of choppy playback.
This can only be seen by trying it. You are right that passing the
pretty long string of such base64 encoded data can slow down some parts,
but i hope it will be offset by the radically reduced composing
complexity for the final bitmap. Yesterday i tried a sample where i
muxed 4 inline pictures (1xPNG,1xJPEG,1xGIF,1xOPS) and testApp did not
seem to lag visibly even tho their size was ranging from 30 to 130KB.
It is currently a major hassle to produce files with embedded images,
but once i have a file with some more pictures in it, im goint to do
timing tests. But of course this will only time pixifier, the additional
lag produced in the splitter and pixishow probably has to be taken into
consideration as well.
Except when you want to use the same picture more than one time in the
script (like a logo or somethin else) in this case it is very useful the
old way (picture in <embedded> + link in the <image>).
Yes, inline pictures are clearly not for logos and other repeated
pictures. Also i expect files with inline pictures not being produced
manually, but by an application that takes a text based script and
renders it into pictures and inlines those.
You can of course create inline pictures manually, for example if you
have a few things that are not possible with USF styling and want them
included in an otherwise text based script.
I think "content-type" name for the attribute fits beter since "content"
seems that the image data are going to be contained in the attribute
itself.
Totally agreed. I will use content-type.
Maybe the default value has to be "inline" and not "reference", since
the most of times you create a picture event the picture is meant for
that subtitle only, the case where you need to share the same picture
among more subtitles is not so common.
I'm unsure about this. From a logical point you are right. But it
invalidates old scripts. Especially bad is that USFV100_sample.usf will
become even more invalid than it already is :)
Maybe we should really use a new USF version number at some time...
At this stage retrocompatibility has not to be a primary goal, since
there are not so many usf subtitles around and even less USF rasterizers.
It's a point. But XML would aready make it so easy to design extensible
and still backward compatible file formats we should not waste it
without always thinking about how we can reduce the impact on
compatibility. Of course in our case, if the compatibility becomes a
major drag it is better to break it and use a clean solution.
Visualizng your proposal, a "reference" image event becomes:
<image content-type="reference">MyPicture.bmp</image>
while an "inline" image event becomes:
<image content-type="inline/bmp">MyBase64EncodedStuff</image>
Correct (except see below)
There are some weak points:
1) Having a node content type that changes so much (a string to a b64
encoded string) is not always a good pratice (often in these cases you
use 2 different node types for that like <image> and <inline-image>,
which is obviously not our goal).
Again a good point. Still i opt for the one element for both content
types. If people there are strong arguments against the "combined" type
from the user side, we could still switch to a seperate <inline-image>.
This is a proposal after all, and not like changing the base
specification draft 0.16.
Now that you mention this, <image> with a reference is actually a bit
unlucky, especially with the reference pointer in the pareable content
(compared to an attribute). You could actually say, a reference is a
reference no matter whether it points to a raster image or a SVG
declaration or whatever else. And the use <image> for inline images only
and <reference type="image" xref="this.bmp" /> to refer to an embedded
or external image.
But this is kind of out of question IMO. A merge idea how it could also
have to be done.
Please do not consider this as a proposal i back :)
2) Maybe is good to place in a separate attribute the MIME type / file
extension, this make "content-type" attribute parsing simpler, and
because otherwise "inline" cannot be the default for <image>).
Agreed.
3) When you use an inlined picture you definitively loose the original
image filename (you only preserve its file extension/MIME type).
I have zero problem with losing the filename. I suspect you think about
extracting the image again. But IMO embedded files are not meant to be
extracted again. Of course you can (text editors and base64 decoders are
cheap today ;)) but the name of an inlined image probably has no meaning
anyway. If the application that embeds inline images specifies filenames
such as "image_0001.png", "image_0002.png" you can as well not mention
them and assign them a random filename at the time you extract them.
In order to solve these issues please consider this modification to your
proposal:
- <image> node text content will become only a base64 encoded string.
- A new attribute for the <image> node is declared: "filename"
in this way a "reference" image event becomes:
<image content-type="reference" filename="MyFileName.bmp"/>
while a "inline" image event becomes:
<image content-type="inline"
filename="MyFileName.bmp">ByBase64EncodedStuff</image>
MMM i should have read all of the mail before writing the other comment
above =>
If you too think the reference does not belong into the content, it is
probably really not that great.
I am a bit in doubt, but i think we should drop the possibility to
specify the MIME-Type. We can give the extension in filename and this
serves as a early-abort for unsupported types as well as a MIME-Type.
Now if you are going to change that much, i also tend to favor "inline"
as default content type. Parially also because i came up with a error
handling that should be "solid" for old images.
Also again, if the filename is no longer "content", the name
"content-type" doesn't fit the meaning too well form the data structure
point of view.
But "type" again is too general, and "content" has not become more
fitting. So ok, let's stick with content-type, from the "user" point of
view the image istill is sort of content of that element and there
probably is no totally fitting name for that attribute.
*Revised* Proposal
the <image> element should be changed in the following way:
*introduce a new attribute "content-type"
This attribute is optional and defaults to the value "inline".
The possible values of this attribute are "reference"|"inline".
*introduce a new attribute "filename"
This attribute is mandatory.
*if the element is a reference the element must be empty. See below for
a note on this. If the element is an inline image the content of the
element is the base64 encoded file data.
Notes:
For reference elements, the value of the filename attribute is the name
of the the ressource to be liked to. The ressource is either an embedded
image with corresonding filename attribute or an external file with that
filename.
For inline image elements, the value of the filename is merely
informational. It is required to at least provide the correct extension
of the inlined image, but it doesn not have to be a filename windows
likes. And it does not have to be unique. It is perfectly legal to
specify filename=".png" for all inlined images (if they are in fact PNGs
, that is).
USF files using the old way for images are in error with this new
specification. But the following error handling in this case is
mandatory for all parsers compliant with this proposal:
If a <image> element does not have the attribute "filename" it is in
error. If such an element is encountered, the parser must assume the
<image> element follows the old standard and interpret the _content_ of
the element as a _reference_. However the parser should notify the user
of this, unless this inflicts with the uability of the application.
In other words, an editing application must inform the user of this
error, but a playback application does not have to, if the notice would
interrupt playback. If a playback application provides an informational
output such as a "properties" or "info" panel or window, the error must
be mentioned there, but it must not pop up an error dialog blocking the
playback. A compliant software must not write the old type of image
elements. Even if it is merely copying data, it must transform such
elements into the new form.
(This will help getting rid of the old type since you only have to load
an old script in a compliant software and immediately save it to
transform into a compliant script).
If an image element "looks" like the new type (it does have the filename
attribute), a parser must not interpret the content as reference, even
if it is not valid base64 data. If the content in this case is not valid
base64 data, the handling must be the same as if a reference were not
found. If the decoded data is of a type the application does not
understand, it may apply error handling for "bad file type". The
handling for "bad file type" may be the same as for "file not found" or
it may later be specified more precisely.
The comments of the old proposal about scope and internal handling still
apply.
-- end of proposal --
PS: pixifier now can detect file types for PNG, BMP, JPEG, GIF and OPS
from the data (without filename/extension), but i still think it's good
to always provide at least the extension. It can help other appications
, it allows for early out if the type is not supported and making the
filename attribute mandatory is a good way to make a distinction between
the new and the old type of image elements.
http://usf.corecodec.org
Other related posts:
