[usf-devel] Re: embedded inline pictures
- From: Luca Della Santina <lucads@xxxxxxxx>
- To: usf-devel@xxxxxxxxxxxxx
- Date: Sun, 14 Aug 2005 15:30:25 +0200
unmei wrote:
For embedding lots of pictures into USF, using the already possible
way with a <embedded> top-level element is not good. When muxing, this
element is seperated from the timed subtitle data and stored somewhere
else. *I assume* the former content of <embedded> is fully read into
memory on startup, therefore seeking on the harddrive is not an issue,
I think is a good assumption to think that USF, as well as any of the
other textual subtitle format, is not going to contain a large amount of
pictures inside itself, as well as no very big picture must be included
in a single subtitle, otherwise it is obvious that playing will be choppy.
Taken this assumption, the idea of having a complete <image> nodes (for
complete i mean that the node contains all the informations needed for
visualizing the picture and not only a link to the real picture data) is
good mainly for streaming:
- if you have the usf file apart from the videoclip, this situation
needs the application who plays it to load the full subtitle into memory
and then apply the right subtitle when a certain video frame occurrs. In
this case having pictures stored inline or in a separate script's
section doesn't bring any pratical difference since you load the whole
script into memory.
- The idea of complete <image> nodes shines when subtitles are streamed
by a splitter, like the case when you encapsulate the subtitle in a
container. Because this approach takes out from the stream's header all
the picture attachments and provide them only when the right subtitle is
streamed.
Pros: less time for starting the media file, less header overhead,
simpler to cut at a certain time for editing applications.
Cons: Since the picture is streamed along with the subtitle, the
processing time for that subtitle increases a lot, and this brings
higher probability of choppy playback.
Prerendered subtitle pictures are used only within "their" subtitle,
only during that subtitle's on-screen time. There is no reason to keep
them globally accessible and accessible over the entire script time.
Except when you want to use the same picture more than one time in the
script (like a logo or somethin else) in this case it is very useful the
old way (picture in <embedded> + link in the <image>).
Proposal
*introduce a new attribute "content" (or suggest a better name)
I think "content-type" name for the attribute fits beter since "content"
seems that the image data are going to be contained in the attribute itself.
This attribute is optional and defaults to the value "reference".
The possible values of this attribute are "reference"|"inline[/type]".
If the value of this attribute is "reference" (either explicit or
implied), the image element contains a reference (file name) like it
used to be.
Maybe the default value has to be "inline" and not "reference", since
the most of times you create a picture event the picture is meant for
that subtitle only, the case where you need to share the same picture
among more subtitles is not so common.
This file is not to be assigned to a globally available and named
image variable. It's scope only extends to this very <image> element
and it cannot be referenced from the "outer world". It is decoded when
this <image> element is first to be used and the decoded data is
unloaded when the <image> element disappears from the screen.
Totally agree with the local scope of the image.
Comment
The proposal is not strictly backward compatible.
But IMO it is about as safe as possible to use.
At this stage retrocompatibility has not to be a primary goal, since
there are not so many usf subtitles around and even less USF rasterizers.
You are encouraged to comment on this proposal or suggest
improvements. I am not fixed to this particular way, but i feel like
having to implement inline pictures in one or the other way sometime
soon.
Visualizng your proposal, a "reference" image event becomes:
<image content-type="reference">MyPicture.bmp</image>
while an "inline" image event becomes:
<image content-type="inline/bmp">MyBase64EncodedStuff</image>
There are some weak points:
1) Having a node content type that changes so much (a string to a b64
encoded string) is not always a good pratice (often in these cases you
use 2 different node types for that like <image> and <inline-image>,
which is obviously not our goal).
2) Maybe is good to place in a separate attribute the MIME type / file
extension, this make "content-type" attribute parsing simpler, and
because otherwise "inline" cannot be the default for <image>).
3) When you use an inlined picture you definitively loose the original
image filename (you only preserve its file extension/MIME type).
In order to solve these issues please consider this modification to your
proposal:
- <image> node text content will become only a base64 encoded string.
- A new attribute for the <image> node is declared: "filename"
in this way a "reference" image event becomes:
<image content-type="reference" filename="MyFileName.bmp"/>
while a "inline" image event becomes:
<image content-type="inline"
filename="MyFileName.bmp">ByBase64EncodedStuff</image>
Regards.
http://usf.corecodec.org
Other related posts:
