[argyllcms] Re: mystery vcgt Re: argyllcms Digest V9 #99
- From: Elle Stone <l.elle.stone@xxxxxxxxx>
- To: argyllcms@xxxxxxxxxxxxx
- Date: Mon, 7 May 2012 10:39:54 -0400
On 5/7/12, Graeme Gill <graeme@xxxxxxxxxxxxx> wrote:
> Elle Stone wrote:
>> Does anyone have a clue what really is loading this "almost linear
>> calibration but with rounding errors" vcgt information upon computer
>> startup?
>
> Hmm. It's not a 16 bit scaling error introduced somewhere in X11 or
> the video driver by using the wrong "convert to 16 bits by shifting by 8
> bits"
> in an attempt at setting a linear VideoLUT is it ?
>
> wrong: val16 = val8 << 8;
>
> If you do this you get:
>
> 0.000000, 0.000000
> 0.003922, 0.003906
> 0.007843, 0.007813
> 0.011765, 0.011719
> 0.015686, 0.015625
> 0.019608, 0.019532
> 0.023529, 0.023438
> 0.027451, 0.027344
> 0.031373, 0.031250
> 0.035294, 0.035157
> .
> .
> 0.976471, 0.972671
> 0.980392, 0.976577
> 0.984314, 0.980484
> 0.988235, 0.984390
> 0.992157, 0.988296
> 0.996078, 0.992203
> 1.000000, 0.996109
>
> If so, this is a coding error, not a stray calibration.
> (Note that 16 bit values are used internally in X11).
>
> The correct way to scale 8 bits to 16 bits is to multiply by 257,
> or shift by 8 bits and add:
>
> val16 = val8 * 257;
> or
> val16 = (val8 << 8 | val8)
>
>
> Graeme Gill.
>
>
Graeme, those values you just gave are exactly the values that I'm
getting after using "dispwin -s" to check the vcgt information after
restarting my computer, allowing for rounding as dispwin only printing
five decimal places for the higher values ( eg 0.98830 instead of
0.988296).
BEGIN_DATA
0.0000 0.0000 0.0000 0.0000
3.9216e-03 3.9063e-03 3.9063e-03 3.9063e-03
7.8431e-03 7.8126e-03 7.8126e-03 7.8126e-03
0.011765 0.011719 0.011719 0.011719
0.015686 0.015625 0.015625 0.015625
0.019608 0.019532 0.019532 0.019532
0.023529 0.023438 0.023438 0.023438
0.027451 0.027344 0.027344 0.027344
0.031373 0.031250 0.031250 0.031250
0.035294 0.035157 0.035157 0.035157
. . .
0.97647 0.97267 0.97267 0.97267
0.98039 0.97658 0.97658 0.97658
0.98431 0.98048 0.98048 0.98048
0.98824 0.98439 0.98439 0.98439
0.99216 0.98830 0.98830 0.98830
0.99608 0.99220 0.99220 0.99220
1.0000 0.99611 0.99611 0.99611
END_DATA
--
Elle Stone
http://ninedegreesbelow.com
Other related posts:
- » [argyllcms] Re: mystery vcgt Re: argyllcms Digest V9 #99 - Elle Stone
