[argyllcms] Re: A question and some suggestions regarding calibration (dispcal)

  • From: Graeme Gill <graeme@xxxxxxxxxxxxx>
  • To: argyllcms@xxxxxxxxxxxxx
  • Date: Wed, 13 Jan 2016 14:04:35 +1100

Ver Greeneyes wrote:

Hi

I noticed recently that my calibration + 3DLUT was causing clipping on very
bright colors, and I've been fiddling around with calibration since then.
I'm getting much better results now, and wanted to share my findings.

First a question: what is the reasoning behind the call to reset_targ_w()
on line 4536 (just before the verify & refine loop)?

I'm afraid I don't really recall, if the comments don't explain it.

Looking through the other use of reset_targ_w(), my best guess is that
perhaps this should only be called in native white point mode.

Moving on, I've been having difficulty maintaining a good black level
during calibration, and so I decided to enable the black point hack and aim
for natural white. However, I noticed the resulting calibration gave me
clipping near white. I think this is because my display suffers both from
black crush and "white crush", where one or more channels stop getting
brighter well before their maximum value.

Anecdotally, both seem to be rare, since I've copped a bunch of implied
criticism for allowing for this case, and not simply wiring things up
to assumed device 0 = black (i.e. black point hack by default.)

I think removing black crush and this "white crush" is one of the most
important roles of calibration, and I realized that by enabling the black
point hack and shooting for natural white, it actually bends the
calibration curve toward (0,0) and (1,1) and destroys its careful work
determining where black ends and white begins.

Native white shouldn't, but black hack certainly will.

But disabling the black
point hack, the calibration raised my black levels again; and setting
chromaticity target coordinates for white, I still saw some clipping. I
think there are two reasons for this:

1) Test points are weighted perceptually.

However, it also means that near white, far fewer points are measured. This
is fine for most of the range, but it means dispcal doesn't do a very good
job of eliminating white crush.

Is simply setting a white target a little below the native white a
workaround for this problem ?

2) After each pass, a curve is fitted to the measurements to ensure
monotonicity and smoothness. However, this smoothing can significantly
change the values near black and white, where the curve is most non-linear.
This undermines the careful measurements near black and can significantly
raise the black point (or lower it, presumably, reintroducing black crush).

Hmm. The problem is that quantization and measurement noise can result in
non-monotonic and bumpy curves. On some operating systems (MSWindows),
the OS will simply reject a non-monotonic calibration curve, so dispcal
has to ensure it doesn't create one.

Ideally the answer is to add "end clip points" to the monotonic spline model,
and then make sure that these get used to better fit such cases.
I'm certain that there would be issues in getting that fitting right,
both for devices that don't clip, and ones that do, so it would be
non-trivial to implement sucessfully.

I'd like to suggest the following changes to address these problems. I've
implemented them locally, and they've significantly improved my results:

1) Weight points in a sigmoidal way, emphasizing points near black *and*
near white. Actual sigmoidal functions don't naturally go through 0 or 1,
so instead I grafted two power functions together. In pseudo-code:

if (x < ½)
y = 2^(p-1) * x^p
else
y = 1 - 2^(p-1) * (1-x)^p

Here x is the input, y is the output, and p is the power used to weight the
points (such as REFN_DIST_POW). A possible C implementation:

#define DEFINE_APPLY(POW)\
static double APPLY_##POW(const double v) {\
if (v < 0.5)\
return exp2(POW - 1.0) * pow(v, POW);\
return 1.0 - exp2(POW - 1.0) * pow(1.0 - v, POW);\
}
DEFINE_APPLY(MOD_DIST_POW)
DEFINE_APPLY(REFN_DIST_POW)
DEFINE_APPLY(CHECK_DIST_POW)
#undef DEFINE_APPLY

That will affect either the speed or accuracy of the result for other cases
though. Ideally dispcal should be more adaptive, and sample where it is actually
needed.

[ Dumb question - this black and white clipping isn't simply a result of
attempting a full range calibration on a video range display is it ? ]

2) Don't smooth the curve after the final pass (e.g. pass 0.0 instead of
RDAC_SMOOTH), or apply the weighting used for x.nat != 0 and x.bkhack != 0
after the final pass. I opted for the former, as I'm pretty confident in
the measurements and I haven't seen any evidence of non-monotonicity. This
solved my black point woes.

See above. I don't think I can adopt this as a general solution.

With those two changes, I get good calibration results on all my monitors.
In addition, since I'm using the madVR TPG, I added a pass to 'quantize'
the measurements into the 8-bit values that my GPU will actually apply
(since I'm using the calibration for more than just madVR), to avoid
possible rounding issues.


It simply measures the 8 possible ways to round each value (2 ways per
color channel) and chooses the one with the lowest hde. It's a fairly long
process, since it measures 2048 patches, and I don't think it makes much of
a difference if any, but I thought I'd mention it. If this is worth doing,
it probably makes more sense to limit it to values near black and near
white.

In theory dispcal is meant to have explored the possible quantization space
and settled for the minimum DE at each point (although the spline fitting
could alter that). I'm not sure if this type of allowance for quantization
is the best thing to do overall though, since it makes the process
very finicky and difficult, since there is no fool-proof way of knowing
the end to end quantization. Attempting to measure this proves to be hit
or miss, depending a lot on the instrument and display repeatability.

In any case, those were my findings. I realize there are trade-offs to
consider here, so I don't know if these changes can be integrated into
dispcal, but they worked well for me :)

Thanks for reporting you experiments. It's something I'll try and allow for in
future changes to displcal. I'm afraid that I'm not very tempted to apply them
as-is for the reasons above, and in the longer term I'd much rather
try re-writing dispcal to use a quite different approach to the current
code (reverting to one of the very first ideas I tried), to see if I
can speed the whole thing up while not compromising the accuracy.

Unfortunately I don't know when that will be - my work queue is
already stuffed with tasks that are incomplete and behind schedule :-)

Graeme Gill.





Other related posts:

  1. Home
  2. argyllcms
  3. Archive
  4. 01-2016