Great post Chris!...and there's nothing there I would disagree with really.
Us humans are weird creatures and I'm positive many of the points you've
stated are all influencing us, even if we're not aware of it! Plus, if you
think about it, all of this is influencing people differently...and that's
what makes us different and gives us what we perceive as choice. Steve.
-----Original Message-----
From: Chris Smart
Sent: Thursday, February 09, 2017 5:05 PM
To: vipaudioaccess@xxxxxxxxxxxxx
Subject: [vipaudioaccess] Re: sonar versus samplitude, and a null test
Ok, here's what I think we should do, if we want to settle this
definitively, amongst ourselves.
First, there are too many variables here, including whichever mp3
decoders are being used! If we're trying to compare audio engines, we
need to eliminate as many other possible differences between the
DAW's as possible, including what happens when both DAW's import the
test audio, store it, rinterpolate and render it on export.
1. Find a multitrack session of some kind, in WAV format. Make sure
everything is at the same bit depth and sampling rate. A high track
count is useful, since some people are claiming the perceived
difference in output increases proportionally to the number of tracks
being summed.
2. Check settings for both DAW's to keep as many things the same as
possible, things like import bit depth and sampling rate, internal
resolution, how temporary files are stored etc. For instance, if the
files are all 44.1K if one DAW is set to import and play back
everything at 48K and the other at 44.1K, one DAW is going to
resample everything on import, where the other is not. We want to
eliminate as many such differences as we can!
3. Import the files into both DAW's.
4. Now, to test the math, adjust each by the same amount, let's say
turning the individual track volumes down by 0.5 dB, just for the
sake of this example. If you turn down each track individually in one
DAW, do the exact same in the other. A small volume adjustment to
tracks results in hundreds, if not thousands, of calculations under
the hood of both DAW's as those tracks are adjusted and recombined.
Adding plugins only adds in more confusion, so don't mess with
panning, EQ, compression, etc. Keep it simple!
5. Render the results from each DAW to a WAV file, keeping bit
depth and sampling rate the same. Not using dither is a good idea,
since we don't know if both DAW's are using the same dithering
algorithms. Also, make sure you're not rendering in real time in one
program but bouncing faster than real time in the other.
Now, the part people don't like. Forget the null test. We're talking
about some people saying they can hear a clear difference between the
output of two DAW's, so let's compare that.
With all due respect to Phil and others who think they can hear a
difference. Unless we're comparing playback results of the same
material from both DAW's, in a long, randomized double-blind trial
consisting of a statistically relevant number of forced choice ABX
comparisons, we are all subject to biases, including the placebo
effect, novelty effect, volume bias, and who knows what else. If you
know, the instant you hit play, which DAW output you are listening
to, your answers simply cannot be trusted, and neither can mine, nor
any supposed expert opinion. For more info on propper testing
procedures, look at:
https://en.wikipedia.org/wiki/ABX_test
Look at the section under the heading "software tests". One of those
should be easy enough to implement, either on your own machine, or on
the Web for group participation.
IMPORTANT!: If the X sample is just the multitrack playing back with
nothing at all done to it, make sure it has been turned down
sufficiently to match the perceived loudness of the A and B test
samples, otherwise volume bias, louder must be better, invalidates the
results!
If you find yourself reluctant to set up and take randomized tests,
ask yourself why that is. What are you worried about discovering?
I think I prefer the output from Samplitude as well. But I could be
thinking that for any number of reasons, such as the following:
* I've grown dissatisfied with operational aspects of Sonar and
CakeTalking over the years, I feel a little sore about Dancing Dots
not supporting newer versions of Sonar, and so, anything else is
going to seem better,, and I really want it to be better.
* Samplitude is newer so hey, tnewer is better, right?
* Samplitude has been around for a long time, so old is better, right?
* I'm in a little niche group of cool people who all chose this tool,
and we all want to be confident we chose wisely, so we want to think
it sounds better than some other thing.
* I read somewhere that Samplitude is very accurate and I trust the
experts, trusted friends who quote those experts, some post in a
forum, Youtube video, magazine review etc.
* I paid good money for Samplitude, instead of going with a popular
cheaper DAW, so I really want it to be amazing.
* I've spent lots of time learning to use this, so I hope it's great.
* Putting myself in Steve's shoes for a second: I've devoted a scary
amount of my life in the past couple of years to making this
accessible, so I really really want it to be great.
There are probably many more reasons that could be added to the list,
all of which add up to "gee, I really want this to sound better than
X," whatever better means to you.
Until someone sets up a valid test and we all take it a sufficient
number of times, it doesn't matter who thinks what sounds superior to
what. It doesn't matter if I, Phil, JP and others all agree
enthusiastically that Samplitude sounds better, and we do. We could
all be fooling ourselves. Several mastering engineers whom I respect
greatly, swear by Sequoia, which apparently uses the same audio
engine as Samplitude. That influenced my purchasing decision, but
really, unless they've done sufficient testing between various
competing products, those opinions are worth no more than the opinion
of your untrained nextdoor neighbor.
I look forward to taking the ABX test a whole bunch of times and
seeing if the results validate or invalidate my opinion.
Chris
--
- To post to the mailing list, simply send email to
vipaudioaccess@xxxxxxxxxxxxx.
- To subscribe to the list, send an email to
vipaudioaccess-request@xxxxxxxxxxxxx
with 'subscribe' in the Subject field.
- To unsubscribe from the list, send an email to
vipaudioaccess-request@xxxxxxxxxxxxx
with 'unsubscribe' in the Subject field.
- For more list options, like setting digest and vacation modes, visit
//www.freelists.org/list/vipaudioaccess
- Online, searchable archives of the list are available at
//www.freelists.org/archive/vipaudioaccess
--
- To post to the mailing list, simply send email to vipaudioaccess@xxxxxxxxxxxxx.
- To subscribe to the list, send an email to
vipaudioaccess-request@xxxxxxxxxxxxx
with 'subscribe' in the Subject field.
- To unsubscribe from the list, send an email to
vipaudioaccess-request@xxxxxxxxxxxxx
with 'unsubscribe' in the Subject field.
- For more list options, like setting digest and vacation modes, visit
//www.freelists.org/list/vipaudioaccess
- Online, searchable archives of the list are available at //www.freelists.org/archive/vipaudioaccess
