The difference between SACD and CD can not be heard

P

Petrus

Well-known member
From AES Journal 2007 September, Volume 55 Number 9:

"Audibility of a CD-Standard A/D/A Loop Inserted into High-Resolution Audio Playback

E. Brad Meyer and David R. Moran 775

Conventional wisdom asserts that the wider bandwidth and dynamic range of SACD and DVD-A make them of audibly higher quality than the CD format. A carefully controlled double-blind test with many experienced listeners showed no ability to hear any differences between formats. High-resolution audio discs were still judged to be of superior quality because sound engineers have more freedom to make them that way. There is no evidence that perceived quality has anything to do with additional resolution or bandwidth."

Test were done with a high grade stereo system and over one hundred test persons. Nobody could hear the difference between SACD and CD standard audio.
 
Well it gets a little more complicated if you look at more than the synopsis. For one thing they were talking only about the delivery format not acquisition and delivery is pretty much a given to be at 44.1 or 48/16.
It cracks me up. It seems like the same folks who went on and on for years claiming you could hear the difference between 44.1 and 48, are frothing at the concept that you can hear the difference between 16 and 24 bit.

It takes the human eye a 1/4 of a second to resolve an image, but we don't shoot at 4 frames a second. And film shot and played back at 60 frames a second looks almost three dimensional compared to 24 fps. But of course none of this can be because the eye can't see that fast. Try watching 4FPS footage and see if you can notice the difference.

It is an interesting study. But again what was the right question and did they ask it or did they have the answer and look for a test to prove it? I don't know the answer but it takes more details than I could find to know.
 
I did see the list of hi-rez recordings they used somewhere, but for an engineering type of study it can be assumed they used only 24/96 or better native recordings as source, otherwise there would not be much point in it, would there?

The SACD side of the system was "clean", only a relay switch, the CD side had a professional CD recorder with 16/44.1 A/D/A converter and a levels setting circuit. This quaranteed that all presumably quality degrading operations were done on the CD side. Still the resolution drop was never heard in normal listening.

The only way to hear the difference was to listen to the noise floor at insanely high levels between the tracks. But that is irrelevant.
 
It was consumer testing. It was played back on a "high end stereo system". The whole point was whether average people could tell the difference between bit stream and CD, they didn't encode CD's they just ran the signal through the converters. As the title says the study was of a" CD standard A/D/A inserted" into the playback. Assuming that everything was done in a very controlled and unbiased way it still has to do with playback over consumer systems. It has very little to do with the advisability of recording at higher bit depths or sample rates. As I have said rather endlessly I and everyone that I know that has A/B'ed 16 VS 24 has heard the difference. But even if the hadn't it still makes sense to record at 24 bit. You can record with more headroom and you have a better format for post work. I don't know what there setup was and I don't assume anything about what they used. I was involved too long in science and engineering to take anything too seriously with out knowing how they got what they got. And I never buy into statements, theory's, studies etc that go against my personal experience unless they can also explain the discrepancy.
 
Of course 24 bit recording has tangible benefits and this delivery media listening test did not question that. Only that for the final output 16/44.1 is good enough.

If you do not know the test setup, why do you say "whether average people could tell the difference between bit stream and CD"? Did you not notice the phrasing:"many experienced listeners showed no ability to hear any differences between formats".

Here is the link about the test setup: http://www.bostonaudiosociety.org/explanation.htm

System used was not über-hifi, but certainly much better than normal home stereo system.
 
It is quite easy to make a test file and try this at home:

- get a true 24/96 recording (I bough one from Linn web shop, FLAC "Studio master" of Prokofiev's 1. symphony)
- convert to AIFF or WAV for processing, keep resolution at 24/96
- Join the traks into one file (4 tracks in this case, total of 14:20)
- cut the file into 30 second clips (28 clips in this case)
- copy clips, convert to 16/44.1 and back to 24/96
- throw dice to deside which resolution clips are used
- join the clips back to form the original perfirmance, some random clips those that were downrezzed 16/44.1, others 24/96 as decided by dice
- listen to the file with a good player capable of true 24/96 performance (I use SD722)
- make you own conclusions.

You can use this with friends by either asking at every 15+n30 seconds it the audio is hi or low resolution, or every n30 seconds if there was a change compared to previous clip.

This method bypasses the need for comparator, level setting etc. If you are really interested in finding out if you can hear the difference this is the way to test it.
 
Here is the one and only comment I'll add to this thread:

Many scientifically minded types will offer the results of such A/B (and often A/B/C tests where one choice is secretly offered twice) as proof that there's no audible difference in one thing or another, such as 24 vs 16 bit, 44.1 vs 96k, x amount of tenths of a db of level difference and so on. To them, the "science" is inarguable. I say, if they want to take it as such, then they should go ahead and knock themselves out.

It is interesting nonetheless. I myself have actually "failed" some of these tests, in spite of "passing" them repeatedly when they're somewhat less formally set up, and in my regular day-in day-out work as a recording engineer [*see note at end of post]. There is no question in my mind that 24 bit sounds different and better than 16 bit, and that 96KHz rates are slightly better sounding on less than exotic converters than 44.1.

But the scientists drone on about their "proof", and given that I have experienced some of the same "comparison confusion" in formal testing that they offer up as such, it's led me to ponder what the deal is here. As I said before, I know I can hear the difference, have done so innumerable times, and will not ever accept the notion that it is in my imagination. Please take that as a stipulation, at least on my side of the discussion, because it is, and will remain, non-negotiable. Thanks.

So...what gives then...why do even decades-trained golden-eared types like me get tripped up sometimes on these tests when the rest of the time it's pretty much a no-brainer? I have a theory, and I suppose that's all it is, but anyway, here goes:

The problem is inherent in the nature of the tests due to their necessarily temporally-shifted presentation (listen to "A", *then* B" and/or "C"), and the utterly fluid, innate temporal instability of the material itself (music). For a listener to decide which of A/B or A/B/C is the same or different, they must mentally "catalog" a series of brief impressions of the precise nature of "A" in *every single detail* for a given amount of time before listening to "B", doing the same, and then mentally weighing their *memories* of the two extremely complex data sets against each other. So the listener not only must memorize all details of "A" at any given moment in time, but also an entire ever-shifting *stream* of "A". Then compare that to a similarly ever-shifting stream of "B". Pretty challenging stuff...

As I shamelessly bragged earlier, and here I go again, I have some seriously trained hearing, and can do the above pretty well. Even so, I find myself trying to store an overwhelming number of data points - the reverb had that sense of depth and breadth at one moment, the cymbal had this amount of top end at another moment, the vocal's midrange had that degree of warmth at yet another - and so on, and on and on. After a few turns around the A/B/C listening merry-go-round I...sometimes...fall off.

If I may be forgiven for my now-uncontrollably escalating arrogance, your typical "experienced listener" can't shine my shoes in this classroom. Why? Retention of the critically relevant information is virtually impossible without serious training and experience given the test circumstances: time offset and the inherently elusive nature of the material. It's hard even for highly trained experts like...you know...

The problem is similar to a conventional memory test: verbally give a test subject a list of words and numbers to repeat, and the average person will fail almost every time on a list that only takes about 10 seconds or so to recite.

So, "scientific music listening comparisons" with this type of critically limiting factor are, in my humble opinion, only useful in a very rough, general way, and thus are not really "scientific" at all, in the strictest sense. Bear in mind that it would be a very simple matter to assemble a group of normally-abled listeners who could provide statistical "proof" that an MP3 compressed version of a music track sounded "the same" as the uncompressed original. Ask a few teenagers about it.

The problem is that since there's no really good way to do this test, thus making the above the "most scientific" one we can manage, it's given the same kind of credibility that other, much more inherenently valid tests do in fact deserve.

Visual ones, for example. If we set aside deliberate optical illusion "trick questions" such as comparing same-sized circles either inside a larger, contrasting colored circle or against a white background (they will appear different), a visual comparison is a good counterpoint:

A test subject is shown two drawings of a tree side by side and asked to report on any differences. In case one, the drawings are on the same page. The test subject can easily spot differences in height, color, contrast, detail and so on by looking at both drawings simultaneously. Even if two separate drawings are shown with a short time lapse between the viewings, we're still looking at a static image. The viewer can focus *indefinitely* on any part of the images to compare them (unless a temporal limitation is deliberately introduced). This will produce results that are hard to dispute. Not so with strictly temporally based material.

Remember the infamous "New Coke" taste test that "proved" it tasted better than "Old Coke"? It's not entirely equivalent to sound, but it was similarly scientifically flawed, with similarly dubious results.

One last point - here's a revealing little test to try on a few different people: play them a fairly simple recording involving, for the sake of discussion, piano, cello, flute and trumpet, and then ask "what instruments are in this recording"? I have personally heard people answer "I can never identify any of the instruments in a recording". Critical listening's not so easy for some of us...

Bottom line:

Music is fluid, invisible, and inherently unstable. Conventional scientific testing methods are not adequate where establishing supposed proof of the difference or lack thereof between such subtle distinctions as 16 and 24 bit or 44.1 and 96 KHz are concerned.

End of rant. Sorry for the long one!

[*Note: actually, I should have mentioned that the only formal listening test I "failed" was a .1db level difference A/B/C test. I've also passed it repeatedly under different circumstances. Conventional scientific wisdom has it that the smallest audible difference is .2 or .3db]
 
Last edited:
Petras, the synopsis mentions that the test was conducted with "experienced listeners". They did not say audio professionals, and they would have if that's what they had used - it is the AES newsletter after all. My neice is an "experienced listener". It was not a "scientific" study it was an engineering study, basically to prove that the industry can stay with CD's because "experienced listeners" can't tell the difference. In typical modern journalistic simplification you get a dramatic (though not really accurate) headline of "The difference between SACD and CD can not be heard". This by the way is also a dead giveaway to the nature of the "facts" in the article. No scientist would make a statement like that because it is by it's very nature false. Blanket statements like that come from folks writing headlines to get your attention.

And it's a moot point for anyone in the film biz because you don't have the option to deliver to your customers at high sample rates or larger bit depths.
 
Ted, very on point. We live in an age where statistical studies are passed off as scientific fact and a poll of a hundred people is used to tell you who the population of the US as a whole will elect. They always put those disclaimers in about "margin of error" at like 5%. Well that must be the margin of error for misconstruing what those 100 people said because in a correct statistical sense the margin of error with that small a sample size would be more like 1,000,000%, but then who would listen?
Judging by the kind of tracks I get from many picture editors those "experienced listeners" would have a hard time telling the difference between 8 and 16 bit recordings, let alone 16 VS 24.
The most significant thing coming from this is probably that the industry wont be moving away from CD's anytime soon. On the other hand if they move to bit-stream it will add a bigger hurdle to copying music... Part of what I think atracts the studios to BlueRay.
 
Interesting new info: SACD does not in reality have any musical information above 22kHz. There is only a strange lump of high frequency shaped quantisation noise with the peak at 40 kHz at -80 dBFS level!!!! It does not carry any musical information, actually SACD resolution falls below CD resolution at above 10 kHz! SACD is NOT a high resolution format in that sense, even though it is marketed as superior HF medium...

http://sound.westhost.com/cd-sacd-dvda.htm

I can see the same thing on frequency analysis of 24/96 "Studio Master" files from Linn Audio. I''l try to post some pictures later.

Did the AES folks actually choose a wrong format for their comparason, a clean 24/96 PCM file would have been better with true HF content (depending on mics and music)? Now the only difference was dynamic range and a LOSS of random HF noise in CD -chain. Actually the CD -chain could sound BETTER under these circumstances.

Looks like the SACD is s**t.

Jury still out on real 16/44 versus 24/96 case...
------------

I made a 14:20 long test file from one of these 24/96 studio masters; 30 second clips randomly downgraded to 16/44 and put back together as a 24/96 file. Tried to hear where it was 24/96, where 16/44. No clues from stereo perspective, noise, HF content (see above), heard no difference. If it were not illegal and inconvenient to publish this 450MB file, I would give it away to all interested. But, like I said, those interested can roll their own.
 
Last edited:
I think to do a "fair" test of the bitstream recording VS CD whatever you would probably have to record one with one of those Korg recorders. Interesting about SACD, I don't own any.
 
Just a quote from the study:

"The subjects included professional recording engineers, students in a university recording program, and dedicated audiophiles. "

So the subjects weren't random Joe Blow off the street, but rather people who should have critical listening skills. Well, perhaps not the audiophiles, since they can hear the difference between the types of wood feet you put under your amplifier :)

Personally I think that 44.1/16 is probably good enough for a delivery format. The world (right or wrong) seems to agree, as nobody buys SACD or other high resolution media. It's not like anyone would use the newly found dynamic range anyway!


Petrus,

If you are performing multiple SRC's ( 96 -> 44.1 -> 96) it could impart a noticeable change in the sound of the files. Not all SRC are created equal. Just a thought.
 
Here are some pictures from Audition frequency analyser. First three are from the Linn Studio Master 24/96 file, notice how the frequency content above 22kHz stays identical no matter what the actual audio content is. Three samples from loud, not so loud and postroll quiet parts of the finale of Prokfiev's 1. symphony:

1203958308.jpg


1203958216.jpg


1203958345.jpg


Finally a 24->16->24 version of the last picture, no HF noise lump for obvious reasons, but there is a dithering bump at about 19kHz:

1203958380.jpg



Conversion might do some harm, but I was not able to hear where the cuts were made, or any difference in sound anywhere. SACD seems like a scam to me, these "Linn Studio Master" 24/96 files are not really that, more like 24/44 with uprezzing and HF noise added. I also high pass filtered the +22kHz portion, normalized it and slowed down 4 fold. It really is just random noise with no musical modulation.

I talked to some audio engineers, they also do not understand why DSD is used as it has absolutelly no benefits in recording compared to honest 24/96 PCM, and SACD sampling rate is just too low, causing these HF noise problems.

So it looks like the only benefit of SACD is 6 channel surraund capability, it gives no "high resolution" benefits; no real HF content if anybody could hear it, and the higher dynamic range at lower frequences can not be utilised in normal listening conditions.
 
You must log in or register to reply here.
Back
Top