audio encoder identification experiment - lossless / lame v4 / vorbis q4
BotB Academy Bulletins
 
 
99782
Level 27 Chipist
post #99782 :: 2018.05.15 4:30pm
  
  Flaminglog, goluigi, Savestate and Slimeball liēkd this
tl;dr – can you distinguish between flac, mp3, and ogg vorbis?

today pigdevil created the thread "BotB should require renders in .ogg instead of .mp3", and provided an audio comparison between lossless encoding, 64 kbps mp3, and 64 kbps vorbis. while i completely agree that vorbis is the better encoder at that bitrate, there's no reason to use 64 kbps on botb given the 10 MB size limit unless your entry is over 20 minutes long. which it should not be.

partly because of that thread and partly from sheer curiosity, i created a sample set of 20 randomly selected songs from botb, given the criteria that:

1. the song must have available source or a >200 kbps mp3 render
2. the song must be less than two minutes long
3. the song must have a score of at least 25
4. the song must have at least five favorites
5. somehow two of them ended up being triangle-only nsfs

each song is rendered once lossless, once as mp3 using lame at v4, and once as ogg using libvorbis at q4 (lame v4 and libvorbis q4 both yield a bitrate of roughly 128 kbps). however, the renders have all been converted back to wav and arranged in a random order.

IF you are interested, please download the set and post (or preferably pastebin/etc) your guesses for which files are encoded which way. the total duration of the files is ~72 minutes, but you don't have to listen to all the renders all the way through, so i expect it would take 30-60 minutes to make your guesses, depending on how thorough you are. if that sounds like too much for you, then feel free to just do the first ten! i'll post the results after a week or something.

https://drive.google.com/file/d/1zKUXyUi42GOYA-RkXS0a8Y0jMDqAySto/view?usp=sharing (365 MB, .7z)

the numbers in the filenames are entry numbers; you can find the page for entry number x at http://battleofthebits.org/arena/Entry//x/ (note the double slashes).
 
 
99792
Level 25 Mixist
post #99792 :: 2018.05.15 6:47pm
  
  pedipanol, raphaelgoulart, Savestate, Slimeball, pigdevil2010, petet and Jangler liēkd this
"there's no reason to use 64 kbps on botb given the 10 MB size limit unless your entry is over 20 minutes long. which it should not be."

best not give Flaminglog any ideas for Opus VII (´・ω・`)
 
 
99793
Level 27 Chipist
post #99793 :: 2018.05.15 7:04pm
 
 
99794
Level 19 Criticist
Xyz
post #99794 :: 2018.05.15 7:53pm
  
  Doxic, VirtualMan, Sintel and RazerBlue6 liēkd this
Fuck the time police
 
 
99804
Level 23 Chipist
post #99804 :: 2018.05.16 1:47am
  
  Jangler liēkd this
 
 
99805
Level 28 Renderist
post #99805 :: 2018.05.16 3:26am :: edit 2018.05.16 3:29am
  
  Jangler and RazerBlue6 liēkd this
 
 
99980
Level 29 Chipist
post #99980 :: 2018.05.18 8:09am
  
  Jangler liēkd this
https://pastebin.com/RUpFSb7Q
i have to do things so here's the 1st 16
 
 
100034
Level 19 Chipist
post #100034 :: 2018.05.19 7:24am
  
  Jangler hæitd this
MP3 render must be 320Kbps High Quality.

This song must be 4 minutes limit. Longer will be resize to low quality 192kbps or lower

Here is the link https://www.quora.com/What-is-the-average-size-of-an-MP3-song
 
 
100035
Level 27 Chipist
post #100035 :: 2018.05.19 7:52am :: edit 2018.05.19 10:04am
  
  Robyn, Cessor Safari and Savestate liēkd this
what? this thread isn't about botb rules; it's a listening experiment/game. but i'll bite on this off-topic comment to say that requiring 320 kbps mp3 renders when possible would be stupid since v0 vbr is kinder to server space, data plans, bandwidth, download speeds, and storage on small devices. and i'll be damned if anyone can hear a difference.

edit: especially since many of the tracks on botb are mono by nature of the format
 
 
100247
Level 27 Chipist
post #100247 :: 2018.05.22 12:10am
  
  funute and Savestate liēkd this
result stats spreadsheet @ google docs


raw results text file @ pastebin


pretty small data set but basically, no one was able to distinguish between lossless and ~128 kbps lossy encoding with better than ~60% accuracy. removing the really minimal & lo-fi entries from the data set didn't make a difference. the sheet doesn't display this but removing the four transcoded entries didn't help either.

additionally, no one was able to distinguish between ogg vorbis and lame mp3 with better than 50% accuracy on the entries where they correctly identified the lossless encoding, meaning that people generally thought the mp3s actually sounded more accurate than the oggs at comparable bitrates.

slimeball didn't correctly identify any of the lossless files. sorry slimeball. also note that savestate didn't even try to guess for three of the sixteen entries he listened to, so his accuracy is probably skewed a bit high.

honestly i was pleased that anyone at all participated in this experiment so thank you all & i hope this was at least a tiny bit illuminating. there are bigger better data sets out there on this type of things but i thought it would be fun and maybe useful to try this specifically with the stuff we deal with here on botb.
 
 
100259
Level 28 Renderist
post #100259 :: 2018.05.22 6:21am
  
  charlotte liēkd this
s/.wav/.oopsie
s/.ogg/.wav
s/.oopsie/.ogg

(´・ω・`)
 
 
100262
Level 22 Chipist
post #100262 :: 2018.05.22 9:29am :: edit 2018.05.22 9:30am
  
  raphaelgoulart, Cessor Safari, puke7, MiDoRi and Slimeball liēkd this
The most important thing imo is to use up to date encoders. If you encode your mp3s using liblame version Tutankhamun-pre.alpha.lol that you installed together with some 10 year old Audacity, chances are it's gonna sound shit even at 256 kbps.

Oh and .ogg is so 2010. Kool kidz use Opus.
 
 
100263
Level 27 Hostist
post #100263 :: 2018.05.22 9:51am
haha this data is the proof in the pudding
 
 
100265
Level 27 Chipist
post #100265 :: 2018.05.22 10:36am
  
  Savestate liēkd this
yeah, for the record, the mp3s used lame 3.100, and the oggs used libvorbis 1.3.6, which are the latest versions of each encoder.
 
 
100266
Level 27 Chipist
post #100266 :: 2018.05.22 10:52am :: edit 2018.05.22 10:54am
i should mention, though, there there were three entries (a vic20, a mod64k, and an allgear) where everyone correctly identified the lossless file. so it's not that lossy encoding is never perceptible; it's just more detrimental in some circumstances than others.

but for an channelf and a mod8k, everyone unanimously misidentified the oggs as wavs. interpret this information how you will lol

edit: this is taking into account slimeball's s// post above
 
 
100293
Level 23 Mixist
post #100293 :: 2018.05.23 1:59am
  
  raphaelgoulart, Robyn and Slimeball liēkd this
Opus FTW! It's a very capable and creative format. I did some basic tests with it a while back and had a ton of fun messing around with all of it's different compression techniques.
 
 

LOGIN or REGISTER to add your own comments!