Hello, I think benchmarking encoders is pretty neat but I'm a little skeptical about the nature of the video gaming footage people use for the benchmarking so I thought I would start my own dataset of sorts. I started with two game titles, different art styles, that I could complete within an hour, a once through at a reasonable high quality*, used ffprobe to export pkt_size and excel to give me a moving average (in MB/s) that graphs pretty well.. but I'm scratching my head here with scene selection criteria that might make sense in practice, for example, . *I'll go back and attempt to recreate the scenes, recording lossless So if you have any ideas surrounding what makes a good video dataset, or how I could approach scene selection differently feel free to leave a comment below.

Some backstory, I couldn't relate to the footage presented in some hardware acceleration comparisons using VMAF so I followed a guide on how to operate the VMAF tool myself. Along with finding that guide I found other things like studies on LCEVC by Jan Ozer and Nabajeet Barman. In they use a plot of SI & TI values as a way to identify a spectrum of complexity (pg18&19 ), which I had trouble with compiling , and eventually found Barmans github where they shared a I could run on .yuv files but the problem with .yuv files is I'd have to split a 4.5hr playthrough into like 28 jobs due to their total size (25TB). That lead me to looking at what Ozer did which was reviewing the datarate of CRF 23 recordings. Time passed, and I had changed it to CQP 14 instead because I was under the impression smaller details like foilage might want more bitrate to be precise but I suspect I pushed it into diminishing returns territory because I believe 's should've been much higher than this content 's or perhaps using the datarate to predict complexity can only get you so far.

did some more playthroughs and probes ( pending approval), with the probes and example xls for anyone to play with. I think I should tackle SI&TI values next, see what they have to say about complexity for these two titles. Edit: UPDATE Success in a vm, then copied to my system but now I have to . my previous headache was "installing cmake", you don't use the installer.. you unzip the portable and edit your system environment variables PATH to include that portables /bin directory Edit: UPDATE2 Success, , ran slower than real time in my case, here's on vampire survivors (threw the -VCA.csv in with the probes), I'll throw THPS1&2 on next then take nap. Edit: UPDATE3 --no-chroma doubled the fps* for VCA and perhaps --no-lowpass will have an impact as well, I'll look into benchmarking the matlab script either with a yuv file or get virtualdub2 to frameserve *it doubled Edit: UPDATE4 I did a , the MATLAB scripts output is much larger than VCAs csv, and only had 33% utilization on my system. after further consideration, the MATLABs output is big because of the extra millions of data entries it had for each 2560x1440 pixel multiple times, not necessarily extra math, also scripts not fully utilizing a system is prevalent in googles search results on why that might be the case. rec p910 or siti-tools mentioned how things near the edges of a frame can throw off the calculations, so I can look into placing a black boarder over footage to feed VCA to see what sort of result we can get but I have my doubts the TI scale will be under 100. I haven't directly compared avg framesize to siti yet, SI appears to correlate pretty strongly with framesize for all those vampire surivor ffprobes.

Threw some commands through powershell, replotted some familiar graphs, and I'll share my thoughts. As a reminder these were four separate recording sessions.. THPS1&2 , ; , Vampire Survivors , ; , If I played through either title again I'm confident the framesize or spatiotemporal (SI&TI) values would continue to plot in a similar manner. A lower CQP appears to decode faster, any forced -hwaccels did not help this. A little puzzled by THPS1&2 results, the decode speed is lower so it must be complex? I interpret the SITI plots don't agree and point to Vampire Survivors being more complex instead*. *I'm aware it's recommended to look at the SITI mean, I can update this later. I'll fix the scale of THPS1&2 plots later. I'll throw the re-ran csv's into the folder later. I left a bugreport on github for VCA, if its supposed to mirror siti-tools output or if theirs stands alone. I currently feel using ffprobe would be the way to identify scenes of interest, and the siti stuff could be a way to show how those scenes plot in a spectrum of complexity. No thought paid towards "an avg scene" yet, watching lets plays on the phone(procrastinating) while tests run.