Optimizing pl_mpeg

[bitbank2]

Hi Dominic,
I recently got approved for a grant from NLnet and one of the projects I proposed was to optimize the pl_mpeg player to make it suitable for use on constrained devices (e.g. Microcontrollers). I've forked it and will be sharing my optimizations as I make them. Besides speed optimizations, I will make some #ifdef changes for the different memory challenges on MCUs (e.g. PSRAM).

[phoboslab]

Very cool! I can't promise to merge any of your changes (if that's even desired), but I'll certainly have a look.

For performance optimizations, have a look at the n64 port of pl_mpeg in libdragon: https://github.com/DragonMinded/libdragon/tree/preview/src/video – from what I understand they added a bunch of general optimizations for bit reading and VLC lookups (and of course some very n64 specific YUV conversion functions etc).

[bitbank2]

I've actually gone beyond most of the optimizations in that repo. I still have a few more to go...

[bitbank2]

One of the strategies I need to implement for MCUs is knowing which MBs have changed each frame and only drawing those. A significant amount of time on MCUs is spent converting pixel colorspace and pushing those pixels to the display. Any MBs that can be avoided per frame helps a lot.

[siteswapv4]

@bitbank2 There's no issues page on your fork page so I'm messaging you here. I tried your optimized version and it's really impressive ! But currently it's unusable for me because it randomly hangs and takes 100% of the CPU, stopping decoding entirely. It happens with every mpg file I tested when seeking through it, sometimes for particular files I don't even have to seek through.
I hope you can replicate this behavior

gcc pl_mpeg_player_sdl.c -o player -O3 -lSDL2
./player bjork-all-is-full-of-love.mpg

It happens with the example video in the readme too.
When I click on the screen to seek through the video, it stops completely and hangs. This behavior also happens on PSVita, which is the platform I target with my app.

[bitbank2]

Thanks for letting me know - I'll figure it out.

[bitbank2]

@siteswapv4 I'm not able to reproduce your hang on the mac. I'm playing the bjork example video and seeking at the beginning to get the duration. I'm not familiar with other player programs. Can you give me the specific function to call with the specific parameter that makes it hang on the bjork video? I didn't add a seek function yet to my test player.

[siteswapv4]

#define PL_MPEG_IMPLEMENTATION
#include "pl_mpeg.h"

#include <stdio.h>

int main(int argc, char* argv[])
{
	plm_t* plm;
	plm_frame_t* frame;

	plm = plm_create_with_filename(argv[1]);
	
	for (int i = 0; i < 100; i++)
	{
		printf("Loop : %d\n", i);
		frame = plm_decode_video(plm);
		
		plm_seek(plm, i, TRUE);
	}
	
	return 0;
}

This hangs forever at loop 4 for me playing bjork example
I compile and run with :

gcc test.c -o test
./test bjork-all-is-full-of-love.mpg 

[siteswapv4]

Maybe you could open your github repo for issues also so people can report bugs better later
@bitbank2

Hope you can reproduce it this time

[siteswapv4]

Also I'm testing this on linux but same behavior on psvita so I don't think it's relevant

[bitbank2]

I was able to reproduce the problem on my MacOS project. It has to do with the changes I made to the buffer/file access. It gets messed up when seeking. I think I can resolve it. How much faster does it run in your tests? On the Mac it's about 230% faster than the original (so far).

[siteswapv4]

I haven't been able to benchmark it yet because of the crash issue but on desktop my whole game went from 25% to 20% single core usage just with that. I'd have to run tests on the Vita where it's most relevant.

[siteswapv4]

@bitbank2 Just tested your version on psvita (without seeking) and I can play videos with a bitrate 2 to 3 times higher than before (I disable audio and only play video), so yup the improvement is crazy lol

[bitbank2]

I fixed the problem, but discovered something in the design that I really don't like. Because of the way it seems to seek audio and video separately, it gums up the buffer such that it reallocates the file buffer to 2x size mid-decode. I will push my fix for you to try, but I would like to redesign the logic to not ever use memrealloc().

[siteswapv4]

@bitbank2 Completely fixed, both the seek issue and the random crashes I was getting, tested with the same files that were failing before. Thanks a lot !

[siteswapv4]

Do you think you can push the optimizations further or are you about done ?