Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The issue with depixelizing pixel art is that you need an AI that can identify semantically what the features in the sprite are. When you look at Mario you know you're looking at a stout little man with a hat, you can identify his eyes, ears, nose, hair, hat, moustache, shirt, gloves, overalls and shoes. Given that knowledge, if I were to depixelize Mario's sprite, I'll know the rough shape of his shoes and where the shading should go and what shape it should have, i.e. I can reason what the 3D shape is and how the 2D shape was derived from that. I'll also be able to identify which lines were put there to make sure separate elements can be legible and not make those thicker than needed. Or that his eyes blend with his hat simply because there aren't enough pixels to make them separate.

For simple shapes, like the background, their algorithm works really well, but for complex objects it fails, because it distorts details that were put in with very careful thought and completely depend on the resolution. Such small sprites rely a lot on being looked at by someone who can identify semantically what they're looking at, and any really successful depixelization solution will need to be able to understand what basic shapes the sprite is made of based on what it's supposed to represent.



I've been thinking about doing something like this. Compress images down to half resolution, then train a neural network to blow it back up and guess the pixels as accurately as possible.


There has been a neural network scaler called NNEDI3 available as a plugin for avisynth for a few years. It was trained in exactly the way you are talking about and does an amazing job at both deinterlacing and upscaling video.

It does a great job of scaling pixel art too. I'm kinda surprised it hasn't been implemented in any emulator. With the newer opencl implantation it is finally able to run in real-time.

http://screenshotcomparison.com/comparison.php?id=46792


Whoa, that's pretty good.


I did something very much like this years ago (upscaling used to be an obsession of mine). The results were pretty disappointing.

But there's a lot of space to explore there in terms of choosing and designing the neural net, choosing the right training set, and figuring out an initial transform for data that is input to the neural net.


New deep learning techniques might help this approach a lot.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: