There has been a neural network scaler called NNEDI3 available as a plugin for avisynth for a few years. It was trained in exactly the way you are talking about and does an amazing job at both deinterlacing and upscaling video.

It does a great job of scaling pixel art too. I'm kinda surprised it hasn't been implemented in any emulator. With the newer opencl implantation it is finally able to run in real-time.

http://screenshotcomparison.com/comparison.php?id=46792