Indeed, back in '95 or so there was a library called CrossELF that'd let you compile ELF so files and use a tiny loader linked to CrossELF for each platform you cared about to load the main .so file, and you could build platform-independent code with it. I remember writing some simple networking code where the loader just had a tiny set of shims for a few calls and the rest of the networking code was a single binary for both Linux and Win32.

The problem is wrapping the relevant APIs - as you can see w/e.g. Wine. For some functionality - like networking - the surface is pretty small, for others its a nightmare.