I've found amazon GPU instances to be really expensive (even the spot prices have been high recently), especially if you need it for longer runs for deep learning. The other issue is that the additional layers of virtualization create bandwidth overhead issues.
I'd like to see something in the cloud thats bare-metal / full access to GPUs (Maybe a good idea to start one). For scaling higher with a very large number of GPUs, you'd need Infiniband but at some point there is going to be a bandwidth tradeoff.
It would be interesting if someone could run some benchmarks of these instances versus a physical server.
I did the analysis about 9 months ago for my team when the 980 ti's came out, and the AWS pricing was expensive (we built a server that paid for itself in 2 weeks compared to a g2.8xlarge). This is largely because the 980 ti is actually a ridiculously good deal for price/performance.
The bigger problem we ran into is that AWS instances use a lot of small GPUs, which don't scale well using a lot of deep neural network tools (e.g. theano). It was never really a viable option for us.
Somebody should make a startup that allows people to sell access to their computers by the minute. Like spot instances in the cloud ... in people's basements. The true sharing economy.
Why not just move forward to realizing some sort of distributed/decentralized internet?
Something like a combination of Freenet, TOR, BOINC, blockchain etc. technologies, using the current "legacy" internet as a backbone, where anyone can voluntarily offer their computing and storage resources to the network at varying levels of participation.
Say you could offer your laptop as a simple discovery/directory node to simply help others connect and find stuff, and your desktop as either a static-content serving node or as a computation node that can host distributed applications, like SETI@home or web apps like Facebook.
Maybe even reward cryptocurrency to those who offer the most resources.
There is Gridcoin which is a cryptocurrency that rewards for BOINC projects.
I wouldn't be surprised if certain services became more centralized, offloading computing to the cloud. Consumer grade devices would become thin clients (like back in the day). NVIDIA has hinted in this direction, I can remember something about GaaS (Gaming as a Service).
Various companies have tried this. What you end up with is 99% of your workforce is compromised machines, and law enforcement at your door every other day asking where checks are being mailed.
What if you limited people to writing in a domain specific language: one that ran distributed on this infrastructure? How would that make it different than folding at home, for example?
"“Monthly Uptime Percentage” is calculated by subtracting from 100% the percentage of minutes during the month in which Amazon EC2 or Amazon EBS, as applicable, was in the state of “Region Unavailable.” Monthly Uptime Percentage measurements exclude downtime resulting directly or indirectly from any Amazon EC2 SLA Exclusion (defined below)."
That's a regional outage, they provide no SLA for individual instances:
Amazon EC2 SLA Exclusions... (v) that result from failures of individual instances or volumes not attributable to Region Unavailability
Presumably if you're using a cloud hosted in people's basements, if someone's basement server dies, you'd just pick one from someone else's basement, so this model could provide better availability than AWS.
That's right, you can install on your own GPU servers. Infiniband+RDMA transport is also supported which typically doubles the number of GPUs you can scale to.
We're adding support for other clouds, particularly ones with higher-end GPUs so feedback like this is good to know.
I'd like to see something in the cloud thats bare-metal / full access to GPUs (Maybe a good idea to start one). For scaling higher with a very large number of GPUs, you'd need Infiniband but at some point there is going to be a bandwidth tradeoff.
It would be interesting if someone could run some benchmarks of these instances versus a physical server.