This past April, MemSQL spent more than $27,000 on Amazon virtual servers. That’s $324,000 a year. But for just $120,000, the company could buy all the physical servers it needed for the job — and those servers would last for a good three years. The company will add more machines over that time, as testing needs continue to grow, but its server costs won’t come anywhere close to the fees it was paying Amazon.
The host configuration is complex enough that even for a small service it is best to use a Configuration Management System (CMS) to configure and manage the system, but the CMS can’t start work until the hosts exist and have network communications. The CMS itself must be installed and configured. Once the hosts exist and are bound together then the CMS can do the rest of the work and a clean boundary of control and access is established. This will later allow the bottom layer (establishing hosts and installing/configuring the CMS) to be replaced without affecting the actual service installation above. So the goal here is: create and connect hosts with a CMS installed using EC2.
That’s what OpenStack is: a way for the rest of the world to compete with Amazon. “Amazon [is] at war with every IT vendor out there,” says Sebastian Stadil, the CEO of an open source cloud management outfit Scalr, the founder of the Silicon Valley Cloud Computing group, and a former resident of the Rainbow Mansion. “I think one of the reasons OpenStack is getting so much traction — despite, to be frank, iffy stability — is that it represents the industry’s only hope to survive.”
(Full Story: http://www.wired.com/wiredenterprise/2012/04/openstack/all/ )
Obama Campaign AWS Infrastructure In One Diagram
(Full Story: http://awsofa.info/ )
So until a startup is rich and successful, it makes sense to just live with your Amazon addiction and suffer the occasional outage with the “blame Amazon” explanation. Startup CEOs should be sure to tell everyone about this dependency in advance so that such a failure doesn’t come as a surprise. But Netflix is beyond that stage. It is time for Netflix to start recovering from Amazon addiction and take full responsibility for the quality of its service.
we can deploy major upgrades on new hardware.
our failure plan for some non-critical systems, where perhaps up to an hour of occasional downtime is acceptable, is to monitor the box, and if it fails, spin up a new box and restore the system from backups
we can scale up in response to load events, rather than in advance of them
we can not worry about pre-launch capacity calculations
Virtual hardware doesn’t last as long as real hardware.
You need to be in more than one zone, and redundant across zones.
Multi-zone failures happen, so if you can afford it, go multi-region too.
(Full Story: AWS: the good, the bad and the ugly | awe.sm: the blog)
With the release of Ruby container support on AWS Elastic Beanstalk, it is now possible to deploy Rails (or any Rack-based) applications onto the AWS infrastructure in just a few easy steps. The container is backed by Amazon EC2 and automatically provisions the machines to use Phusion Passenger (with nginx under the hood). You can run your own local database with your application, or better yet, you can have Elastic Beanstalk automatically set you up with a MySQL database backed by Amazon Relational Database Service (RDS). All of the machines sit behind a load balancer with auto-scaling support automatically configured so you don’t even have to think about ops. Scaling is built-in.
Edda is a service that polls your AWS resources via AWS APIs and records the results. It allows you to quickly search through your resources and shows you how they have changed over time.
Previously this project was known within Netflix as Entrypoints (and mentioned in some blog posts), but the name was changed as the scope of the project grew. Edda (meaning “a tale of Norse mythology”), seemed appropriate for the new name, as our application records the tales of Asgard.
Icebox is a simple archiving application built on top of Dropbox, the file sharing application that we all know and love. Everything you love about Dropbox—it’s simplicity, ease of use, reliability—comes with IceBox because of this. Add IceBox and you’ll be able to archive your files just as easily and quickly as you would share files on Dropbox.
(Full Story: Icebox Pro – move files to Glacier vis Dropbox)
At ZeroTurnaround, we need this for testing LiveRebel with larger deployments. LiveRebel is a tool to manage JEE production and QA deployments and online updates. It is crucial that we support large clusters of machines. Testing such environments is not an easy task but luckily in 2012 it is not about buying 800 machines but only provisioning them in the cloud for some limited time. In this article I will walk you through setting up a dynamic Tomcat cluster and running some simple tests on them. (Quick note: When I started writing this article, we had only tested this out with 100 Tomcat machines, but since then we grew to be able to support 800 instances with LiveRebel and the other tools).