So until a startup is rich and successful, it makes sense to just live with your Amazon addiction and suffer the occasional outage with the “blame Amazon” explanation. Startup CEOs should be sure to tell everyone about this dependency in advance so that such a failure doesn’t come as a surprise. But Netflix is beyond that stage. It is time for Netflix to start recovering from Amazon addiction and take full responsibility for the quality of its service.
Edda is a service that polls your AWS resources via AWS APIs and records the results. It allows you to quickly search through your resources and shows you how they have changed over time.
Previously this project was known within Netflix as Entrypoints (and mentioned in some blog posts), but the name was changed as the scope of the project grew. Edda (meaning “a tale of Norse mythology”), seemed appropriate for the new name, as our application records the tales of Asgard.
Fourteen Commandments of Web Scale Cloud Deployment
1. Thou shalt not have Sticky in-memory sessions
2. Thou shalt not directly use a Central SQL database in the user request path
3. Thou shalt not store important data on ephemeral instances
4. Thou shalt embrace a homogenous architecture
5. Thou shalt understand and embrace the CAP theorem
6. Thou shalt guard all external calls using the Dependency Command Pattern
7. Thou shalt be prepared to scale according to thy needs
8. Thou shalt keep a wary eye on they cost
9. Thou shalt secure thy data and access points
10. Thou shalt instrument thy code
11. Thou shalt effectively monitor thy instances
12. Thou shalt deploy thy instances in multiple regions and zones
13. Thou shalt be wary of SPOF
14. Thou shalt always plan for failure
(Full Story: Architectural Building Blocks of Netflix Cloud Platform)
JMeter allows us to customize our test cases based on our application logic/datamodel. The Cassandra JMeter plugin we are releasing today is described on the github wiki here. It consists of a jar file that is placed in JMeter’s lib/ext directory. The instructions to build and install the jar file are here.
(Full Story: JMeter Plugin for Cassandra)
1. Fancy ML techniques don’t matter. The winning BellKor/Pragmatic Chaos teams implemented ensemble methods with something like 112 techniques smushed together. You know how many of those the Netflix team implemented? Exactly two: RBM’s and SVD.
2. Domain knowledge trumps statistical sophistication. This has always been the case in the recommendation engines I’ve done for clients. We spend most of our time trying to understand the space of your customers’ preferences — the cells, the topology, the metric, common-sense bounds, and so on. You can OO program these characteristics. And (see bottom) doing so seems to improve the ML result a lot.
3. What you measure matters more than what you squeeze out of the data
Cloud is run by developer organization, traditional IT roles are going away
One of the best stories in digital entertainment during the past decade was how Netflix wrested control of the home-video rental market from national chain Blockbuster.
(Full Story: Former Blockbuster CEO tells his side of Netflix story)
Netflix 30%, HTTP 18%, YouTube 11%, BitTorrent 10%…