Brent Sordyl's blog

Architectural Building Blocks of Netflix Cloud Platform

July 27, 2012 by Brent Sordyl 0 Comments

Fourteen Commandments of Web Scale Cloud Deployment
1. Thou shalt not have Sticky in-memory sessions
2. Thou shalt not directly use a Central SQL database in the user request path
3. Thou shalt not store important data on ephemeral instances
4. Thou shalt embrace a homogenous architecture
5. Thou shalt understand and embrace the CAP theorem
6. Thou shalt guard all external calls using the Dependency Command Pattern
7. Thou shalt be prepared to scale according to thy needs
8. Thou shalt keep a wary eye on they cost
9. Thou shalt secure thy data and access points
10. Thou shalt instrument thy code
11. Thou shalt effectively monitor thy instances
12. Thou shalt deploy thy instances in multiple regions and zones
13. Thou shalt be wary of SPOF
14. Thou shalt always plan for failure

Full slides at http://www.slideshare.net/stonse/netflix-cloud-platform-building-blocks

(Full Story: Architectural Building Blocks of Netflix Cloud Platform)

aws, cloudcomputing, scalability

aws, cloudcomputing, netflix, scalability

StubHub Architecture: The Surprising Complexity Behind the World’s Largest Ticket Marketplace

July 3, 2012 by Brent Sordyl 0 Comments

Scalability is specialization. Every problem space has its unique characteristics and any system must be built to solve that particular problem. StubHub is constrained by the need for a safe buying experience, the unique nature of the ticket market, bursty traffic, and the vagaries of events. Their system must reflect those requirements.

(Full Story: StubHub Architecture: The Surprising Complexity Behind the World’s Largest Ticket Marketplace)

architecture, jboss, scalability

Jetpants: a toolkit for huge MySQL topologies

June 12, 2012 by Brent Sordyl 0 Comments

Today, we’re happy to announce the open source release of Jetpants, Tumblr’s in-house toolchain for managing huge MySQL database topologies. Jetpants offers a command suite for easily cloning replicas, rebalancing shards, and performing master promotions. It’s also a full Ruby library for use in developing custom billion-row migration scripts, automating database manipulations, and copying huge files quickly to multiple remote destinations.

Dynamically resizable range-based sharding allows you to scale MySQL horizontally in a robust manner, without any need for a central lookup service or massive pre-allocation of tiny shards. Jetpants supports this range-based model by providing a fast way to split shards that are approaching capacity or I/O limitations. On our hardware, we can split a 750GB, billion-row pool in half in under six hours.

(Full Story: Jetpants: a toolkit for huge MySQL topologies)

mysql, scalability

mysql, scalability, tumblr

Startups are Creating a New System of the World for IT

May 25, 2012 by Brent Sordyl 0 Comments

One reason for this revolution is explained by Etsy in terms of Conway’s Law:

When a team makes a product the product ends up resembling the team that made it.

I’ll extend this notion to say the team and thus the product end up resembling the underlying technology used to make it. When you change the underlying development infrastructure, by moving to a cloud, you are bound to change teams and processes they create.

(Full Story: Startups are Creating a New System of the World for IT)

architecture, cloudcomputing, innovation, opensource, scalability

architecture, cloudcomputing, devops, innovation, opensource, scalability

High Scalability – 7 Years of YouTube Scalability Lessons

April 5, 2012 by Brent Sordyl 0 Comments

Cheating – Know How to Fake Data -
Awesome technique. The fastest function call is the one that doesn’t happen. When you have a monotonically increasing counter, like movie view counts or profile view counts, you could do a transaction every update. Or you could do a transaction every once in awhile and update by a random amount and as long as it changes from odd to even people would probably believe it’s real. Know how to fake data.

(Full Story: High Scalability – 7 Years of YouTube Scalability Lessons)

mysql, scalability, youtube

Why did Bonobos have such an epic fail on Cyber Monday 2011? – Quora

December 7, 2011 by Brent Sordyl 0 Comments

(Full Story: Why did Bonobos have such an epic fail on Cyber Monday 2011? – Quora)

loadtesting, scalability

NoSQL is a Premature Optimization

July 31, 2011 by Brent Sordyl 0 Comments

Point 1: NoSQL technologies require more investment than Relational to get going with.
Point 2: There is no particular advantage to NoSQL until you reach scales that require it. In fact it is the opposite, given Point 1.
Point 3: If you are fortunate enough to need the scaling, you will have the time to migrate to NoSQL and it isn’t that expensive or painful to do so when the time comes.

(Full Story: NoSQL is a Premature Optimization)

database, nosql, scalability

Netflix Cloud Architecture Slideshare – Velocity Conference 2011

July 5, 2011 by Brent Sordyl 0 Comments

Cloud is run by developer organization, traditional IT roles are going away

(Full Story: Netflix Cloud Architecture Slideshare – Velocity Conference 2011)

aws, cloudcomputing, scalability

aws, cloudcomputing, netflix, scalability

Akka Project – event-driven, scalable and fault-tolerant architectures on the JVM

June 7, 2011 by Brent Sordyl 0 Comments

Using the Actor Model together with Software Transactional Memory we raise the abstraction level and provide a better platform to build correct concurrent and scalable applications.
For fault-tolerance we adopt the “Let it crash” / “Embrace failure” model which have been used with great success in the telecom industry to build applications that self-heal, systems that never stop.

(Full Story: Akka Project – event-driven, scalable and fault-tolerant architectures on the JVM)

java, scalability

eventdriven, java, scalability

Evernote Architecture – 9 Million Users and 150 Million Requests a Day

June 2, 2011 by Brent Sordyl 0 Comments

Application redundancy. Each box runs two VMs. A primary VM runs the core stack: Debian + Java 6 + Tomcat + Hibernate + Ehcache + Stripes + GWT + MySQL (for metadata) + hierarchical local file systems (for file data). DRDB is used to replication a primary VM to a secondary VM on another box. Heartbeat is used to fail over to a secondary VM is the primary VM dies. A smart way to use those powerful machines and make a reliable system with fewer resources.

(Full Story: Evernote Architecture – 9 Million Users and 150 Million Requests a Day)

scalability