Tag Archives: scalability

Startups are Creating a New System of the World for IT

One reason for this revolution is explained by Etsy in terms of Conway’s Law:

When a team makes a product the product ends up resembling the team that made it.

I’ll extend this notion to say the team and thus the product end up resembling the underlying technology used to make it. When you change the underlying development infrastructure, by moving to a cloud, you are bound to change teams and processes they create.

(Full Story: Startups are Creating a New System of the World for IT)

High Scalability – 7 Years of YouTube Scalability Lessons

Cheating – Know How to Fake Data -
Awesome technique. The fastest function call is the one that doesn’t happen. When you have a monotonically increasing counter, like movie view counts or profile view counts, you could do a transaction every update. Or you could do a transaction every once in awhile and update by a random amount and as long as it changes from odd to even people would probably believe it’s real. Know how to fake data.

(Full Story: High Scalability – 7 Years of YouTube Scalability Lessons)

NoSQL is a Premature Optimization

Point 1:  NoSQL technologies require more investment than Relational to get going with. 
Point 2:  There is no particular advantage to NoSQL until you reach scales that require it.  In fact it is the opposite, given Point 1.
Point 3:  If you are fortunate enough to need the scaling, you will have the time to migrate to NoSQL and it isn’t that expensive or painful to do so when the time comes.

(Full Story: NoSQL is a Premature Optimization)

Akka Project – event-driven, scalable and fault-tolerant architectures on the JVM

Using the Actor Model together with Software Transactional Memory we raise the abstraction level and provide a better platform to build correct concurrent and scalable applications.
For fault-tolerance we adopt the “Let it crash” / “Embrace failure” model which have been used with great success in the telecom industry to build applications that self-heal, systems that never stop.

(Full Story: Akka Project – event-driven, scalable and fault-tolerant architectures on the JVM)

Evernote Architecture – 9 Million Users and 150 Million Requests a Day

Application redundancy. Each box runs two VMs. A primary VM runs the core stack: Debian + Java 6 + Tomcat + Hibernate + Ehcache +  Stripes + GWT + MySQL (for metadata) + hierarchical local file systems (for file data). DRDB is used to replication a primary VM to a secondary VM on another box. Heartbeat is used to fail over to a secondary VM is the primary VM dies. A smart way to use those powerful machines and make a reliable system with fewer resources.

(Full Story: Evernote Architecture – 9 Million Users and 150 Million Requests a Day)

On HTTP Load Testing

————————————————————0. Consistency.1. One Machine, One Job.2. Check the Network.3. Remove OS Limitations.4. Don’t Test the Client.5. Overload is not Capacity.6. Thirty Seconds isn’t a Test.7. Do More than Hello World.8. Not Just Averages.9. Publish it All.10. Try Different Tools.

(Full Story: On HTTP Load Testing)

How SmugMug survived the Amazonpocalypse

  1. Spread across as many AZs as you can.
  2. Beyond mission critical? Spread across many providers.
  3. Build for failure.
  4. Understand your components and how they fail.
  5. Try to componentize your system.
  6. Test your components.
  7. Relax. Your stuff is gonna break

(Full Story: How SmugMug survived the Amazonpocalypse)

6 Scalability Lessons from Dropbox – One Million Files Saved Every 15 minutes

Poll – Polling 30 Milion Clients All Over the World Doesn’t Scale: Created an HTTP notification structure to avoid polling the server on the client site.

(Full Story: 6 Scalability Lessons from Dropbox – One Million Files Saved Every 15 minutes)

Follow

Get every new post delivered to your Inbox.