Lessons Learned: Innovation inside the box

Here was my prescription for this situation. I asked the team to consider creating what I call a sandbox for experimentation. The sandbox is an area of the product where the following rules are strictly enforced:
1. Any team can create a true split-test experiment that affects only the sandboxed parts of the product, however:
2. One team must see the whole experiment through end-to-end.
3. No experiment can run longer than a specified amount of time (usually a few weeks).
4. No experiment can affect more than a specified number of customers (usually expressed as a % of total).
5. Every experiment has to be evaluated based on a single standard report of 5-10 (no more) key metrics.
6. Any team that creates an experiment must monitor the metrics and customer reactions (support calls, forum threads, etc) while the experiment is in-progress, and abort if something catastrophic happens.
(Link: Lessons Learned: Innovation inside the box)

Advertisement

No comments yet... Be the first to leave a reply!

Leave a Reply

Fill in your details below or click an icon to log in:

Gravatar
WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.