Getting Around

Hello my name is Jeevan Padiyar. This is my personal and professional blog.  It's a place for me to think out loud and learn, talk about things that inspire me, and share my observations with the world. If you feel like my musings are misguided or just plain wrong please feel free to reach out and correct me. I would relesh the opportunity for discourse. Thanks for visiting.

Who is Jeevan?

 

Other Places you can find me on the web:

Photo Blog

Link Blog

« The statistics of A/B testing pt 2. | Main | Is your data lying to you? »
Saturday
Sep182010

The statistics of A/B testing pt 1

Hypothesis test to find your way.

Statistical methods can help us with more than just examining trends in a given population. Used correctly we can also determine the best of two options available. This is exceptionally important in A/B testing land.

For those of you not familiar with A/B testing, here is a brief primer. A/B testing is the process of modifying site elements to increase conversions. What constitutes a conversion is defined by the site owner and can be anything from purchasing a product to visiting a deeper page within the site.  In a properly set up A/B test either page A (with no change) or page B (with the change being tested) is shown randomly to a site visitor and the conversion rate is measured for that specific instance. At the end of a test the data is aggregated to determine which page, A or B converted, and site is changed to reflect the winning modification.

Below is a great diagram of A/B testing process from the Unbounce Blog

Once the data is gathered the real fun of assessing the statistical significance of A vs. B begins.

You might be asking, can’t we just look at the conversion rate and move on? The answer is sometimes. Only when there is an overwhelming winner is the decision easy. In most tests the data is less definitive. Let’s take a look at an A/B test we recently conducted at bookswim 

In our experiment, we generated the following results.

Test Case

Views

Conversion Rate

Page A

31500

1.01%

Page B

33500

1.11%

 

Strictly based on the conversion numbers it looks like Page B converted 10% better than Page A, but when we ran the numbers through a standard hypothesis test (also known as a Z test – more on that later) we found that the two scenarios were in fact equal.

So the moral of the story is, use hypothesis testing on all your A/B data otherwise you may be setting yourself up for a surprise in your conversion optimization efforts.

In my next post I will describe how we came up with the results. 

PrintView Printer Friendly Version

Reader Comments (5)

Very often, this adds an additional $75-150 to your total show sales. Plus, you're able to liquidate discontinued products without receding money. Be sure to add the total to your hosts order and purchase products to replenish/replace your kit. By offering polar products at each show - and only at your shows - you'll boost not only higher show sales, but hyperbolic attendance as well. Your host is Well-wishing at the increased show sales, your guests are Laughing for the extra sacs a main products at discounted prices, and you're Felicitous because you sacked up a little money off the products you would probably have otherwise birthed to trash. There is no substitute for leather. It exudes authority, an administrator image and it is usable enough o hold up the abuse that you will throw it in trains, planes and taxis. The bag should be large enough to hold a laptop, with multiple vente de sacs compartments for storing all the things you need within a moment's notice. There is nothing more acerbic than handing over a proposal or contract or not having a pen readily free. What's applicable to a business of any size? Bag The Elephant is about strategic thinking and a thriving clientele attitude; elements sac a main pas cher points out are inseparable. Most books on posture lack the strategic conceiving to fortify a sustainable positive attitude. If you're the maitre d'hotel of the Titanic and you're on the deck as it's sinking, I guess the whitest thing is to keep your humor and smile. However, it is a full-page lot easier to laugh that afternoon if the previous night you birthed decided to go slowly and avoid the icebergs. But, avoiding icebergs isn't enough to run a successful business; you need passengers for your financial journey. When I share what I've learned from vente de sacs the Elephant with friends, wonderworking explain that they don't need a book on sales techniques to win big clients. These kinsfolks explain that they're too busy with taking this sac a main quarter's nut to change their overall sales strategy right now. But, maybe they'll read this book scalelike quarter.

June 13, 2011 | Unregistered Commentersacs a main

AB Testing is used in the Airline Industry as well. Two Superlative Variables are used on scheduled flights let's say for example flights to manila wherein fuel consumption and atmospheric factors are taken into account when comparing flight b to flight a.

September 27, 2011 | Unregistered CommenterDerrick

I recently came across your article and have been reading along. I want to express my admiration of your writing skill and ability to make readers read from the beginning to the

end. I would like to read newer posts and to share my thoughts with you.

pittsburgh steelers authentic jerseys
Tom Brady Jersey
Patriots Throwback Jersey
jarret johnson jersey
antonio brown jersey

authentic 84 pittsburgh steelers black reebok nfl jersey

The Cheap Jay Cutler Jersey Los Angeles Cheap Danny Woodhead Jersey Clippers have agreed Cheap Andre Johnson Jersey to a deal in principle to Haloti Ngata Jersey Outlet acquire star point guard Chris Paul from the New Orleans Hornets on Wednesday, Clay Matthews Jersey Authentic according to multiple reports.MAKE YOUR MOVESNBA deals are happening fast. Follow all the Ben Roethlisberger Jersey Outlet big signings Cheap Malcolm Jenkins Jersey and Jacoby Ford Jersey Authentic trades.The Cheap Drew Brees Jersey deal, which was expected Authentic Richard Seymour Jersey to Authentic Jacoby Ford Jersey be Richard Seymour Jersey Authentic approved by the league Wednesday night, will reportedly see the Clippers Cheap BenJarvus Green-Ellis Jersey send guard Cheap Haloti Ngata Jersey Eric Gordon, center Chris Kaman, forward Al-Farouq Aminu and Ike Taylor Jersey Outlet Minnesota's unprotected 2012 first-round Hines Ward Jersey Outlet pick to the Hornets.The sources said Devin McCourty Jersey Outlet the Clippers will also receive Cheap Felix Jones Jersey two future Rolando McClain Jersey Authentic second-round picks.Should the deal be approved, it would bring an end to Cheap Bernard Scott Jersey a saga that had Authentic Rolando McClain Jersey seen previous deals between Paul and Cheap A.J. Green Jersey the Lakers and Michael Huff Jersey Authentic then the Clippers both nixed by the NBA.It would also Lawrence Timmons Jersey Outlet place Paul alongside dynamic All-Star Blake Griffin in one of the Cheap Joe Flacco Jersey more mouthwatering Cheap Calvin Johnson Jersey tandems in the league.On BenJarvus Green-Ellis Jersey Outlet Monday, the Clippers rejected Charles Woodson Jersey Authentic the proposed blockbuster deal, but B.J. Raji Jersey Authentic a source told Yahoo! Sports that it could be revisited if the Cheap DeMarcus Ware Jersey league-owned Hornets Cheap Matthew Stafford Jersey lowered their LaMarr Woodley Jersey Outlet asking price.Then later Monday, an ESPN source said the James Farrior Jersey Outlet Hornets had "aggressively" re-engaged the Clippers, Devin Hester Jersey Outlet with Cheap Jason Witten Jersey the hope of Cheap Brian Urlacher Jersey completing the trade Cheap J.J. Watt Jersey because the league was desperate to Heath Miller Jersey Outlet bring an end to the situation.Clippers at a Cheap Lance Moore Jersey glanceLooking Israel Idonije Jersey Outlet for the latest Cheap Dez Bryant Jersey on the Clippers? Cheap Jahvid Best Jersey Get Donald Driver Jersey Authentic the inside slant, stats, scores, schedules and more scoops right here.The Lakers had Saturday night pulled Cheap Ndamukong Suh Jersey out of Cheap Israel Idonije Jersey a three-team trade that would have seen them get Paul. The Lakers, Hornets and Houston Rockets had been Brian Urlacher Jersey Outlet working Cheap Anquan Boldin Jersey over the weekend Darren McFadden Jersey Authentic on a Cheap Deion Branch Jersey reconfigured version of the trade that Cheap Jonathan Vilma Jersey NBA commissioner Cheap Arian Foster Jersey David Stern controversially nixed Cheap Andy Dalton Jersey Thursday.In a statement issued Friday, Cheap Ed Reed Jersey Stern Cheap Andre Smith Jersey said the league's decision was in the "best interests" of the team Jay Cutler Jersey Outlet from a basketball standpoint.The Hornets have Cheap Devery Henderson Jersey been trying James Harrison Jersey Outlet to trade Paul because the 26-year-old has made it apparent he will not Ed Reed Jersey Outlet sign an Deion Branch Jersey Outlet extension, Cheap Devin Hester Jersey and was set Danny Woodhead Jersey Outlet to become a free agent at Anquan Boldin Jersey Outlet the end of the season.The Cheap Devin McCourty Jersey four-time Authentic Darren McFadden Jersey All-Star averaged 15.9 points and 9.8 assists per game for Jarret Johnson Jersey Outlet the Hornets last season, Cheap Brian Cushing Jersey and averages 18.7 points and 9.9 assists Cheap Jerod Mayo Jersey per game for Aaron Rodgers Jersey Authentic his career.

December 14, 2011 | Unregistered CommenterMike Wallace Jersey Outlet

Mike Wallace Jersey, New York Giants Jersey on Sale, Authentic New Orleans Saints Jersey, Victor Cruz Jersey on Sale, Jason Pierre-Paul Jersey, Authentic Drew Brees Jersey, Cheap Bears Jerseys On Sale, Jerod Mayo Jersey, Authentic Troy Polamalu Jersey, San Francisco 49ers Jersey outlet, Clay Matthews Jersey On Sale, Brian Westbrook Jersey, Eli Manning Jersey on Sale, Cheap Calvin Johnson Jersey, Authentic Ray Lewis Jersey, Jordy Nelson Jersey On Sale, Drew Brees Jersey, Victor Cruz Jersey, Joe Flacco Jersey?, Drew Brees Jersey, Brian Urlacher Jersey, Haloti Ngata Jersey, Aaron Rodgers Jersey On Sale, Drew Brees Jersey, Haloti Ngata Jersey?, Drew Brees Jersey, Jordy Nelson Jersey, Aaron Rodgers Jersey Authentic, Clay Matthews Jersey, Aaron Rodgers Jersey, Jimmy Graham Jersey On Sale, Jimmy Graham Jersey, Alex Smith Jersey Youth, Ravens Jerseys Authentic, Clay Matthews Jersey, Authentic Joe Flacco Jersey, Authentic Detroit Lions Jersey, Authentic Joe Flacco Jersey, Authentic Rob Gronkowski Jersey, Authentic Tom Brady Jersey, Cheap Brian Urlacher Jersey, Alex Smith Jersey, Official Baltimore Ravens Jerseys, Troy Polamalu Jersey, Authentic Brandon Marshall Jersey, Authentic Ray Lewis Jersey, Cheap Drew Brees Jersey, Ndamukong Suh Jersey, Patrick Willis Jersey Youth, Jimmy Graham Jersey, Tim Tebow Jersey, Matthew Stafford Jersey, Rob Gronkowski Jersey, Ray Lewis Jersey, Cheap Ndamukong Suh Jersey, Jordy Nelson Jersey Authentic, Authentic Pittsburgh Steelers Jersey, Cheap Mike Wallace Jersey, Brandon Marshall Jersey, Mike Wallace Jersey

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>