Most DVD-by-Mail Comparisons Are Flawed

Almost every week, some well-intentioned but clueless journalist releases the obligatory DVD-by-mail comparison story. The story is generally titled something like "Customers Win in War of Online DVD Rental Firms", “Who’s Winning the DVD War?”, "In the Competition for DVD Rentals by Mail, Two Empires Strike Back", “Netflix vs. Blockbuster”, etc.

The titles of the articles don’t matter, because the stories are all the same. The brilliantly original, hard-hitting investigative reporter always signs up with Blockbuster Online and Netflix and tracks the turnaround of DVDs for one month. The reporter then compares the numbers and draws conclusions about who is better. The results are almost always like this:

  • Netflix has a better Web site
  • Netflix processes DVDs a little faster
  • Netflix carries NC-17 titles
  • Blockbuster Online has more new releases
  • Blockbuster Online processes DVDs on weekends
  • Blockbuster Online provides free coupons for in-store rentals

The comparison stories always seem to wrap things up so cleanly and finally get to the bottom of the issue about who is better in the DVD-by-mail industry.

The problem, however, is that anyone who has tested Netflix and at least one other rental company knows the methodology used for most of these investigations is flawed. Anyone who has tried renting a lot of DVDs from Netflix for more than a few weeks in a row knows that Netflix begins applying throttling measures to limit the flow of DVDs after as little as one month of heavy use. Since Netflix throttles users, no comparison story is complete without explaining this concept.

The failure to address throttling is the major flaw in most of the comparison stories; however, the stories often have other major errors and glaring omissions that make the results useless. Please read these comparison stories with a critical eye, and keep in mind that no DVD-by-mail comparison story is complete unless it meets the following criteria:

  1. The study covers a single time period lasting at least three continuous months.
  2. The investigator took full advantage of each rental plan by returning DVDs as quickly as possible.
  3. The investigator lives near distribution centers for all companies studied.
  4. The investigator rents DVDs that are representative of the readers’ viewing habits. (For example, an investigation conducted for a mainstream newspaper should include a sample of DVDs that represents mainstream viewing tastes.)
  5. The investigator details the fine points of each rental plan (e.g., throttling, plan limits, poor inventory quality, number of shipping days per week, etc.)

DVD-by-mail comparison stories have become a popular diversion for lazy journalists and provide the readers with misleading or useless information. As readers, we need to be demanding more. When you see these incomplete comparison stories, consider contacting the authors to let them know what they missed. Hopefully, journalists will start conducting some thorough investigations of these companies, and we can finally reach some real conclusions about where consumers can get the best value.


melgru23 said...

You are so right on this subject. I have used both Netflix and Blockbuster. Netflix definitely slows down the shipping of DVD about 1-2 months into the agreement. I would say that the numbers drop by 50%.
Blockbuster on the other hand is truly incompetent and will purposely stop you from canceling your subscription when you request. They will bill your credit card for "lost" DVD's after you cancel your subscriptions. Make sure if you have Blockbuster to save the notifications that your DVD's were received. Also, all of the Blockbuster stores in our region are privately owned and refuse to take Online DVD returns.

Overall NetFlix is better but I am going to write Reed Hasting, CEO from Netflix about the problem and continue to support you in your action.

Anonymous said...

Disagree. I've had Netflix for years and my next DVD is almost always 2 days away--- one day sending, one day receiving. If they were to apply throttling measures, I would have seen it.

Anonymous said...

I have used Netflix for several years and have never had a problem with 'slow-down' I alway get the DVDs usually 2 days after I return. Never had a complaint about this, although I do feel that they are getting slow with sending out newer releases.

Anonymous said...

"Anonymous" Netflix employees respond:

Anonymous said...

Beside maybe the hearing impaired (as mentioned somewhere else on the site) why would anyone rent DVD by mail? Sure Netflix's inventory for DVD by mail is larger, but you can almost certainly find those same movies you must have on one of the many other online movie streaming services. Just google Hulu, Vudu, or if you're the more avant garde film enthusiast try indiefilx or fandor.