I saw this recent list published by PrivCo today, and I couldn’t stop myself from writing a quick post.

It’s actually a teaser report for readers to go purchase the full report and underlying data. I have to say, DO NOT BUY this report. Even if the data is accurate, the marketing is so misleading I think one should avoid it out of sheer principle.  The report is here.

The title of the report is “Top 20 Tech Venture Capital Firms of 2012”

The chart in the main body is titled “PrivCo’s Ranking of the Most Successful Venture Capital Firms (2012)”

The giveaway is in the subtitle below that: “ranked by total number of private tech co. exits”.

Hmmm… ok, this is wrong in so many ways.

No disrespect to the outstanding firms on the list.  Many of them are really excellent and deserve to be on anyone’s top 20 list.  But there are some very obvious omissions and clear mistakes from the list if you are judging based on 2012 exits.

First obvious problem – Facebook went public in 2012.  There is no way on earth any publication should be able to share a list of top firms in 2012 without putting the earliest backers of Facebook at the very very top. That would be Accel and Greylock, which are number 8 and 16 respectively.

Second obvious problem, I don’t see the underlying data, but there are some funds that had many many solid exits in 2012 that are not on this list.  Most puzzling to me is the omission of General Catalyst.  In 2012, they have IPOs for Demandware, Kayak and Brightcove.  That’s Billions of dollars in exits, and I imagine they must have had many other smaller exits as well. I’m puzzled why they wouldn’t make the list even under PrivCo’s metrics.  Consider also the omission of CRV (Yammer series A investors) and USV (Indeed series A investors).

The bigger issue is just that the headline and the actual data for performance are so skewed.  Main problems in a nutshell:

1. Ownership, cost basis, and preferences matters. You could own a lot of a company at a very low cost basis at exit, and do fantastically well.  You could own very little of a company at exit and not do nearly as well as it would seem.  You could own stock that is junior to other investors and actually lose money in a mediocre sale.  Not taking these things into account is like judging a basketball team by the number of dunks they have, not by their number of wins.

2. Exits in a given year is a misleading statistic, because it might mask the underlying funds underneath those investments.  If all the exits in 2012 come out of the same fund, then perhaps that is a great year.  But if a fund has 5 small exits, and all 5 are out of 4 different funds, well, that’s not very good at all.

3. Venture Capital performance tends to be about outlier, outsized returns that drive fund economics. The one great exit can be much more meaningful than dozens of mediocre exits.  Moreover, one great exit for a $1B fund might return a lot of money, but only return a relatively small percentage of the fund.  Whereas a more modest exit for a $75M fund could return the whole fund.  From the perspective of a $5M investor in those funds, the more modest exit is much much better than the big exit.

4. More should not be better.  You just can’t look at exits in the vacuum of an overall fund.  The theoretically best VC in the world makes only 1 investment in 1 successful company. Yes, based on number of exits, that fund would not look very good. Measuring based on volume of exits tends to favor funds that make lots and lots of investments, even if each investment is tiny and/or a tiny percentage of their funds.  The best funds tend to have relatively concentrated portfolios rather than a “spray and pray” approach. Now, this isn’t always true, but that’s why you need to factor in ownership, cost basis, and preferences.

Ultimately, I just think it’s misleading to put a headline like that on a report called “The Top 20 Venture Capital Firms” without taking these factors into full account.  And I’m honestly shocked that other sites with sophisticated writers republished the post.  Ugh.