What should you look for when trying to get value from a goalkeeper in Major League Soccer?
If you’re a Vancouver Whitecaps fan, you’ll know about the Joe Cannon vs. Brad Knighton controversy. It follows the Joe Cannon vs. Jay Nolly controversy from two years ago; I guess we Vancouverites just like arguing over goalkeepers.
I don’t intend to enter into Knighton-v.-Cannon today, except indirectly. What I do mean to do is take a look at the third option. There’s a sizable chunk of fans who argue that neither Knighton nor Cannon are good enough and the Whitecaps should invest in a first-tier goalkeeper. It is granted that neither Knighton nor Cannon are at the top of the MLS class, so should the Whitecaps spend assets on someone who is? I don’t know that there has been any research into what the best MLS goalkeepers are worth, year over year, compared to the average or the lower end. Luckily, I’m here.
Those of you familiar with my earlier look into MLS player shooting percentages will recognize this format. I took any goalkeeper who, in one of the past five MLS seasons (2008 – 2012), played at least 720 minutes. 720 minutes is eight matches and is the point at which I think we can start calling a goalkeeper “regular”. That goalkeeper’s statistics for the past five years were put into a table, which I have reproduced below. Click the headings in the table to sort by that value.
This article has one very large table and two graphs; it therefore comes after the jump.
On anything but the widest of screens, this table will look like an explosion of numbers that will be completely unreadable. Therefore I also provide the table in convenient separate HTML file format for somewhat easier use on other screens.
Important Note: MLS’s stats department cannot do arithmetic. For the large majority of goalkeepers listed, the stats you’ll see does not reflect what you’ll see on MLS’s goalkeeper stats pages (the ones accessed through the Statistics menu on their site rather than the player profiles). In these cases (goals against + saves) does not equal (shots against), even though that should be impossible by definition. It doesn’t look like they’re erratically counting penalties, defender blocks, or own goals: some of the differences are too high, some of them are too low, and there is no obvious pattern. For example, in 2012 Sean Johnson is credited with 38 goals against, 108 saves, and 143 shots faced while 38 + 108 = 146; meanwhile, Donovan Ricketts gets 52 goals against, 87 saves, and 143 shots faced while 52 + 87 = 139. Basically, MLS is high on needle drugs. I’ve personally held stats sheets where the number of shots didn’t match the number of saves and moaned about it on Twitter; maybe I shouldn’t be surprised.
My statistics take goals against (because they’re too obvious to get wrong) and saves (because I had to pick something, and that is consistent with the statistics in MLS player profiles as well as how they calculate save percentages), then use them to derive all other stats. This is unlikely to be perfectly accurate; assume a margin of error for all values. I could go through every MLS game for the past five years to compile an accurate account, or I could slit my throat with a dull rock.
In addition, bear in mind that all this data is manually pulled from the MLS website, manually entered into a spreadsheet, then manually entered again into this table. I’ve done my best checking but typos are probable. Please submit all corrections via e-mail, Twitter, or in the comments.
I list goals-against average for completeness’s sake, since many of us are used to it. But it’s a useless statistic to evaluate goalkeepers, as it directly measures the quality of a defense. If two goalkeepers have a .800 save percentage but one faces five shots a match and one faces ten, the latter will have double the goals-against average through no fault of his own. So I give you the GAAs in case you want them; please don’t want them. We’re looking at save percentage.
In addition, we will primarily assess the top 20 goalkeepers by minutes played. This is because this provides us with a larger sample size over more than one season and therefore a better estimation of a goalkeeper’s true skill. We can count on those goalkeepers having been regular starters with over 5,000 minutes each in our five seasons; we can draw a few conclusions about them without worrying they’re a flash in the pan.
You will see the gap between single-season performances and average performances is very large; in short, as with shooting percentages, goalkeeper save percentages regress to the mean. As always, I have a graph to illustrate what I mean.
This graph shows the top 20 goalkeepers in MLS over the past five years by minutes played. Their save percentages are shown, year by year, in the first five dots. The final dot is their save percentage over the five seasons, and as you can see the average save percentages are much more grouped-together than the most extreme single season save percentages. Anyone who took Matt Reis’s 0.832 save percentage in 2009 (in 2,160 minutes!) to be his true skill level would be disappointed; then again, so would someone who took Reis’s 0.671 the next year. This is bad news for you Michael Gspurning boosters.
As we can see, there’s still a real difference between top-tier and the worst goalkeepers over our five seasons. The best top-twenty-minutes goalkeeper over our five years was Bouna Condoul with 0.760 in 5,715 minutes; the worst was Pat Onstad with 0.689 in 7,138 minutes. Even Condoul’s impressive number was achieved in less than two full seasons of work, and by missing most of 2011 and all of 2012 he avoided a 2.5% increase in league shooting percentage; the fact that he has the 17th-most minutes of any goalkeeper in MLS for the past five seasons should probably tell you how much turnover there is at that position. Meanwhile Onstad, as well as third-worst-man Zach Thornton (0.693), was completely washed up. In 2008 and 2009 Onstad’s save percentage was excellent; he was then victimized by a 2010 which may have been the worst season for any starter in our five-year sample and didn’t improve in his 2011 post-retirement cameo.
There is a catch. This seemed like a good chance to answer another question I’ve always had: is there a correlation between facing more shots and having a higher save percentage? Will someone like Stefan Frei post a higher percentage than he otherwise would just because he’s facing so much rubber? Here is a chart with our top 20 goalkeepers sorted by total save percentage over the five years, along with the number of shots these goalkeepers faced per 90 minutes.
There is a visible, although not an extremely strong, correlation between shot rate and save percentage increasing.
EDIT, April 10, 7:30 AM: Thanks to commenter RicardoB, below, for suggesting I get the R² value for the correlation between SoG/90 and save percentage. In the case of our top 20 goalkeepers R² = 0.062.
FURTHER EDIT, April 10, 5:00 PM: Additional thanks to Colby Cosh via Twitter for pointing out the original R² value of 0.133 I gave is incorrect. I drew it from a third-party Google Docs template, but have since re-checked the result with Excel and gotten the result Cosh said I should get. Oops.
Bear this in mind when evaluating goalkeepers behind crappy or brilliant defenses; a goalkeeper will generally be flattered a bit if he has no help from his defenders, and he’ll be a bit underrated if he’s behind an excellent defensive team. In case you’re wondering, the poor bastard with a midrange save percentage but the highest SoG/90 of all is Stefan Frei, whereas the two happy stiffs who were behind very good defenses (fewer than four SoG/90) but also in the top five for save percentage were Kasey Keller and Nick Rimando; pretty much the guys you’d think.
While the gaps between average save percentages is relatively small, it’s still significant. Last season, the 19 teams in MLS conceded an average of 150.84 shots on target. If your starting goalkeeper played all 3,060 minutes, the difference between Jon Busch’s 0.745 and Pat Onstad’s 0.686 would be 8.9 goals per season. That’s very considerable; there’s probably no simpler way to add nine goals to your goal differential than upgrading Onstad to Busch.
On the other hand, most goalkeepers aren’t as weak as latter-day Onstad or as strong as Busch. The cumulative save percentage of our twenty goalkeepers is 0.720 with a standard deviation of 0.019. So Matt Reis (0.739) is a standard deviation above the cumulative percentage and Matt Pickens (0.702) is approximately a standard deviation below. The difference between Reis and Pickens is 5.6 goals per season: still enough to pay attention, not enough to change a season.
From a Whitecaps perspective this would be very good news for Brad Knighton’s 0.748 if he’d played enough minutes for me to trust him, and bad news for Joe Cannon’s 0.707, especially given that he has two consecutive seasons under 0.700. Anybody below 0.700 is solidly in the lower rank of MLS starters and should be replaced. Most of the 0.710 to 0.720 options are affordable players; indeed, what stands out looking at this list is that you can get quite reasonable goalkeepers who aren’t really heralded. Will Hesmer hasn’t had a team since 2012, Josh Saunders and Kevin Hartman were both available if you whistled, and if you’re willing to take a chance on players with 2,000 or so minutes you have even more options. Ray Burse is doing very well in the NASL, for example. Not every second-division keeper works out; look at the great Bill Gaudette. But he hardly got any minutes and came to MLS off a serious head injury.
Goalkeepers like Reis, Busch, and Rimando tend to cost between $150,000 and $200,000 on the latest salary list. But if you go down to the likes of Hamid ($84,750) or Saunders ($77,678), there are savings to be had. Meanwhile, teams pay big bank for the likes of Kennedy ($175,000), Pickens ($181,036), or, heaven help us, Cannon ($175,666). There are plenty of teams paying top dollar for replacement-level or sub-replacement-level goalkeeping. Why do it? Not every second-division pickup is going to work out, but sensibly picking an established top NASL goalkeeper gives you a better chance of value for money than $150,000 for someone who’s proven they’re not good enough.
High-priced international acquisitions don’t guarantee anything. Michael Gspurning, half a season into his MLS career, is top of the charts; we’ll see where he ends up. If you count Kasey Keller as an international he was a winner. Jimmy Nielsen is in fair-starter territory and comes with a $220,000 price tag and an international slot. The rest of the internationals… well. Faryd Mondragon was an expensive Jay Nolly, Designated Player Frank Rost washed out, even latter-career Dario Sala was a waste of money.
So your best bet? Unless you can get an established top keeper with a few years left like Busch or Rimando, or an old bargain with something left like Hartman might be, or young quality like Hamid, go for value. Sign more Brad Knightons.
 — All statistics from: Major League Soccer. “Statistics – Goalkeeping.” Accessed April 8, 2013. http://www.mlssoccer.com/stats/season?season_year=2012&season_type=REG&team=ALL&group=GOALKEEPING&op=Search&form_id=mls_stats_individual_form.
 — Massey, Benjamin. “Repeatability of Shooting Percentage in MLS.” Maple Leaf Forever!, November 26, 2012. Accessed April 9, 2013. http://www.maple-leaf-forever.com/2012/11/26/repeatability-of-shooting-percentage-in-mls/.
 — Major League Soccer Players Union. “2012 MLS Player Salaries.” October 1, 2012. Accessed April 9, 2013. http://www.mlsplayers.org/files/October%201,%202012%20Salary%20Information%20-%20Alphabetical.pdf.