What’s a Goalkeeper in MLS Worth?

By Benjamin Massey

April 9th, 2013 · 3 comments

What should you look for when trying to get value from a goalkeeper in Major League Soccer?

If you’re a Vancouver Whitecaps fan, you’ll know about the Joe Cannon vs. Brad Knighton controversy. It follows the Joe Cannon vs. Jay Nolly controversy from two years ago; I guess we Vancouverites just like arguing over goalkeepers.

I don’t intend to enter into Knighton-v.-Cannon today, except indirectly. What I do mean to do is take a look at the third option. There’s a sizable chunk of fans who argue that neither Knighton nor Cannon are good enough and the Whitecaps should invest in a first-tier goalkeeper. It is granted that neither Knighton nor Cannon are at the top of the MLS class, so should the Whitecaps spend assets on someone who is? I don’t know that there has been any research into what the best MLS goalkeepers are worth, year over year, compared to the average or the lower end. Luckily, I’m here.

Those of you familiar with my earlier look into MLS player shooting percentages will recognize this format. I took any goalkeeper who, in one of the past five MLS seasons (2008 – 2012), played at least 720 minutes. 720 minutes is eight matches and is the point at which I think we can start calling a goalkeeper “regular”. That goalkeeper’s statistics for the past five years were put into a table, which I have reproduced below. Click the headings in the table to sort by that value.

This article has one very large table and two graphs; it therefore comes after the jump.

On anything but the widest of screens, this table will look like an explosion of numbers that will be completely unreadable. Therefore I also provide the table in convenient separate HTML file format for somewhat easier use on other screens.

Important Note: MLS’s stats department cannot do arithmetic. For the large majority of goalkeepers listed, the stats you’ll see does not reflect what you’ll see on MLS’s goalkeeper stats pages (the ones accessed through the Statistics menu on their site rather than the player profiles). In these cases (goals against + saves) does not equal (shots against), even though that should be impossible by definition. It doesn’t look like they’re erratically counting penalties, defender blocks, or own goals: some of the differences are too high, some of them are too low, and there is no obvious pattern. For example, in 2012 Sean Johnson is credited with 38 goals against, 108 saves, and 143 shots faced while 38 + 108 = 146; meanwhile, Donovan Ricketts gets 52 goals against, 87 saves, and 143 shots faced while 52 + 87 = 139. Basically, MLS is high on needle drugs. I’ve personally held stats sheets where the number of shots didn’t match the number of saves and moaned about it on Twitter; maybe I shouldn’t be surprised.

My statistics take goals against (because they’re too obvious to get wrong) and saves (because I had to pick something, and that is consistent with the statistics in MLS player profiles as well as how they calculate save percentages), then use them to derive all other stats. This is unlikely to be perfectly accurate; assume a margin of error for all values. I could go through every MLS game for the past five years to compile an accurate account, or I could slit my throat with a dull rock.

In addition, bear in mind that all this data is manually pulled from the MLS website, manually entered into a spreadsheet, then manually entered again into this table. I’ve done my best checking but typos are probable. Please submit all corrections via e-mail, Twitter, or in the comments[1].

2008 2009 2010 2011 2012 Cumulative
Name Min SoGA GA Sv Sv% GAA Min SoGA GA Sv Sv% GAA Min SoGA GA Sv Sv% GAA Min SoGA GA Sv Sv% GAA Min SoGA GA Sv Sv% GAA Min SoGA GA Sv Sv% GAA
Burpo, Preston 1170 61 24 37 0.607 1.85 990 39 16 23 0.590 1.45 939 64 16 48 0.750 1.53 0 0 0 0 nan nan 0 0 0 0 nan nan 3099 164 56 108 0.659 1.63
Burse, Ray 180 16 4 12 0.750 2.00 1503 89 27 62 0.697 1.62 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 1683 105 31 74 0.705 1.66
Busch, Jon 2700 155 33 122 0.787 1.10 2700 137 34 103 0.752 1.13 1620 100 19 81 0.810 1.06 2970 157 44 113 0.720 1.33 2835 117 40 77 0.658 1.27 12825 666 170 496 0.745 1.19
Cannon, Joe 2700 162 38 124 0.765 1.27 2520 144 47 97 0.674 1.68 1080 52 14 38 0.731 1.17 1800 98 32 66 0.673 1.60 2277 107 34 73 0.682 1.34 10377 563 165 398 0.707 1.43
Cepero, Danny 180 11 6 5 0.455 3.00 1088 83 21 62 0.747 1.74 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 1268 94 27 67 0.713 1.92
Conway, Jon 2520 140 42 98 0.700 1.50 810 48 15 33 0.688 1.67 72 3 1 2 0.667 1.25 540 23 8 15 0.652 1.33 0 0 0 0 nan nan 3942 214 66 148 0.692 1.51
Coundoul, Bouna 1530 75 21 54 0.720 1.24 720 42 10 32 0.762 1.25 2430 133 28 105 0.789 1.04 1035 50 13 37 0.740 1.13 0 0 0 0 nan nan 5715 300 72 228 0.760 1.13
Crayton, Lewis 1080 62 19 43 0.694 1.58 540 29 8 21 0.724 1.33 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 1620 91 27 64 0.703 1.50
Cronin, Steve 1935 136 44 92 0.676 2.05 180 11 2 9 0.818 1.00 0 0 0 0 nan nan 126 10 4 6 0.600 2.86 0 0 0 0 nan nan 2241 157 50 107 0.682 2.01
Dykstra, Andrew 0 0 0 0 nan nan 0 0 0 0 nan nan 1530 76 21 55 0.724 1.24 0 0 0 0 nan nan 0 0 0 0 nan nan 1530 76 21 55 0.724 1.24
Frei, Stefan 0 0 0 0 nan nan 2295 136 38 98 0.721 1.49 2520 145 37 108 0.745 1.32 2406 155 49 106 0.684 1.83 0 0 0 0 nan nan 7221 436 124 312 0.716 1.55
Gaudette, Bill 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 1350 58 20 38 0.655 1.33 1350 58 20 38 0.655 1.33
Gruenebaum, Andy 90 5 3 2 0.400 3.00 900 33 12 21 0.636 1.20 0 0 0 0 nan nan 180 5 0 5 1.000 0.00 2920 165 41 124 0.752 1.26 4090 208 56 152 0.731 1.23
Gspurning, Michael 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 1845 74 15 59 0.797 0.73 1845 74 15 59 0.797 0.73
Guzan, Brad 1530 68 20 48 0.706 1.18 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 1530 68 20 48 0.706 1.18
Hall, Tally 0 0 0 0 nan nan 0 0 0 0 nan nan 450 26 8 18 0.692 1.60 3060 140 41 99 0.707 1.21 2946 127 39 88 0.693 1.19 6456 293 88 205 0.700 1.23
Hamid, Bill 0 0 0 0 nan nan 0 0 0 0 nan nan 720 30 10 20 0.667 1.25 2392 114 37 77 0.675 1.39 2087 112 24 88 0.786 1.03 5199 256 71 185 0.723 1.23
Hartman, Kevin 2700 156 39 117 0.750 1.30 2700 147 42 105 0.714 1.40 1755 68 12 56 0.824 0.62 2970 120 35 85 0.708 1.06 2700 140 42 98 0.700 1.40 12825 631 170 461 0.731 1.19
Hesmer, Will 2610 130 33 97 0.746 1.14 1710 86 18 68 0.791 0.95 2880 117 33 84 0.718 1.03 2880 136 44 92 0.676 1.38 0 0 0 0 nan nan 10080 469 128 341 0.727 1.14
Johnson, Sean 0 0 0 0 nan nan 0 0 0 0 nan nan 1170 69 17 52 0.754 1.31 2520 113 37 76 0.673 1.32 2766 146 38 108 0.740 1.24 6456 328 92 236 0.720 1.28
Keller, Kasey 0 0 0 0 nan nan 2549 104 26 78 0.750 0.92 2655 116 34 82 0.707 1.15 3060 147 37 110 0.748 1.09 0 0 0 0 nan nan 8264 367 97 270 0.736 1.06
Kennedy, Dan 699 31 8 23 0.742 1.03 0 0 0 0 nan nan 594 30 8 22 0.733 1.21 2880 139 39 100 0.719 1.22 2880 163 54 109 0.669 1.69 7053 363 109 254 0.700 1.39
Knighton, Brad 0 0 0 0 nan nan 540 50 14 36 0.720 2.33 652 31 8 23 0.742 1.10 0 0 0 0 nan nan 782 34 7 27 0.794 0.81 1974 115 29 86 0.748 1.32
Kocic, Milos 0 0 0 0 nan nan 308 17 8 9 0.529 2.34 107 8 3 5 0.625 2.52 654 45 10 35 0.778 1.38 2430 129 47 82 0.636 1.74 3499 199 68 131 0.658 1.75
MacMath, Zac 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 675 24 8 16 0.667 1.07 2880 136 43 93 0.684 1.34 3555 160 51 109 0.681 1.29
Meara, Ryan 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 1620 86 27 59 0.686 1.50 1620 86 27 59 0.686 1.50
Meredith, Bryan 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 990 34 14 20 0.588 1.27 990 34 14 20 0.588 1.27
Mondragon, Faryd 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 2385 77 28 49 0.636 1.06 0 0 0 0 nan nan 2385 77 28 49 0.636 1.06
Nielsen, Jimmy 0 0 0 0 nan nan 0 0 0 0 nan nan 2610 114 34 80 0.702 1.17 2767 114 35 79 0.693 1.14 3060 104 27 77 0.740 0.79 8437 332 96 236 0.711 1.02
Nolly, Jay 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 1260 62 23 39 0.629 1.64 0 0 0 0 nan nan 1260 62 23 39 0.629 1.64
Onstad, Pat 2098 100 24 76 0.760 1.03 2700 118 29 89 0.754 0.97 2070 89 40 49 0.551 1.74 270 15 7 8 0.533 2.33 0 0 0 0 nan nan 7138 322 100 222 0.689 1.26
Perkins, Troy 0 0 0 0 nan nan 0 0 0 0 nan nan 1980 109 37 72 0.661 1.68 2610 129 38 91 0.705 1.31 2767 144 43 101 0.701 1.40 7357 382 118 264 0.691 1.44
Pickens, Matt 2430 133 31 102 0.767 1.15 1710 61 22 39 0.639 1.16 2610 120 32 88 0.733 1.10 3060 118 41 77 0.653 1.21 2859 156 49 107 0.686 1.54 12669 588 175 413 0.702 1.24
Reis, Matt 2485 145 38 107 0.738 1.38 2160 137 23 114 0.832 0.96 1260 76 25 51 0.671 1.79 2430 154 43 111 0.721 1.59 2430 131 39 92 0.702 1.44 10765 643 168 475 0.739 1.40
Ricketts, Donovan 0 0 0 0 nan nan 2274 108 26 82 0.759 1.03 2610 105 26 79 0.752 0.90 1284 52 11 41 0.788 0.77 2913 139 52 87 0.626 1.61 9081 404 115 289 0.715 1.14
Rimando, Nick 2700 135 39 96 0.711 1.30 2285 102 29 73 0.716 1.14 2430 96 18 78 0.813 0.67 2970 131 36 95 0.725 1.09 2790 116 33 83 0.716 1.06 13175 580 155 425 0.733 1.06
Rost, Frank 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 945 40 14 26 0.650 1.33 0 0 0 0 nan nan 945 40 14 26 0.650 1.33
Sala, Dario 2520 129 37 92 0.713 1.32 1195 63 20 43 0.683 1.51 945 50 16 34 0.680 1.52 0 0 0 0 nan nan 0 0 0 0 nan nan 4660 242 73 169 0.698 1.41
Saunders, Josh 270 14 6 8 0.571 2.00 426 17 5 12 0.706 1.06 90 1 0 1 1.000 0.00 1639 72 17 55 0.764 0.93 2430 116 36 80 0.690 1.33 4855 220 64 156 0.709 1.19
Seitz, Chris 0 0 0 0 nan nan 360 18 5 13 0.722 1.25 2047 109 41 68 0.624 1.80 90 9 4 5 0.556 4.00 360 15 5 10 0.667 1.25 2857 151 55 96 0.636 1.73
Sutton, Greg 2160 151 35 116 0.768 1.46 135 5 2 3 0.600 1.33 270 10 1 9 0.900 0.33 900 45 15 30 0.667 1.50 24 1 1 0 0.000 3.75 3489 212 54 158 0.745 1.39
Thornton, Zach 651 29 13 16 0.552 1.80 2385 106 23 83 0.783 0.87 2015 96 33 63 0.656 1.47 180 7 4 3 0.429 2.00 0 0 0 0 nan nan 5231 238 73 165 0.693 1.26
Wells, Zach 1530 84 28 56 0.667 1.65 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 1530 84 28 56 0.667 1.65
Wicks, Josh 495 104 26 78 0.750 4.73 1672 104 26 78 0.750 1.40 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 2167 208 52 156 0.750 2.16
Willis, Joe 0 0 0 0 nan nan 0 0 0 0 nan nan 0 0 0 0 nan nan 270 16 4 12 0.750 1.33 972 50 19 31 0.620 1.76 1242 66 23 43 0.652 1.67

I list goals-against average for completeness’s sake, since many of us are used to it. But it’s a useless statistic to evaluate goalkeepers, as it directly measures the quality of a defense. If two goalkeepers have a .800 save percentage but one faces five shots a match and one faces ten, the latter will have double the goals-against average through no fault of his own. So I give you the GAAs in case you want them; please don’t want them. We’re looking at save percentage.

In addition, we will primarily assess the top 20 goalkeepers by minutes played. This is because this provides us with a larger sample size over more than one season and therefore a better estimation of a goalkeeper’s true skill. We can count on those goalkeepers having been regular starters with over 5,000 minutes each in our five seasons; we can draw a few conclusions about them without worrying they’re a flash in the pan.

You will see the gap between single-season performances and average performances is very large; in short, as with shooting percentages, goalkeeper save percentages regress to the mean. As always, I have a graph to illustrate what I mean.

top20minutessavepercentaegs

This graph shows the top 20 goalkeepers in MLS over the past five years by minutes played. Their save percentages are shown, year by year, in the first five dots. The final dot is their save percentage over the five seasons, and as you can see the average save percentages are much more grouped-together than the most extreme single season save percentages. Anyone who took Matt Reis’s 0.832 save percentage in 2009 (in 2,160 minutes!) to be his true skill level would be disappointed; then again, so would someone who took Reis’s 0.671 the next year. This is bad news for you Michael Gspurning boosters.

As we can see, there’s still a real difference between top-tier and the worst goalkeepers over our five seasons. The best top-twenty-minutes goalkeeper over our five years was Bouna Condoul with 0.760 in 5,715 minutes; the worst was Pat Onstad with 0.689 in 7,138 minutes. Even Condoul’s impressive number was achieved in less than two full seasons of work, and by missing most of 2011 and all of 2012 he avoided a 2.5% increase in league shooting percentage[2]; the fact that he has the 17th-most minutes of any goalkeeper in MLS for the past five seasons should probably tell you how much turnover there is at that position. Meanwhile Onstad, as well as third-worst-man Zach Thornton (0.693), was completely washed up. In 2008 and 2009 Onstad’s save percentage was excellent; he was then victimized by a 2010 which may have been the worst season for any starter in our five-year sample and didn’t improve in his 2011 post-retirement cameo.

There is a catch. This seemed like a good chance to answer another question I’ve always had: is there a correlation between facing more shots and having a higher save percentage? Will someone like Stefan Frei post a higher percentage than he otherwise would just because he’s facing so much rubber? Here is a chart with our top 20 goalkeepers sorted by total save percentage over the five years, along with the number of shots these goalkeepers faced per 90 minutes.

EDIT: April 10, 9:40 AM: Thanks to Colby Cosh via Twitter for suggesting an improved way to graph this data. If you need it, the old version is here.

svpercentagesog90correlation2

There is a visible, although not an extremely strong, correlation between shot rate and save percentage increasing.

EDIT, April 10, 7:30 AM: Thanks to commenter RicardoB, below, for suggesting I get the R² value for the correlation between SoG/90 and save percentage. In the case of our top 20 goalkeepers R² = 0.062.

FURTHER EDIT, April 10, 5:00 PM: Additional thanks to Colby Cosh via Twitter for pointing out the original R² value of 0.133 I gave is incorrect. I drew it from a third-party Google Docs template, but have since re-checked the result with Excel and gotten the result Cosh said I should get. Oops.

Bear this in mind when evaluating goalkeepers behind crappy or brilliant defenses; a goalkeeper will generally be flattered a bit if he has no help from his defenders, and he’ll be a bit underrated if he’s behind an excellent defensive team. In case you’re wondering, the poor bastard with a midrange save percentage but the highest SoG/90 of all is Stefan Frei, whereas the two happy stiffs who were behind very good defenses (fewer than four SoG/90) but also in the top five for save percentage were Kasey Keller and Nick Rimando; pretty much the guys you’d think.

While the gaps between average save percentages is relatively small, it’s still significant. Last season, the 19 teams in MLS conceded an average of 150.84 shots on target. If your starting goalkeeper played all 3,060 minutes, the difference between Jon Busch’s 0.745 and Pat Onstad’s 0.686 would be 8.9 goals per season. That’s very considerable; there’s probably no simpler way to add nine goals to your goal differential than upgrading Onstad to Busch.

On the other hand, most goalkeepers aren’t as weak as latter-day Onstad or as strong as Busch. The cumulative save percentage of our twenty goalkeepers is 0.720 with a standard deviation of 0.019. So Matt Reis (0.739) is a standard deviation above the cumulative percentage and Matt Pickens (0.702) is approximately a standard deviation below. The difference between Reis and Pickens is 5.6 goals per season: still enough to pay attention, not enough to change a season.

From a Whitecaps perspective this would be very good news for Brad Knighton’s 0.748 if he’d played enough minutes for me to trust him, and bad news for Joe Cannon’s 0.707, especially given that he has two consecutive seasons under 0.700. Anybody below 0.700 is solidly in the lower rank of MLS starters and should be replaced. Most of the 0.710 to 0.720 options are affordable players; indeed, what stands out looking at this list is that you can get quite reasonable goalkeepers who aren’t really heralded. Will Hesmer hasn’t had a team since 2012, Josh Saunders and Kevin Hartman were both available if you whistled, and if you’re willing to take a chance on players with 2,000 or so minutes you have even more options. Ray Burse is doing very well in the NASL, for example. Not every second-division keeper works out; look at the great Bill Gaudette. But he hardly got any minutes and came to MLS off a serious head injury.

Goalkeepers like Reis, Busch, and Rimando tend to cost between $150,000 and $200,000 on the latest salary list[3]. But if you go down to the likes of Hamid ($84,750) or Saunders ($77,678), there are savings to be had. Meanwhile, teams pay big bank for the likes of Kennedy ($175,000), Pickens ($181,036), or, heaven help us, Cannon ($175,666). There are plenty of teams paying top dollar for replacement-level or sub-replacement-level goalkeeping. Why do it? Not every second-division pickup is going to work out, but sensibly picking an established top NASL goalkeeper gives you a better chance of value for money than $150,000 for someone who’s proven they’re not good enough.

High-priced international acquisitions don’t guarantee anything. Michael Gspurning, half a season into his MLS career, is top of the charts; we’ll see where he ends up. If you count Kasey Keller as an international he was a winner. Jimmy Nielsen is in fair-starter territory and comes with a $220,000 price tag and an international slot. The rest of the internationals… well. Faryd Mondragon was an expensive Jay Nolly, Designated Player Frank Rost washed out, even latter-career Dario Sala was a waste of money.

So your best bet? Unless you can get an established top keeper with a few years left like Busch or Rimando, or an old bargain with something left like Hartman might be, or young quality like Hamid, go for value. Sign more Brad Knightons.


[1] — All statistics from: Major League Soccer. “Statistics – Goalkeeping.” Accessed April 8, 2013. http://www.mlssoccer.com/stats/season?season_year=2012&season_type=REG&team=ALL&group=GOALKEEPING&op=Search&form_id=mls_stats_individual_form.

[2] — Massey, Benjamin. “Repeatability of Shooting Percentage in MLS.” Maple Leaf Forever!, November 26, 2012. Accessed April 9, 2013. http://www.maple-leaf-forever.com/2012/11/26/repeatability-of-shooting-percentage-in-mls/.

[3] — Major League Soccer Players Union. “2012 MLS Player Salaries.” October 1, 2012. Accessed April 9, 2013. http://www.mlsplayers.org/files/October%201,%202012%20Salary%20Information%20-%20Alphabetical.pdf.

Comments are closed.

3 responses to “What’s a Goalkeeper in MLS Worth?”

  1. RicardoB says:

    R^2 for SoG/90 v save percentage rank? Seriously, that is literally the definition of correlation. And Excel will just pop it out.

    • Benjamin Massey says:

      Thanks! I’ve added the R² to the article (0.133, to save you the searching). I am not a statistics expert and eagerly welcome correction from those who know what they’re doing.

      • RicardoB says:

        No worries! R^2 loosely illustrates what percentage of variance in Y is explained by X. In this case ~13% of a keeper’s save percentage is explained by SoG/90min, so the rest is a result of other things (quality of shot, quality of keeper, etc).