Jordan vs. Olajuwon: Who's Better? Using a Scientific Method

5 February, 1996

A few weeks ago, ESPNet published an article by Allen Barra where he used a very simple formula to evaluate Michael Jordan and Hakeem Olajuwon. Using his version of a linear weights formula (his version is here), he proclaimed Olajuwon to be the better player. Not surprisingly, when ESPNet published responses to the article, those responding claimed Barra's results to be bunk. Using my professional judgment as a scout/coach, I would tend to agree with those letter writers. Using some technical tools to analyze the stats, I feel stronger in that conviction. I will demonstrate the use of these tools below.

My principal method for evaluating the overall effectiveness of players is to look at their individual win-loss records. I calculated these for Jordan and Olajuwon all the way back to the '86-87 season (before that, I haven't entered into the computer all the data I need):

	Jordan				Olajuwon
	W	L	%		W	L	%
'87	17.0	2.4	0.877		12.3	2.2	0.850
'88	19.4	0.7	0.967		11.9	2.1	0.850
'89	18.0	0.7	0.961		14.9	1.7	0.898
'90	18.0	1.1	0.942		14.2	1.8	0.888
'91	17.5	0.4	0.980		 9.3	0.5	0.945
'92	15.9	0.6	0.966		10.6	1.5	0.873
'93	17.0	0.8	0.953		17.3	0.8	0.957
'94					16.8	1.1	0.937
'95	 2.8	0.9	0.757		14.0	2.3	0.859
'96	 5.8    0.2     0.964		 5.4	1.1	0.834

Total  131.4	7.8	0.944	       126.7   15.1	0.894

(1996 season through games of 1/7/96) As is obvious, Jordan has consistently done slightly better than Olajuwon in winning percentage and overall wins. There is no doubt that both are among the top in the league. By being so good and such exceptions to the average, both players defy the use of standard statistical methods for analysis. This is just one reason (of many) why Barra's method didn't work. When players are this good, you have to get into the details and identify the small things that are important and exactly how they are important. That's what my methods do. And they consistently show that Jordan has been a better player than Olajuwon (and everyone else, for that matter).

(Late Note: Though I said that Jordan was only "slightly better" than Olajuwon above, the difference is actually a significant one. I have a method (from Bill James) for determining how often a team with one record beats a team with another record and it indicates that a team with a 0.944 winning percentage beats a team with a 0.894 winning percentage 66.7% of the time This is a rather significant edge and, though, there is no way of evaluating what this really means since we can't compare a team of Jordans to a team of Olajuwons, it indicates very strongly that Jordan contributes more to winning than Olajuwon, another great player.... For people in the statistics field, this represents one way of comparing probabilities close to one or zero, which can be useful in evaluating rare events, such as earthquakes, storms, or floods.)

Win-Loss Records Come From Offensive and Defensive Ratings

I want to now split up some of the above numbers to show that we are comparing very different players, but that we can do it consistently and logically to obtain the rational results shown above. The way the above win-loss records were found was by determining both the offensive and defensive ratings of both players. These are shown below:

	Jordan	Rtg	Olajuwon Rtg
	Off.	Def.	Off.	Def.
'87	116.5	103.4	106.4	95.8
'88	123.2	100.4	105.4	94.9
'89	122.5	100.8	104.2	91.4
'90	122.5	103.5	101.9	89.9
'91	126.0	 99.6	106.0	89.2
'92	121.8	 99.4	106.7	94.9
'93	120.2	100.2	111.2	92.2
'94			107.8	91.5
'95	107.6	100.4	107.5	96.4
'96	121.4	 99.4	102.9	93.3

Jordan has consistently been the better offensive player, which we know, while Olajuwon has been the better defensive player, which is debatable to some people. I would affirm, however, that big men are generally more valuable than guards because they occupy the middle of the defense and are more likely to be involved in stopping the opposition. My defensive method reflects this (and I've managed to convince Doug Steele of it, too). The way we put these two ratings together to obtain a winning percentage is through the Pythagorean Method, a method that accurately predicts winning percentages from offensive and defensive ratings for teams. (Of course, we are assuming that the same method applies for individuals, but that is a good first assumption.) Hence, with accurate offensive and defensive ratings, we have an accurate measure of winning percentage. The next step (going backward from how I actually calculate it) is how we get offensive and defensive ratings for players.

Offensive Ratings

Offensive contributions come through field goals, free throws, assists, and offensive rebounds. (Yes, yes, I know -- we are ignoring the little things, like picks and the pass right before an assist. I know their importance, but I know that importance is relatively small or else they would be official already.) An offensive rating is just an evaluation of how many points per 100 possessions a player produces. Hence, we must evaluate how a player produces points and how a player uses possessions. Field goals and free throws obviously produce points, but they also use a possession. Assists also produce points and use a possession. An offensive rebound keeps a possession alive, but doesn't directly lead to points.

An important rule I've held all my work to is that, when I sum all the individual points created and possessions used, I get the team values of points and possessions. With this in mind, we must do two things:

Devise a way to evaluate how the points produced on an assisted basket should be split up between assistant and assistee. Also devise how that possession should be split up.
Evaluate how offensive rebounds affect both a player's ability to indirectly "create points" and reduce his own number of possessions used.

My theory on weighting assists is based on two factors. First, the player shooting the ball should get at least as much, if not more credit than the assistant because making the shot is only a 50% proposition (normally), while passing is safe at a much higher rate. Second, when a potential assist is not converted, only the person missing the shot looks bad, whereas the potential assistant has no statistical record (though I've actually taken some). What my method comes down to is weighting assists by the chance that the shot has for going in. In other words, a pass to a wide open Shaquille O'Neal under the basket is worth half of the two points that are guaranteed to be scored because both the pass and the shot are 100% sure to happen. However, a pass to O'Neal at the three point line that he throws up at the end of a quarter and happens to go in the basket is worth less to the assistant because Shaq did the more difficult, or less likely, action. Since there are no numbers on this sort of thing, I have had to take some myself. I have roughly found that players don't shoot much better on average off a potential assist than they do otherwise. Because of this, I have defined the assist figure, or how much to weight the assist, to be 0.25, or about half of 50%, a benchmark field goal percentage.

This then gives the first part of the formula for estimating an individual's scoring possessions:

Scoring Poss.=FG+ASTFIG*AST-ASTFIG*FG*(TMAST/TMMIN*5*Min-AST)
                                      -----------------------
                                      (TMFG/TMMIN*5*Min-AST)

First, we give credit for all field goals (FG), then for all assists (ASTFIG*AST), then we take away credit for the baskets a player scored that were assisted on (the final part of the formula). That final part of the formula estimates how many baskets were assisted on by determining roughly how many field goals were made while the player was in that he didn't assist on and how many assists by other people while the player was in. It's a decent approximation, though it has a slight bias that I won't discuss here.

We can add free throws into scoring possessions easily because no assists are credited on most free throws. Using the well established approximation (in this not-so-well-established field) that about four-tenths of all free throws end possessions and finding out how many of these possessions are scoring possessions, we add to the above formula the term:

(1-(1-FT%)^2)*0.4*FTA

This, plus the term above, make up the simple form of the scoring possession formula:

Scoring Poss.=FG+ASTFIG*AST-ASTFIG*FG*(TMAST/TMMIN*5*Min-AST)
                                      -----------------------
                                      (TMFG/TMMIN*5*Min-AST)
		+(1-(1-FT%)^2)*0.4*FTA

This is essentially the same form as that seen in The Basketball Hoopla, which I did in 1988. I have modified the formula slightly since then to account for offensive rebounds, which I will now discuss.

Offensive rebounds are offense. They help the offense by preserving a possession. This has been my stand since the Hoopla. However, using my logic above for giving credit to assistants, I should also give some credit for scoring possessions to offensive rebounders. How much? It depends on how well the team scores when you look at the offense without offensive rebounds. If the team never makes any baskets, obviously, getting an offensive rebound doesn't mean much. If the team shoots well, getting a second shot is more advantageous probabilistically. This is how I've chosen to weight offensive rebounds in scoring possessions; it is reflected in the complex form of scoring possessions below:

Scoring Poss.=[FG+ASTFIG*AST-ASTFIG*FG*(TMAST/TMMIN*5*Min-AST)
                                      -----------------------
                                      (TMFG/TMMIN*5*Min-AST)
		+(1-(1-FT%)^2)*0.4*FTA]*
		[1-ORFIG*(TMFloor%-TMPlay%)]+
		OR*TMPlay%*ORFIG

ORFIG=0.1
TMFloor%={(FG+(1-(1-FT%)^2)*0.4*FTA)/Possessions}_Tm
TMPlay%={(FG+(1-(1-FT%)^2)*0.4*FTA)/(Possessions + OR)}_Tm

The Team Floor % (TMFloor%) and Team Play % (TMPlay%) are, respectively, estimates of what percentage the team scores on its possessions and what percentage the team scores on its possessions when offensive rebounds are looked at as starting a new possession. Basically, the difference between these two team values indicates how much an offensive rebounds helps the team's chance of scoring. By arranging them in the formula as I do, I take away some credit from the earlier formula (where OR's are not included) and add it back in through individual offensive rebounds at the end. The ORFIG says a player gets about one-tenth credit for each scoring possession that occurs this way. This roughly means that, after an offensive rebound, there is likely to be about two other people handling the ball before a scoring possession. This is a guess and it's very complicated as it is. So if you didn't get all this, don't worry about it. This is a new science. There were only a couple people in the world who understood quantum physics when it was developed. Now everyone understands it. (That's a joke.)

So that is scoring possessions, but we want to know how many points that leads to. Since we know that there are usually two points for every scoring possession, we can multiply scoring possessions by two as a first approximation. That is exactly what I do...as a first approximation. There are some extra things I do to make it more accurate:

Points Produced={2*[(FGM+0.5*3ptFGM)+ASTFIG*AST*(TMFG-3ptFGM)
                                                -------------
                                                (TMFG-TM3ptFGM)


		-ASTFIG*(FGM+0.5*3ptFGM)*(TMAST/TMMIN*5*Min-AST)
                                         -----------------------]
                                         (TMFG/TMMIN*5*Min-AST)

		+0.4*FTA*(2*FT%^2+2*FT%*(1-FT%))+qmk*FTM}

		*(1-(TMFloor%-TMPlay%)*ORFIG)

		+ORFIG*TMPlay%*OR*TMPtsPerScPoss

qmk = 0.06
TMPtsPerScPoss = Team Points Per Scoring Possession

The term qmk comes from additional free throws resulting from things like technical fouls and three point plays; think of it as a glorified fudge factor. The term TMPtsPerScPoss is evaluated using

TMPtsPerScPoss=      Team Points
               ------------------------
               FG+0.4*FTA*(1-(1-FT%)^2)

The logic is not transparent from the equations. I weight the two point scoring possessions by two, the one point scoring possessions by one, and the three point scoring possessions by three. I ignored scoring possessions on which more than three points were scored because they are very rare.

Whew! Remember where we are. I'm trying (perhaps in vain) to demonstrate where the offensive rating for individuals comes from. What I just did was give the formula for the points produced by an individual. Since the offensive rating is just points produced per 100 possessions used, I now have to show you the formula for how many possessions a player uses. If you're still with me now, you can handle the next part...

Possessions used include scoring possessions, of course, plus all those possessions a player ends without a score. These are possessions with a turnover and missed shots (field goals or free throws) that aren't rebounded by the offense. Easy enough. Here's the formula:

Possessions = Scoring Possessions
		+ FGA-FG - (FGA-FG)*(TMOR/(TMOR+Opp.DR))
		+ (1-FT%)^2*0.4*FTA + TO

Told you that was easy. Now divide points produced by possessions and multiply by 100 to get a rating. (Thank goodness for computers.)

Defensive Ratings

That was the offense. The defensive formulas are much simpler, because methods for defensive evaluation aren't as well developed theoretically. Simply put, I have struggled with evaluating players' defenses for many years now. I came up with a basic method three or four years ago called defensive stops, which are a way of estimating how many times a player stops the opposition from scoring. It's not a bad first approximation, but it misses out on players like Joe Dumars and Glenn Rivers, who prevent their assignments from scoring by not allowing them good looks at the basket. They don't get many defensive rebounds or blocks and don't steal the ball much, but they shut down their men. Doug Steele came up with a good way for accounting for this type of player, the kind of player I call a good man defender. On the other hand, my method does best at evaluating team defense, which would include blocks, steals, and defensive rebounds. Doug has begun including these in his method, too, but he uses a form of linear weights, something I disapprove of rather heartily.

Defensive stops occur every time the opposing offense ends a possession without scoring. This can happen via a turnover or a missed shot that the defense rebounds. I evaluate every player on the team as being roughly even in man defense. I do this by finding out how often (times per minute played) the opposition is stopped not through a steal or block, then multiply it by the individual's minutes played. In the formula, this is

Min*[(OppFGA-OppFGM-OppOR-TMBLK)/2+(OppTO-TMSTL)]/TMMIN

The first part inside the square brackets is how many times the opposition misses a shot, then divided by a factor of two. I divide by two because half of a defensive stop is the missing of the shot -- the other half is getting the defensive rebound. (There is a slight error in logic in the previous statement. See if you can pick up on it.) The second part in the square brackets is the number of turnovers not caused by steals.

Now we've taken out the individual contributions of blocks, steals, and defensive rebounds to get an average estimate of man defense. It's time to add those back in for each individual. Just adding in an individual's steals and half of both his defensive rebounds and blocks gives an overall formula for defensive stops:

Defensive Stops =
	Min*[(OppFGA-OppFGM-OppOR-TMBLK)/2+(OppTO-TMSTL)]/TMMIN
	+ STL + 0.5*(DR+BLK)

Again, remember where we are and where we want to go. We have an offensive rating from above and now we need a defensive rating. We have defensive stops, not a defensive rating. We need to estimate "how many points per possession a player gives up."

This, I'm sorry to say, is difficult to define. How do you evaluate the guard who lets his man drive past him, where his center saves him by blocking the layup attempt? Who gets credit for the stop? Who gets the blame if the layup isn't blocked and goes in? These are theoretical difficulties. Generally, a team plays a style of defense that either asks for help defense or asks for straight man defense or something in between. Depending on whether that guard was supposed to let his man go by determines whether he did something right. If he was supposed to let the man go by, then rotated to the big man's assignment to cut off the dump pass, he apparently did OK. If he was supposed to play straight, then any points scored should mostlly be blamed on him, not the big man covering the little guys' back when it's not his responsibility.

Given this dilemma, how do we proceed? Basically, I try to fit defensive stops into a framework that makes some sense if we ignore the above questions. Ignoring the difficulties sometimes allows one to see a solution. That is my guiding paradigm here.

If we can somehow evaluate how many stops per possession a player has, we essentially have the defensive equivalent of a floor percentage Going from this to a points per possession rating is fairly straightforward. This is the method.

To find the stops per possession, divide a player's total defensive stops by his minutes played, multiply by the team's minutes played, and divide by the team's total number of possessions:

             Stops*TMMIN
Stops/Poss= -------------
             Min*TMPoss.

This number turns out to be very high for some players. For example, Olajuwon consistently has a stops/poss value of 0.7 or so. This would mean that he stops seven tenths of all possessions he is responsible for. Or his opponents score only 30% of the time against him, for a rough rating of 60. Olajuwon is a good defender, but he's not that good.

In order to compensate for this flaw, I weight the team's defensive rating with the individual's stops/poss value. I weight them 80% to 20% because I figure that one player is 20% of his defensive team. It's a little hokey, logically, but it gives results that I'm fairly pleased with, as I mentioned above:

Def. Rtg = 0.8*TMDef.Rtg + 0.2*(200*(1-stops/poss))

For completeness, the factor 200*(1-stops/poss) is an approximation of the points per 100 possessions arising from the given value of stops/poss.

So that's it. Those are the methods. Their derivation was probably pretty dry, but science can be that way. The results, however, can be trusted and are often very enlightening because there is good (but dry) science behind them. Linear weights methods, on the other hand......

Why Barra's Linear Weights Didn't Work

My feeling on linear weights methods has been described elsewhere in this Journal, but I will restate it here. Linear weights methods are like the SAT, that dreaded test that supposedly determines who is smarter, because both linear weights and the SAT are very approximate tools to help determine the overall ability of certain people. Both also cannot be taken too literally. Just as a person scoring a 1200 on the SAT isn't guaranteed to do better in college classes (or in a job, for that matter) than a person scoring 800 or 1000, a basketball player with a production rating (PR) of 20 is not guaranteed to be a better player than one with a PR of 15. The reason for this, of course, is that neither the SAT nor the PR are what are important; rather, success in college (or work) and winning basketball games are important. Although a good SAT and college success are statistically correlated and a high PR and winning percentage are statistically correlated, these correlations are not close to one, which would indicate essentially a perfect indicator. Rather, the correlation between linear weights results and winning percentage is only about 70-80% (and Barra's formula was the worst of the ones I tested). Any pure statisticians out there know that this isn't very good. For non-statisticians, this value can be equated to how well you can measure 100m sprinters' times with an analog watch: you can tell whether they finished in 10 seconds or 11 seconds, but the difference between first and fifth place is often no more than 0.2 seconds. In contrast to this, the Correlated Gaussian method and the Pythagorean Method are both correlated with actual winning percentage at the 95% level; in other words, they are much better indicators of winning than linear weights.

I should also note that linear weights are very subject to individual biases because there is no theory behind them. I have seen at least six different linear weights formulas that arbitrarily weight blocks, rebounds, assists, steals, etc. How you assign those weights determines your answer to questions like "Who is better?". If Barra thought that blocks were not very important he could reduce their weight from 2 to 1. It would certainly change Barra's conclusion. Because there is no theory to help determine what weights should be, Barra (and others) hint at equating blocks, rebounds, etc. to points. But this makes no sense. When a player blocks a shot, his team doesn't get a point. When a player rebounds a shot, he doesn't get a point. For example, say one player blocks an opponent's shot, gets the rebound, then misses the layup at the other end (and the opponent rebounds). According to Barra's formula, this player has accumulated 2.5 points in this sequence, but no points were scored! Maybe we should look at it another way -- his opponent has accumulated 0.5 points in this sequence by rebounding and missing a shot. Still, three Max Points (as Barra calls them) were scored, but no points that really counted were scored. By Max Points, our hypothetical player is also leading by two (2.5 to 0.5), but by real points, he is still even. Finally, there is no reason for linear weights to equate with points since it accounts for both offensive and defensive statistics (which prevent opponents' points). Even if it did better equate to points scored, where does it account for points allowed?