Introduction to the Extensively Updated and Expanded User Guide for Real Coach Ratings
On November 23 (2010) Quest for the Ring (QFTR) published the first Real Coach Ratings featuring the new coach playoff scores (that are the primary component of the playoffs sub ratings). The coach playoff scores are derived from the QFTR playoffs database which was constructed in 2010. The November 23 posting was the “Look Back Version” which rates coaches from the season gone by, in this case 2009-10. Three days later the Look Ahead Version was published which rates the NBA coaches for 2010-11.
Strictly speaking we should have held back publication of the ratings until the new User Guide for Real Coach Ratings was completed, because this is a rule which we always have followed (until this little exception here) and because the ratings were extensively revamped which made the old User Guide not very useful. We held back publication of the ratings for more than a month (a record hold) and at that point could not stand to wait any longer. We wanted to and expected to have a completely revamped and much larger User Guide for Real Coach Ratings done by the end of November, or by about December 10 at the very latest, but not only were those targets missed, we were not able to finish the new Guide until January 18, 2011.
One of two main reasons it took so long was that there was a serious production slowdown in November and December 2010 (being made up for in early 2011). The other main reason it took so long is that the new Guide is massively long, over 22,000 words. This is one third the size of a short book and almost one fourth the size of a lengthy book. This is by far the largest User Guide in history (and also by far the largest QFTR posting of all time).
Let no one say that we don’t take rating coaches correctly seriously! QFTR has come very close now to achieving the objective we swore we would eventually achieve: to make Real Coach Ratings as accurate and useful as Real Player Ratings are.
The Guide is too long for many people to be able to read in one sitting. Only certain key people are going to ever read it in full and most of those people will read it in stages at various times. The people who read it in full are going to be the people who really, really want to win games and especially playoff games (and hopefully make a lot of extra money in the process).
Regardless of whether you ever read it all or not, like all guides, this one is a carefully organized manual that can be used as needed. The Guide is conveniently divided into six sections and each section is further divided into clearly indicated sub sections. The section and subsection headers make it easy to find exactly what you want. Alternatively, with the User Guide loaded on the QFTR Reference Site, you could use the Google blog search at the very top of that page, and/or you could use control / F to find what you are looking for.
Like all our User Guides, aside from being a guide to the product (in this case Real Coach Ratings) this Guide is also a very business like and very thorough explanation of very critical aspects of the game of basketball. Actually, in all that we do but especially in this particular Guide, QFTR goes beyond the game and the players and goes extensively into team coaching and management. For each point made in a User Guide, whether at one extreme the point is very well known by the general public or whether at the opposite extreme the public believes the opposite (or is totally unaware of the point) all the points are explained and in most cases proved beyond any shadow of a doubt. Moreover, User Guides such as this one are like "Super Reports" that are the equivalent of a lot of QFTR reports combined into one giant but carefully organized one.
Here is the Introduction Section for the new User Guide for Real Coach Ratings (RCR) which hopefully will motivate you to read other sections and will at least explain the RCR system in general but important terms.
======= SECTION ONE: INTRODUCTION =======
Quest for the Ring is proud and pleased to present what is apparently the world's first serious effort to scientifically and accurately rate and rank all of the current NBA head coaches. Even the academic oriented basketball statistics sites do not have any formulas or specialized ratings for coaches, although some of them thank goodness keep track of basic coach data including wins and losses.
The QFTR coach rating product is called Real Coach Ratings (RCR). The first edition of these annual ratings, which compared to the latest version was relatively crude (and yet still much more than mere opinion) was published in October 2008. The second edition, which features substantial but relatively modest improvements over the 2008 edition, was published in early December, 2009. In late November 2010 the third edition, which featured relatively large scale and important improvements over the 2009 edition, was published. At this time it is not known to what extent it will be desirable and possible to improve RCR further, but there is a fairly high probability that most and possibly all future changes to and expansions of RCR will all be small compared with the changes and expansion in 2010.
Why should the coaches hide behind a black curtain as they do in the USA? Concerning coaches, there is virtually a total lack of the kind of statistical comparing and contrasting that goes on with players 24/7. To say there is a double standard where players get the short end of the stick would be an understatement. Coaches can get away with relative incompetence and negligence for many years, in some cases indefinitely, whereas players will within days, weeks, or a few months at the most have their minutes cut at the least, and they can easily be bounced around the NBA or demoted to some other League. When QFTR started to rank and rate Coaches in 2008, it was way, way overdue that someone did it.
The big Corporation sites such as ESPN have editorial limitations which prevent them from being severely critical of NBA head coaches, managers, or owners. ESPN writers can be mildly critical at the most (which in practice means they have to hint at criticism rather than directly criticize). For heavy criticism of NBA coaches, managers, or owners, you have to go somewhere other than ESPN, CBS Sports, Fox Sports, and NBC Sports. As one of many examples, you might see some heavy criticism at SlamOnline.com. And then even when you do venture elsewhere and see some heavy criticism of coaches, managers, or owners, you are most often going to see only opinions as opposed to conclusions based on hard research. I mean, if you are lucky, the opinions are dead on accurate, but since there is little if any evidence from research backing up those opinions they could easily be wrong. Here at QFTR it is the reverse: you seldom will see a mere opinion and most of what you see are conclusions backed up by valid and adequate research.
I can pretty much guarantee you that no one has ever, even with the capabilities created by the Internet age, put in as much effort, thought, and technology as QFTR has into fairly comparing NBA coaches with widely different lengths of time spent in professional head coaching. Despite the fact that QFTR has little or no competition for coach ratings, it applies full scale quality control to RCR and provides a very detailed User Guide that exceeds 20,000 words. And the Real Coach Ratings (RCR) system CAN be used in other Leagues, other countries, and on other planets, if there are any other basketball planets, that is!
The Real Coach Rating system has been extensively improved in the second half of 2010. The biggest improvement is the new factor called "Playoffs Games Coaching Score". A lot of time went into developing this factor, much of which went into developing an underlying data base called the "NBA Playoffs Series, Teams and Coaches Database". This database consists of every playoff series ever played since 1980 except for twenty best of three first round playoff series played between 1980 and 1983.
To summarize simply, for each series, a statistically valid estimate of exactly how many games should have been won by each team is calculated (to two decimal places, for example, 3.25 wins) and then the actual number of wins is compared to this and either a positive score or negative score is derived from this.
THE NBA PLAYOFFS SERIES, TEAMS, AND COACHES DATABASE
In 2010 Quest for the Ring developed a database which has details about virtually all playoff series of the world’s premier pro basketball League, the NBA, from 1980 to the present. The number one reason why the database was developed was so that RCR could be substantially improved. Specifically, one of the main objectives for creating this database was to identify which coaches of pro teams win more games in the playoffs that they “were supposed to lose” than they lose games that they were "supposed to win" (net playoff winners). And of course, we also want to find out which coaches lose more playoff games they were supposed to win than they win playoff games they were supposed to lose (net playoffs losers). (And of course there are some coaches who win some that should have been losses and lose some that should have been wins whose overall record on that is about even up.)
In late November 2010 and in very early 2011 much of the information that can be obtained from the database was published in various Reports. See especially:
“NBA Playoffs Upsets: How Many are There and Why do They Happen?”
“Real Coach Ratings for the NBA, 2010-11, Look Ahead”
“Official NBA Coach Recommendations: Can the Coach of Your Team Win the Quest or Not?”
Note however that the actual database has not been published and is not scheduled to be at this time. Not all of the information that can be obtained from this database has been published in Reports yet. And although QFTR has more and more in recent years published Excel worksheets that are products of databases or in effect are micro databases, the templates for the largest databases can not be published due to risks associated with copyright violation. The QFTR public email address can be used for inquiries about how someone could possibly obtain a copy of the database and about the terms of use for it. For the email address, at the QFTR home page, click the “Contact” link that is on one of the horizontal menus just under the banner.
Using what is formally known as the “NBA Playoffs Series, Teams, and Coaches Database", and also using knowledge about statistics and basketball, it has been proven beyond a shadow of a doubt that some coaches are better in the regular season than they are in the playoffs. Actually, to be more precise, the playoff losing coaches are ones who have their teams playing in ways that lead to relatively more wins in the regular season than in the playoffs. And vice versa: coaches who win extra games in the playoffs have selected strategies and tactics that work better in the playoffs than they do in the regular season.
This is not really all that surprising as long as you know that the game of basketball itself changes a little in the playoffs from what it is in the regular season. The rules stay the same and to the untrained eye it may seem like the same game, but in reality the way it is played changes a little and the way the referees call games changes a little. Although most people do not know all of the details of the changes (the magnitudes and the components and so forth) most people are aware in general terms that defending is more important in the playoffs than it is in the regular season. To state it a little differently, most people are aware that many if not most teams ramp up their defending for the playoffs; they play defense more aggressively, more energetically, more athletically, and sometimes smarter.
Defending can be improved almost overnight through will and effort. But this is not really true with offense. Here it’s appropriate to insert a few paragraphs from the User Guide to Real Team Ratings:
DO NOT MAKE THE MISTAKE OF OVERSTATING THE IMPORTANCE OF DEFENSE
But don’t fall into a trap here; don’t get carried away. In basketball defense is relatively less important than it is in many and very possibly most other sports. Basketball is designed to be a game that favors the offense more so than for many, many other sports.
The tightrope here is that on the one hand you have to realize that defense is more important in the playoffs than it is in the regular season. On the other hand you have to understand that in basketball exactly how important the defense can be is limited fairly strictly. Defense alone can not possibly win you a Championship in basketball.
By contrast, in American pro football the limitations on how important the defense can be are far weaker, meaning that unlike in basketball, you can win the Super Bowl Championship in football pretty easily with the best defense in the League but a below average offense. For example, the Pittsburgh Steelers have done this several times over the years. But in basketball it is extremely difficult to win the Championship (and you are going to need some luck) to win it with even the best defense in the League but only the 20th best offense (out of 30). What you really need in basketball to go along with the best defense in the League is at the very least the 15th best offense (out of 30); and to have a good chance you need at least the 10th best offense to go along with the best defense.
So even though in basketball defense is more important in the playoffs than it is in the regular season, the magnitude of the change is not really all that large; in basketball defense is only a little more or, arguably in some cases, moderately more important in the playoffs than in the regular season.
Note also that, ironically, the teams that are the very best defensively in the regular season are unable to increase the quality of their defending in the playoffs as much as teams that come into the playoffs with lower ranked defenses. Coming into the playoffs, teams with one of the best two or three offenses in the League but whose defenses are down around 10th best are generally more likely to win the Championship then teams which come in with one of the top two or three defenses but only about the 10th best offense.
It’s obvious that teams have the opportunity to be better defensively in the playoffs than they were in the regular season; after all, this happens all the time. Defensively in the playoffs, it’s mostly a matter of doing the same things that were done in the regular season harder, faster, and/or smarter. But the opportunity for a team to be better offensively in the playoffs than it was in the regular season is very limited. In other words, offensively, what you saw in the regular season is pretty much all you are going to see in the playoffs. Teams should assume they can improve a little defensively but they should never ever assume they can get substantially better offensively when the playoffs come, because that is unlikely to happen.
This is indirectly another reason why teams that run slightly organized offenses are much smarter and more likely to win The Quest for the Ring than are the teams that run more street ball type offenses. Coaches who run the street ball type offenses often think that that strategy will work better in the playoffs than in the regular season. They may think that unlike a slightly organized offense a street ball type offense can be ramped up in the playoffs. And they may think that a street ball type offense is exactly what you want to try to offset the ramped of defenses you see in the playoffs.
All of these suppositions are false to one extent or another. First, street ball type offenses work less well in the playoffs against ramped up defenses than they do in the regular season against lesser defenses. Second, you can not substantially ramp up any type of offense in the playoffs including the street ball type. For offense more so than defense, it is crucial that in the regular season you are playing in a way that will allow you to win in the playoffs. For defense it is theoretically very recommended but not required that you in the regular season play in a way that will allow you to win in the playoffs. Third, ramped up defenses are relatively more effective against street ball type offenses than they are against slightly organized offenses.
For convenience, this Guide is developed into main sections and subsections. The main sections are:
Section 1 Introduction (Which ends here)
Section 2 Components of and Format of Real Coach Ratings Reports
Section 3 Discussion of and Calculation of Factors used for the Playoffs Sub Rating
Section 4 Discussion of and Calculation of Factors used for the Regular Season Sub Rating
Section 5 Interpretation of Ratings and Evaluation of Coaches
Section 6 Cautions Including the Well Known Experience Gap Problem
Within each section subsections are in all caps as shown.
The complete User Guide to Real Coach Ratings is here.