June 13, 2007

Ripping ESPN.com, continued...

This probably isn't the best way to start this blog - it's probably more interesting when I blog about, you know, actual baseball. But I thought I'd do a follow-up on yesterday's "discussion" of ESPN's new player rating system. Later on Monday, the creator of the ratings - Jeff Bennett - did a chat on ESPN's SportsNation explaining the rankings some more. Here's the link (I think it's Insider-only).

Here are some of the more interesting questions and responses...


Scot (TX): Some questions on formulas for you Jeff: 1. Why team win percentage? Is Teixeira really a "better" player if he played for Detroit instead of Texas? 2. Why not differentiate CF from the corner OF spots in the defensive standings? A player who can play CF seems to have more value. 3. Did you ever try to work in quality of defense? Right now an excellent fielder and a poor fielder get the same amount of defensive points as long as they both play SS.

Jeff Bennett: A multi-part question off the top....

1) I think there is some value to player on a wining ball club. This is evident with the BBWAA voting. However, it make up only 5% of a score and the difference between Detroit and Texas currently ranslates to about 0.4 rating points.

2. Currently this is a function of the way our stats are coded. Looks for this to be adjusted for in the future. the defensive spectrum would go obviously CF, RF, LF.

3. Yes, much like VORP, quality of defense is not accounted for.

1. Yes, there is some value to a player on a winning ball club. There is also some value to a player on a losing ball club. Typically, the players on losing ball clubs will, on average, have worse stats than players on winning ball clubs, so you shouldn't have to make an extra adjustment for this. By the way, I absolutely love that he thinks the fact that the MVP voting is based largely on team performance is a valid reason to adjust for this in his system.

2. This has been changed, according to the explanations. The difference isn't enough yet, but it's a step in the right direction.

3. So, you're defending a new system by pointing out an earlier, better system that has the same flaw. Remember, this was the stat that was labeled "cutting-edge" on the front page of ESPN.com. This wasn't what I was looking for.

Otter (Boston MA): What do you do for players that have a significant split in where they play defensively? Weighted average of difficulties? Is there any bonus given for this type of versatility?

Jeff Bennett: The position the player plays the majority of his games at is designated his primary position. There is no bonus of versatility, but it is something I considered.
So, if Victor Martinez plays 75 bad defensive games at catcher next year and 70 games at first, and Joe Mauer plays 145 good games at catcher next year...they get exactly the same points for their defense? VORP at least accounts for different positions played during the year (I think it does, anyways)...

Paul (Wildomar, CA): Anybody who has read Bill James knows saves are overrated. Why 40% allocated to saves? Seems to me the other 4 categories, maybe especially inherited runners scored, should be higher. We all know Pat Neshek is better than David Weathers.

Jeff Bennett: Saves are overrated. That is what the actual stat used here has a stiff penalty for blown saves. Neshek does quite well for himself here. He is #116 today overall. Not bad and ahead of many, many closers.
Saves are overrated. Therefore, there is a penalty for when you pitch poorly in a save situation. Does that make sense? And, although there is a penalty for blown saves, getting actual saves is the way relievers get 40% of their points. Pointing out how the best middle reliever in baseball is at #116 doesn't really help your case...

Taylor (Escondido CA): Did you take park effects into account at all? Of course the Padres hitters are going to rank poorly and the Padres pitchers awesome because of Petco. Or, for example, Ian Kinsler (Texas launching pad) v. Jose Lopez (Safeco death to hitters), have the same OPS, yet your rankings have Kinsler higher?

Jeff Bennett: Excellent question. Park effects are not directly applied to adjust the ratings. BIg parks doesn't seem to hurt the Comeria hitters this year or say Dmitri Young. I have a hard time de-valuing Peavy. He is 3-0 with 1.06 ERA on road this year. I see he is ranked 31st in Win shares today behind many pitchers. That seems a little low.
These rankings are flawed in many ways, but the lack of park factors is the biggest one. Saying "Big parks doesn't seem to hurt the Comerica hitters this year or say Dimitri Young" not only is very gramatically flawed, it is logically flawed as well. If a player has good stats in a pitcher's park, he would have even better stats in a hitter's park. That's simple logic. Hitters are always "hurt" by big parks, whether or not their stats are good. And Comerica has been a hitter's park this year, by the way, with a park factor of 1.071 (above 1.000 is hitter-friendly).

Bryan (Madison, WI): Don't you think that your formula for relievers makes it too difficult for actual relievers to be rated highly? You've said that saves are overrated, yet they comprise 40% of the reliever rating whereas holds are ignored completely. It seems to me a reliever could have a excellent season and unless he picks up a lot of wins (which is generally determined by the events earlier in the game and by the actions of his offense, not the pitcher himself) it would be very difficult for him to crack the top 100 or 150 players.

Jeff Bennett: Fair question, but set-up men can do well here by having a low ERA, Opp. BA, K-to BB-ratio. Last year Cla Merideth was one of the majors to 20 pitchers when applying retroactively. I think that would be news for most of mainstrream baseball fans. He has unhittbable in 2006 and didn't register many wins (5).
Yes, but those numbers are only 60% of a reliever's value. The other 40% is wins and saves. Basically, from this and some of the other questions, what Bennet seems to be saying is that great middle relievers can move themselves up to the level of a solid-to-good closer if they have a great year. When, in reality, a great middle reliever is often worth just as much as a great closer.

Ian, NYC: I don't understand what your point is behind this list. There are people who have put a heck of a lot of science and research into coming up with formulas like this (Win Shares, VORP, etc), while much of what you have selected here is totally arbitrary (why exactly %10 for BA for eaxmple?), and by pretending this is somehow scientific degrades the whole field of work on this subject. Much of what you are including here has been proven to be no reflection on individual player quality (like saves, wins and RBI to a large extent), not to mention penalizing someone because they play on a bad team. Why should anyone take this list seriously?

Jeff Bennett: Ian, I think you hit on something. There is no sucjh thing as the perfect way to evaluate a baseball player. Win Shares and VORP are great, but you can ask the same types of questions about their lists. This system is very fluid and puts players in perspective based on where they rank in the majors vs their peers. Nothing more scientific than that.
Actually, there are things more scientific. It's great that you compare relievers to relievers, hitters to hitters, etc....but Win Shares compares all players to all other players. Wouldn't that be "more scientific"? And I can't ask the same question about WS and VORP that I've been asking about this one: What the hell do the final numbers mean? WS and VORP numbers have meaning. This system just gives you numbers.

There's some more, but it's mostly just about individual players. Still no answer as to why there's no park factor adjustment, still no answer as to what the numbers mean, still no actual reason there's no defense adjustment. Hopefully ESPN gives up on this quickly.

I'll try to get to some actual on-the-field baseball stuff later today, namely Verlander's no-hitter...

No comments: