In his excellent interview with John Sickels, Mark Newman made a passing remark that caught my eye:
I don’t worry about his glove, Romine can really catch. He turns bullets into marshmallows. His arm is strong and accurate. By the internal defensive metrics we use, Romine rates as a very strong defender, and Montero isn’t far behind him.
Many clubs are using internal metrics at this point, and I have no inclination to pat Yankee management on the back for utilizing tools that even casual fans access in some form on a regular basis. However, I do think that Newman’s comment highlights the informational divide that exists between fans and the teams. It is quite possible that Newman is lying through his teeth to Sickels here, and the internal metrics are actually down on both Romine and Montero. Putting the veracity of his statement aside, the fact that they are using internal data to make player evaluations means that we are unlikely to ever know exactly how they value players when making decisions. As one of our writers, Matt Imbrogno, said to me this morning, a baseball team is like a factory making a product. We can enjoy the product (the team) and possibly take a tour of the factory and see the basics of how things are made (using our own amateur scouting and the available metrics), but there are certain things (internal metrics and scouting reports) that the manufacturer won’t let us see, and those things might be vital to the production process.
To provide an illustration of how this might impact our view of team decision-making, I want to refer back to the post I wrote last week about ‘s defense. In the post I noted that the various defensive metrics diverge wildly when attempting to quantify Cano’s defense, with some having him as well below average and others placing him at the top of the league. Now imagine that Cano hits the free agent market and a number of teams are considering pursuing him. How much a team decides to offer him will in part depend on their internal evaluations of his defense, which can apparently range from excellent to awful. An offer based on those metrics might leave them with some confused fans who depend on other metrics to reach a wildly different conclusion about Cano’s defensive prowess.
A similar point can be made about scouting reports. We make subjective evaluations based upon our observations all the time, and we buttress our personal knowledge with information from media members and scouting services. We use these sources of data to rank and compare players, which in turn colors our thinking regarding all sorts of transactions. Teams, however, have scores of scouts whose opinions might diverge wildly from the consensus of the masses. A perfect example of this can be found in the 2010 draft, in which the Yankees took Cito Culver in the first round despite most draftniks pegging him as a 3rd or 4th round pick. Particularly with regard to prospects, scouts from various teams are likely to differ significantly from the public’s view on a frequent basis. Again, this alternate source of information may lead to decisions that fans might find to be confusing.
Let me be clear: I am not making an appeal to authority, claiming that because teams have different, and possibly better, information they will always be right and fans will always be wrong. Rather, I am suggesting that decisions that can seem befuddling to us might be a bit clearer if we had the same information that the team has. Because we do not, a decision that makes at least a modicum of sense internally can seem ridiculous to outsiders, despite the fact that many of those outsiders might make the same move if presented with the same data.
Does this mean that we cannot analyze trades, signings, and player performance using the data that we have? Absolutely not. The informational imbalance does not impact whether the decision was an objectively good one or not, it simply alters our view of the decision-making process. To elaborate further, I have no issue with fans disliking the trade for the Diamondbacks based on the information that we have available to us. There are plenty of worthy methods to determine value and all of them cut against the trade. However, leaping from that evaluation to one in which we deride the Diamondbacks’ front office and their decision-making requires an assumption that they are using the same evaluations that we are. For example, it is quite possible that something in their internal reports caused them to have a much higher opinion of Tyler Skaggs than most external scouting services do. If we knew that, we still might dislike the trade, but at least we would better understand the thinking behind it. While the informational divide does not mean that the team’s decisions are always correct, it does suggest that we may not be as equipped to evaluate the decision-making process as we think we are.
If you enjoyed this article, please consider sharing it!
Of course, if the Dbacks really liked Skaggs, but no other objective source did, they would be foolish for assigning their inflated value to the trade equation. Rather, they’d have been wise to consider him an undervalued asset and acquire him at a discount.
That’s a good point. I may have done my argument a bit of a disservice by using such an obviously bad move as an example. This would come in more often about a close trade, where we would better understand the perceived loser’s decision if we had their data. While Im not sure it changes the calculus of the move itself, it might change out expectations of how that front office will act moving forward.
While this might not be exactly on-topic, we wouldn’t be as informed even if we all used the same metrics. obviously, clubs are far more informed than we are regarding clubhouse manner and the extensiveness and history of injuries (see: joba). i would also be interesting to see how internal metrics rate different KIND of stats that we have: for example, a low walk rate is a better crappy stat to have a high CS%, or whatever. I might say “Dude, he has crap for discipline, but at least he doesn’t get thrown out,’ where someone else might go, “the guy can run like a motherfucker but he never walks.” Different might be that the first guy is me, blogger extraordinare, and the second guy is, you know, Theo.
ramble ramble.
That’s definitely true as well. We all have stats that we glom on to and tend to stress when making evaluations, such that me and another blogger with similar philosophies still might reach different results on an issue. I can only imagine that teams have internal philosophies in which they stress certain skills, and by extension certain stats.
I wonder if teams hire scouts that:
a) conflict, to figure out who has the best idea and reasons. if the guy promoting wOBA brings over more successful players with his more legit arguments, then the rest of the organization would be going, ‘Huh, wOBA works!’ and could constantly force this particular guy to argue against multiple guys promoting stats (ISO, BA, OPS, etc).
b) all agree, so they have a standard ‘fare’ so to speak, to go after.
/muses and whimsy
Just wanted to say I really enjoyed this piece. Wish I had something of substance to add, but that’s all I got.
Thanks, Larry.
I think one of the things we don’t have that they do is what they do while they’re being trained. They may take things like time between catching and throwing in catching drills (sticking with your Romine/Montero example.) Girardi and his staff do a lot of stop-watch stuff, and the organization may take a lot of that stuff into account. (On the other hand, that may have nothing to do with it. Some of the comments the past few springs-before any games have been played-make me think it does.)
YEah nice piece here Moshe Man, well done.