SB Nation - Login for mobile commenting

Lookout Landing

Stats vs. Scouts

So I've been thinking about this post for about a year now, but I've never got around to writing it properly. Well, I have 45 minutes to kill, so it's as good a time as any. I realise that I'm probably preaching to the choir here but whatever.

"Stats vs. scouts" is one of the central arguments in baseball circles, leading to mildly horrifying levels of vitriol leveled at members of either side. It's one of Moneyball's lasting legacies, as anything that portrays the old guard as curmudgeonly incompetents doomed to fall under the armies of cleverness is going to provoke a reaction from said curmudgeonly incompetents (NB: I am not calling the old guard curmudgeonly incompetents), and rightly so.

The problem is of course that statistical analysis and scouting are by no means natural enemies - in fact, they're closely allied. Advanced statistics are proxies for good scouting. Sometimes they pick up things that scouts miss, and sometimes scouting will tell you things that stats will not, but they are complementary. A team's front office doesn't have to be one or the other - the point is that both methods are tools to evaluate players.

Interestingly, the argument doesn't even seem to be about player evaluation, and the dogmatic camps generally make absolutely no sense at all. Those in the "scouts" camp spout off about RBI and batting average (which are statistics last time I looked), and people trying to be "statsy" will misuse numbers they don't understand to come to stupid conclusions. Does anyone* seriously think that 'gritty' would be the main point in a real scouting report? Or that analysts throw random numbers at the wall until the come up with their stats?

No, this isn't about statistics against scouting. Not at all. It's about people believing that they're already experts on baseball player evaluation. Imagine if people took the same attitude towards, say, engineering as they did baseball. You'd see people looking at construction sites with total disdain:

"I've walked through a lot of buildings and I can say for sure that using an eccentric braced frame for the lateral force resisting system is completely stupid. SCBFs are much better in the clutch."

Watching a lot of baseball doesn't mean you know anything. Playing a lot of baseball doesn't mean you know anything. Watching a lot of baseball while listening to the opinions of people who've played a lot of baseball doesn't mean you know anything. But it's amazing how many people seem to think it does.

It's not stats against scouts, and it never has been. It's acknowledging one's own ignorance against the belief that one already knows everything.

Because if you already know everything, why would you ever need to think about it?

Star-divide

*Well, yes I'm sure someone does, but if that person is you and you feel inclined to comment on it you're going to end up banned so don't.

13 recs  |  47 comments

Comments

Rec'd and bookmarked to I can link it to other sites whenever this topic comes up.
Calibrating judgment

My mantra is that the purpose of quantitative information is to calibrate judgment. Even in structural engineering, where most of the design may done with calculations, at some point an experienced person needs to look at the output and decide, “does this make sense?”

Part of the process is assessing the quality of the numeric information with which we are working. The better the quantitative data, the more weight we give to the numbers.

True of engineering, but I'm not sold that it's true here

Baseball player evaluation is a lot more fluid than structural design. Ultimately I think both statistical analysis and scouting need to be used as checks for each other.

If they clash, which do you trust?

It’s an interesting philosophical question – if you’re breaking new ground and your results clash with established wisdom, does that mean your formula fails the reasonableness test, or that you’re revealing faults in the established wisdom?

Nowadays it seems there’s enough work that there are baselines from which to start, but I wonder how Bill James and the other early trailblazers handled questions like those. If you’re Bill James and the late 1970s version of Raul Ibanez is racking up RBIs and is well-respected by virtually everybody in baseball (left-handed sock!), what do you think when your formulas show he’s essentially of no incremental worth?

There’s no baseline at that point. Do you just toss out the formulas as not passing the smell test, or do you take your result as gospel and start telling the baseball establishment they don’t know how to value players?

Well

Back in the day that definitely would have been an issue. The reason we’re seeing a lot of progress now is the vast improvement in ability to collect data. When evaluating Raul’s defense, for example, now we have ball-in-play data to figure out how many plays an average defender makes vs. how many he made.

I'm always learning

I remember when I though FRAR was worth a damn!

People cling to FRAR because it's available historically.

They’re not willing to accept that we just can’t measure defense from 1960, so they’re going to use something that purports to.

My introduction to sabermetrics was WARP.
MLEs and RC/G for me
EqA!

Mine was useful.

I got into sabermetrics from FJM.

So yeah, WARP, VORP and EqA.

I think I started with BP 2002
God you all make me feel old.

My intro was most likely defensive average in 1993. I honestly do not remember who devised it.

Maybe if you cared more about it you'd be running this website and I'd have more free time in my life
If I was running this website I would not be married to the coolest woman in the world.
Don't say that to me
Let's try this then.

If I was running this website no one would read it.

Therefore you would be running some other website with tons of traffic.

The moral of the story is that it sucks to be Jeff
I started at the beginning of '04 with USSM
It started to gain traction in my mind with this post.

http://www.lookoutlanding.com/2007/7/23/121950/049#3567682

Me too.

That was the Eric Hinske issue, right?

And I still like EqA.

An offensive metric that includes stolen bases. Nice.

wOBA doesn't do this?
I don't believe so, no.
It doesn't, but it does include times reached base via error

which is a sort-of proxy for speed.

Bill James Baseball Abstract . . . 1984?

I think it was ‘84. I learned to use spreadsheets setting them up to calculate RC and RC/G; and I’ve lost count of how many players I cranked through the Brock2 spreadsheet. (I even took to inventing players according to certain types to see how they aged.)

I have the re-released Bill James Abstract.

Lots of good stuff in there. What a writer. I keep it by my bed and crack it every couple of weeks.

It's impossible for me to read straight through.

He digresses or mentions other players and seasons, I start skipping around.

Rec'd

It’s about time we stop with the retarded squabbling.

Bookmarked.

Thanks Graham.

As it comes down to in many entrenched debates,

it’s not actually about Side A vs Side B, it’s about closed vs open minds (which will exist on both sides).

Am I wrong to be this excited about Zduriencik?

The fact that a guy with such a scouting background has come in and done this is blowing my mind. I don’t know how much of that is based on how shitty his predecessor was, but I have more faith than I’ve had in years.

No, Zduriencik has been awesome so far

Yeah, he’s not the 30-year-something Ivy League hotshot we’ve all been hoping for, but I’m absolutely psyched that our GM is embracing both traditional scouting and sabermetrics.

Maybe he'll hire

some 3020-year something Ivy League hotshot(s)…

Maybe I will figure out cold fusion and buy the Mariners
But Cambridge isn't in the Ivy League...
This might be better

Not to put down the younger guys by any stretch, but when a veteran scout with a record of success like Zduriencik’s comes in and sets up his most trusted assistant to build an entire research department — especially given how good Zduriencik has proven himself to be at dealing with people over the course of his career — if he shows himself to be as open-minded in listening

to that department as he has in creating it in the first place, that could be the best possible combination.

This is pretty much

exactly what I’d been thinking about this whole thing, too. Bottom line is, there are people who want to be right and approach new information objectively, and then there are people who don’t want to admit they’re wrong—which is necessary to eventually be right; none of us were any good at evaluating from the outset.

Very well put

You can say pretty much the same thing in any part of life — and that says it about as well as I’ve ever seen.

Wow, that ~$1700, and it's only the weekend.

I’m surprised this post hasn’t been rec’d enough to make the recommended fanpost list.

Front page post.
And I put this comment in the wrong post anyway, what a dummy.

I was experimenting with multiple open windows.

This is nice to hear...

I think that evaluation of players is not unlike American laws; Congress writes the laws (stats) which are then interpreted by judges (scouts). Enforcing all laws to the letter would make as much sense as ranking players statistically and then aquiring them in that orders regardless of intangible circumstances. As usual, extremism for the loss.

zomg no politcis plz

You must Login with your SB Nation account and be a member of Lookout Landing to post a comment.