Jump to content

cruzer1

Al Avila on analytics...

Recommended Posts

https://www.fangraphs.com/blogs/sunday-notes-kids-in-mind-kris-medlen-got-back-on-the-horse/

Quote

Like every other team, the Detroit Tigers have a special sauce they use when grading players. How spicy is it in terms of analytics versus old-school scouting? Here is what Al Avila shared with me during the GM Meetings in December:

“We’ve developed a system where our scouting reports come in from our scouts, and those go into our system,” explained Detroit’s general manager. “We’ve also developed an analytical formula to rank players. We have a ranking of players that’s all based on their numbers., and we have a ranking of the players based on the scouts’ preferences. We mix them together and come up with one list.

“If there’s something that’s out of whack… sometimes the numbers come out really weird. A player might be ranked very high over (another) player, and you know for a fact that the player who is ranked lower is better. Sometimes numbers work that way, so there are bugs that have to be fixed along the way.”

Based on Avila’s description, the sauce in question is closer to Chipotle than it is to Carolina Reaper or Trinidad Scorpion.

 

Share this post


Link to post
Share on other sites

Avila is really, really terrible at messaging. Like the complete opposite of DD who said nothing. Al talks way too much.

Share this post


Link to post
Share on other sites

I don't get what the problem is.

So they have a system. Where straight numbers are produced. Then they have the old fashioned system. They put them together and come up with one list --- basically the shredder from MLB network. 

But, they've determined some kinks in the system, for example when they put some numbers in (you can't use all of them) and McCann comes out as the best player on the team. You know that's not right so you need to look at it further. 

Make sense to me. 

Share this post


Link to post
Share on other sites
3 hours ago, Keepleyland2 said:

I don't get what the problem is.

So they have a system. Where straight numbers are produced. Then they have the old fashioned system. They put them together and come up with one list --- basically the shredder from MLB network. 

But, they've determined some kinks in the system, for example when they put some numbers in (you can't use all of them) and McCann comes out as the best player on the team. You know that's not right so you need to look at it further. 

Make sense to me. 

I think it makes sense, too. 

The fear is that they end up discounting the numbers that are spit out in favor of gut feelings, even after putting some resources into creating the system. I'd hate to see the next Roy Cullenbine overlooked. :-)

Share this post


Link to post
Share on other sites

I would think any system created is going to have an outlier result here and there.  So that aspect doesn't bother me as long as the majority of results are reasonably consistent.

I just wonder what attributes and skills they place a heavy value on.

Share this post


Link to post
Share on other sites

Well, Nuke's scared because his eyelids are jammed and his old man's here. We need a live... is it a live rooster?  We need a live rooster to take the curse off Jose's glove and nobody seems to know what to get Millie or Jimmy for their wedding present.  Is that about right?

We're dealing with a lot of ****.

  • Like 1
  • Haha 1

Share this post


Link to post
Share on other sites
1 hour ago, Mr. Bigglesworth said:

I suppose I don't understand what the criticism is here?

People just look to criticize him at every turn.  I do advanced analytic work at my job and my bosses don't understand. They trust me, empower me, and supervise me. That's effective. I don't know why a 30+ year scout should have to become engrossed in the analytics, if he has a system in place and empowers the people to do properly utilize it, who cares if he knows what he's talking about?

  • Like 1

Share this post


Link to post
Share on other sites

What he described is basically any analytical system in the world... "you put data in and it comes out and we study it"

Sounds like my job.

As I read that I was thinking of the Planet Fitness commercial... "I pick things up. I put them down"

Looks like he's hired a bunch of smart guys over the past few years.  Hopefully they listen to them.  I would think someone like the guy from Apple wouldn't stick around if he didn't think his work would have merit or a payoff.  These kinds of people are not those who just show up for a paycheck.  

Regarding the outlier.... that's not always a flaw in the systems.... it could be a flaw in measurement.  But I don't expect him to get into any of that.  

Share this post


Link to post
Share on other sites

I would suggest there will always be a few guys who fit the things they are looking for statistically, but is not as attractive as a prospect otherwise.  That doesn't mean the system doesn't work, rather, it means someone just happens to fill a couple of key stats or metrics well and for various reasons it is believed he just won't provide nearly as much value long term as those stats would normally suggest.

 

An example of a guy who fills some key stats well is Jim Rice, who had remarkably good triple crown counting stats for his career, and got into the Hall of Fame as a consequence.   But if you look a little deeper, you will see a guy who had a bunch of RBI in large part because he hit in the middle of the line-up, did not walk but batted after batters who did get on.  He was a guy who hit a bunch of HR in Fenway, but hit a pedestrian number of HR in other parks.  He had one of the highest GIDP rates in the history of baseball, but those extra outs don't get counted against him, because who looks at GIDP rates?  Defensively, Jim Rice was, well, not good.

Was Jim Rice a HoF caliper player, or was he a very good player that probably doesn't belong in the HoF despite having HoF numbers in a key set of statistics?

Sometimes a guy fits, or doesn't fit, the metrics well, and his value is not captured accurately.

Share this post


Link to post
Share on other sites

Last week..... I got into a rabbit hole on the internet that fits something here.

A friend posted the Boston Red Sox media guide.  I started to look at during my lunch break.  Among other things I noticed they had 5 people listed who work in behavioral and mental health.  A whole host of systems/IT guys and it wasn't obviously clear which ones were for baseball operations and which ones were just organizational operations (The kind any company would have).  I also noticed Ron Roenicki as a coach.  I remembered Ernie Harwell's call of the 84 clincher and he says "Roenicki off the bag at first..." and I wasn't sure if it was Gary or Ron.  So I looked up Ron.  It was him.  I also noticed that Ron was drafted 5 times.  Twice in 76 including in the 3rd round of the January draft by the Tigers in the same year they drafted like 3 future HOFers, (tram/morris/Ozzie)  (has any team ever drafted 3 HOFers in the same year?).  Finally he signed with the Dodgers after being drafted in the 1st round in 77.

A guy like Ron Roenicke was heavily sought after in the mid 70's by clubs and amounted to pretty  much **** at the ML level.

 

Share this post


Link to post
Share on other sites

The Tigers had about a five year stretch where they had the draft thing down.  And then they became about as bad about it as a team could for about 20 years.

Danny Goodwin was drafted #1 overall twice.  David Clyde was perhaps the highest rated prospect of the 2nd half of the 20th century.  Todd Van Poppel.

Share this post


Link to post
Share on other sites
32 minutes ago, Oblong said:

Last week..... I got into a rabbit hole on the internet that fits something here.

A friend posted the Boston Red Sox media guide.  I started to look at during my lunch break.  Among other things I noticed they had 5 people listed who work in behavioral and mental health.  A whole host of systems/IT guys and it wasn't obviously clear which ones were for baseball operations and which ones were just organizational operations (The kind any company would have).  I also noticed Ron Roenicki as a coach.  I remembered Ernie Harwell's call of the 84 clincher and he says "Roenicki off the bag at first..." and I wasn't sure if it was Gary or Ron.  So I looked up Ron.  It was him.  I also noticed that Ron was drafted 5 times.  Twice in 76 including in the 3rd round of the January draft by the Tigers in the same year they drafted like 3 future HOFers, (tram/morris/Ozzie)  (has any team ever drafted 3 HOFers in the same year?).  Finally he signed with the Dodgers after being drafted in the 1st round in 77.

A guy like Ron Roenicke was heavily sought after in the mid 70's by clubs and amounted to pretty  much **** at the ML level.

 

Drafts in January and June back then.  Seems kind of ridiculous, but I guess it probably made some sense at the time.

Share this post


Link to post
Share on other sites

I've seen Avila say this kind of thing before: that the Tigers take scouting ratings, plug them into the system, and weight these ratings against actual on-field performance to come up with some blended rating, and then disregard the results they don't agree with.

That sounds reasonable on its face, but the issue I have with this is that they are taking subjective numbers submitted by scouts, who as people inevitably have different internalized numerical standards for rating players, and blending them with objective on-field performances to come up with some bottom line assessment. I don't think they should be blending subjectivity with objectivity to come up with a synthetic result and pretending it's data science, when they admit they don't even trust the results that don't comport with their own views.

This is not the same as using scouting reports as a tiebreaker to analytics rankings, which I'm completely fine with; or using analytics as a tiebreaker to scouting rankings, which I am far less fine with but is at least a defensible strategy rooted in consistency. This is insinuating subjectivity into a process that's designed for objectivity. I think the analytics and the scouting reports should be regarded separately, and this is in part why I believe the Tigers aren't taking analytics seriously. But I guess that's to be expected when the general manager is all at once the son of a renowned scout, an ex-scouting director himself, and a member of the Professional Baseball Scouts Hall of Fame. 

Share this post


Link to post
Share on other sites

If I am reading it correctly, the system creates a rank order based on scouting reports, a rank order based on statistics, and a rank order based on the two blended.

Share this post


Link to post
Share on other sites

I wonder if there's a sure fire, can't miss prospect for the #1 overall pick this June? That would be awesome. ANY position.

Share this post


Link to post
Share on other sites

And FWIW, based on my career experience, I have seen plenty of engineering design decisions made because a study or regression analysis said to do X, even though it ran counter to design experience / what was considered design best practice(s).

Far more often than not, those decisions proved to be a bad decision.

Statistical analyses are only as good as the data, and assumptions about the data, that go into it.

Share this post


Link to post
Share on other sites
27 minutes ago, Mr. Bigglesworth said:

If I am reading it correctly, the system creates a rank order based on scouting reports, a rank order based on statistics, and a rank order based on the two blended.

I read it as, they rank order on one and rank order on the other, and then combine them to rank order them on both blended, so they are looking at one blended bottom line number considered on its own, rather than three bottom line numbers considered as a group. I don't know that this is correct, but that's how I read it.

None of us can see behind the curtain how they are actually employing analytics in their decision-making, so we'll all believe what we want to about it. If you want to believe they do so based on a way that avoids the bad decisions you saw in your own experience, that's as valid as anything else.

Share this post


Link to post
Share on other sites
17 minutes ago, Mr. Bigglesworth said:

And FWIW, based on my career experience, I have seen plenty of engineering design decisions made because a study or regression analysis said to do X, even though it ran counter to design experience / what was considered design best practice(s).

Far more often than not, those decisions proved to be a bad decision.

Statistical analyses are only as good as the data, and assumptions about the data, that go into it.

Yup - 'corner cases', extrapolation beyond the range of the data set - all kinds of pitfalls in simulations and correlation that the end user may not only not understand well, but may not even have the information to evaluate if he did. It's amazing stuff when it shows you something you would never have seen any other way, but nothing is foolproof.

Share this post


Link to post
Share on other sites

Not directly related, but I am reminded of the apocryphal military quote, contained in orders from a staff officer to the meteorological department who are protesting against having to provide long-range forecasts:  "The General realizes that the weather forecasts are useless, but he needs them for the invasion plan."

Share this post


Link to post
Share on other sites

I think we're reading too much into his quote or description.

He was being very generic.  I think his point was "We use scouting data and analytical data and try to put the two together for a ranking system.  We're doing something we haven't always done here...".

It's something I imagine 29 other clubs do too.   They just have different methodologies.  

Share this post


Link to post
Share on other sites
42 minutes ago, chasfh said:

None of us can see behind the curtain

This would be the relevant bit.

All we really know is the system creates a rank order based on scouting reports, a rank order based on statistics, and a rank order based on the two blended, presuming Avila wasn't being misleading.

What they do with those lists is speculation.

Share this post


Link to post
Share on other sites
1 minute ago, Mr. Bigglesworth said:

This would be the relevant bit.

All we really know is the system creates a rank order based on scouting reports, a rank order based on statistics, and a rank order based on the two blended, presuming Avila wasn't being misleading.

What they do with those lists is speculation.

They then turn to David Chadd and ask "Do you know any of these guys personally?"  Then they sign that player.

  • Haha 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×