DL2010.csv

Already 77 DL moves, and the season is less than 4 days old (includes spring training injuries and DL transactions from off season and previous season surgeries). 52 (~68%) are pitchers.

Sammy Gervacio isn’t included. Yet.

Enjoy!

http://insider.espn.go.com/espn/blog/index?entryID=4986916&name=keating_peter

No time now, but Peter posts pages from the MLB Redbook, kind of my personal Dead Sea Scrolls. Very excited. He even lays out a number of subjects in need of study because, he claims, no one in MLB will undertake them.

I’ll post his list here later. I’m off to San Francisco to taste overpriced Burgundy that smells of wet donkey.

Prost!

The database is referenced by mlbID for easy pitchFX analysis and research. Particularly useful for folks doing classification using K-means analysis, each pitcher has a field showing the number of pitches in his repertoire. Sven’s pitch types are also included (ex: fastball (91-93)).

I’m grateful to Sven for making his scouting information publicly available, which in turn makes research like this possible.

CSV and mySQL dump archive.

Enjoy!

They are updated nightly, and are current as of tonight.

Enjoy!

I am deeply appreciative of the incredible support shown for Rotobase. I think the potential for this little player evaluation and information aggregator really is sky high thanks to the thoughtfulness and foresight of so many of the very best information providers.

Thanks again Will.

Background:

In 2006 Joseph Adler was the first to describe how to download Gameday data from mlb.com’s xml files. Mike Fast later updated the data structure and provided some excellent Perl scripts to to grab pitch f/x data and parse it into a mysql database.

The problem, if you are a Mac user, is that the DBI (plug ins that allow Perl and Python to insert data into a mysql database) are insanely difficult to work with. Indeed, one of the great frustrations of being a Mac user is getting any scripting language to work with a mysql database.

PHP, which this tool is written in, is free of any of those issues. My tool also allows the user to choose the directory to download and parse, as well as the year from an easy to use GUI interface, making the process painless for baseball analysts. No time has been spent making it pretty, but it works and works well.

==================================

If you already have an *AMP stack set up on your system, just download the files here, change the config.inc in the includes directory to reflect your mysql database settings and away you go.

For folks who are still trying to get their own saber system up and running on their Mac (or any other system really – PHP is cross platform – but this tutorial is Mac specific) you’ll need to download a few items first.

1. Go to the MAMP download page and grab the latest version.

2. Download Sequel Pro.

3. Install both. Simply drag and drop into your Applications folder.

4. Configure MAMP so when you begin extracting data, you don’t get time outs. To do this, open your MAMP folder in your Applications directory after install, and find the following file. Here is the full path to avoid confusion:

/Applications/MAMP/conf/php5/php.ini

Open the file with TextEdit, TextWrangler or TextMate (or even VI if you are a command line guy). Whatever you have handy. The line you are looking for is “max_execution_time”. The quickest way to find it is to hit command-f and search for the term. It will be set to 60 or something similar. Change it to 0. Save and exit.

5. Start MAMP by double clicking on the MAMP.app in the MAMP folder. It will bring up a small window that looks like this (though the green lights should be red for you. If not press “stop servers”):

Press the “Preferences” button, then click the “Ports” tab and hit “Set to default apache and mysql ports”. It should look like this when you are done:

Finally, hit OK and go back to the main screen and press “Start Servers”. The red lights will turn green and you may be asked for your password.

6. Now download the Rotobase Extraction tool. Open the rotobase folder, then the includes folder and open the config.inc.

The full path is: rotobase/includes/config.inc

You’ll note that the user name and password are root, which are the user names and passwords for the default installation of MAMP. It should work out of the box for you unless you make any changes or choose not to follow the instructions below. If you do, this is the file you will need to modify to get things working, especially the “dbname”, which should be set to the name you gave your database.

7. Launch Sequel Pro which you have already downloaded and installed. Launch it and you will be greeted by a login screen. It will look something like this:

Navigate to the “Socket” tab. Name the database whatever you want. I chose Baseball. For username and password enter “root” without the quotes. Leave database blank. Copy and paste the following path into the Socket field:

/Applications/MAMP/tmp/mysql/mysql.sock

Hit “Add to favorites” to save your settings for the next time you launch Sequel Pro.

Now hit connect.

8. In the upper left hand corner you’ll need to add a database. Here is what the selection window looks like:

Click Add database, and name it “milb” without the quotes. NOTE: if you name it ANYTHING else, you will need to modify the file in step 6.

Hit Ok.

9. Now you will need to import the database structure. I have provided a pre-populated admin table with login credentials for the tool already for you, so it is important that you get this part correct.

Go to File > Import:

And browse to wherever you downloaded the Extractor tool.

rotobase/rotobase.sql is the file you want. It should look like this:

Hit open and the database tables should be imported. You can click through them to verify that all but tbladmin is empty.

10. The default installation directory for PHP scripts in MAMP is

/Applications/MAMP/htdocs/

Copy the rotobase folder to that location.

11. You are almost finished! Open your browser of choice and enter the following URL

http://localhost/rotobase

You will be greeted by a login screen like the following:

Enter “admin” without the quotes for Username.
Enter “password” without the quotes for Password.

Hit Login. You will be greeted by a config screen where you can change your username and email address. I HIGHLY recommend adding your email address. Extractions can take days, and the tool will email you when it is completed.

In the upper left of the screen click on the Extract tab. You will be presented with the following screen:

From here you can choose the league data you would like to download. If you are only interested in MLB Pitch f/x data, choose the mlb directory and choose a year from 2007-2010.

The minor league directories (aaa, rok) correspond to MILB leagues. Play by play data are available for most back to 2005. Chose a directory and a year and hit extract.

You’ll be waiting a while, so to troubleshoot I recommend downloading 2010 mlb data to test.

You can verify things are working by Loading up Sequel Pro, going to the games, atbats or pitches tables and refreshing to check that the data are being inserted.

NOTE: Whenever you download a new directory (aaa, mlb, rok) you will need to create a new database to hold the data. When you do you will have to give it a unique and descriptive name, like “AAA”. For this to work correctly, you will need to follow the directions in Step 6 to change the database name to the appropriate name (in this example AAA).

==================================

There are only 2 restrictions to using this tool.

1. Please link back to rotobase.com so others can find and enjoy the tool as well.

2. If you improve upon the tool you must make the source code available for others to use and enjoy.

==================================

Hope this helps analysts out there who have been struggling to get minor league play by play or pitch f/x data.

For those who just want a SQL dump, check back in a few weeks as I will provide downloads for minor league raw and MLE data.

Enjoy!

I’ve become increasingly aware of the split between fantasy analysts and pure sabermetricians since I began actively developing and marketing Rotobase to the public. I haven’t personally had any negative experiences with either camp, indeed my experience so far has been universally positive, but I have noted that there is a definite “credibility gap” between the two worlds.

If an analyst uses advanced metrics to shed insight on how to play the former game (fantasy) better, he will, on average, accrue less respect from the community than if he had applied his analysis to the latter game (real baseball). There are a couple of likely reasons for this, none of which hold up under much scrutiny, as I’ll explore below.

First, some context. I make wine and run a vineyard. Why does this matter? It matters because as passionate as I am about baseball and as much fun as I have playing with numbers, I’m twice as passionate about crafting great pinot. And I am not alone. Not by a long shot. There are many, many talented and passionate individuals just like me who make, or want to make wine. Sound familiar?

While doing something you love is a blessing, there are certainly trade-offs. To continue to do the thing I love I’ve had to become adept at marketing, selling, and managing a business. In fact, those three things now constitute my core competency, not making wine.

The reality is, the hardest thing isn’t making a great wine. In fact it’s not even close. The hardest thing, by far, is selling it.

On to baseball.

It seems that one of the main drivers of the credibility gap between fantasy analysts and “real baseball” analysts is something to do with the fact that one is considered frivolous (fantasy) and the other is considered important and “impactful” on real world business (real baseball).

Besides the fact that I loathe biz speak like impactful, I submit the the converse of the above is generally more true.

How many sabermetricians will ever go on to to actually work for one of the 30 major league baseball teams? Answer: very few. How many make a living providing analysis solely on real life baseball for a media company, or pay information site? Again, very few.

The vast, commodious majority do the work they do out of a passion for the game, and the folks who are fortunate enough to be paid for their work often do so at a steep discount or work at it only part time (again, much like wine).

Against this gloomy monetary backdrop are the trade-offs inherent in making your passion your work. You lose autonomy in choosing your projects. Business politics become a factor. Marketing yourself well, and developing the skill, becomes the highest value endeavor you can engage in. All of this comes at the expense of baseball research, which is ostensibly the reason you were attracted to the job in the first place.

And, again, we’re talking about a vanishingly small percentage of saberists here. Most do it for free, out of love. And while that is a beautiful thing, it doesn’t really justify holding a pure baseball analyst in higher regard than a fantasy analyst who is also motivated by the same love of the game.

The real kicker however, is this: while there are only 30 possible places of employment for someone bent on working in the business of major league baseball, there is unlimited potential for the entrepreneur to parlay his or her baseball passion into the business of fantasy. Fantasy Sports Ventures pegs the market at something like 2 billion. I think that overstates the case, as it includes things like internet connection speed upgrades and other ancillary expenditures, but it does illustrate that demand for fantasy products is there, and growing.

You stand a much, much greater shot at turning your passion into a viable business via fantasy than you do in real baseball. In fact, I would submit that you stand to earn more at every level of success in the fantasy sphere verses real baseball, with the exception of general manager. And there are only 30 of those spots to go around.

So a credibility gap that is based on money or real world relevance doesn’t wash, at least from my point of view.

Strangely, there is another reason for the credibility gap that is the polar opposite of the above. Many fantasy business people are seen as hucksters by the community, shilling their snake oil to the uninformed; motivated solely by profit. You’ll get no argument from me that there are aspects of fantasy that reek of sleaze. But I think the same is true for the hot dog and beer vendors at “McAffee” coliseum, especially on Bank of America souvenir blanket day.

Again, I don’t see why the folks who devote their efforts to helping billionaire owners run their clubs full of millionaire players more effectively should have any kind of moral or intellectual high ground compared to folks who devote their efforts to helping Joe six-pack win his fantasy league.

=========================

I’m a big believer in the sharing of data. Analysis, now that you should pay for.

Too often folks on both sides (fantasy and saberiests) hoard data, formulae, and processes (this of course doesn’t apply to the awesome folks who have helped supply data to the community for years – Retrosheet, Lahman, Tango, Smith, Szymborski and companies like Roto Sports Inc. who generously share their data). But the thinking elsewhere is that it will provide them a defensible moat. It never works. A dedicated analyst will find ways to achieve a similar result.

The only way to succeed in any business is to consistently delight your readers/customers and add value by communicating knowledge (or providing a great product) in a frictionless manner. Much better to take the lead in the sharing, earn goodwill through cooperation, and gain customers and brand advocates through genuine affection.

Your true moat is how you treat people, your personality and your passion. Your true moat is your writing style, your insights, and your aesthetic sense.

Data is not your moat.

==========================

Frankly, I would never want to work in real baseball. I just don’t care that much about franchises and their machinations. I don’t get fired up at the prospect of joining a large company that is unable to treat its customers to the type of hands-on, high-touch attention they deserve. I wouldn’t want to sacrifice my autonomy, or my roots. I’m attached to the land on which I live and work. It is one of the true joys of my chosen profession.

No, if I were to try and make a buck in baseball, it would be in fantasy. And there’s no shame in that game.

What do you think? Did I miss something in my analysis. Think I’m full of it? Leave a comment and let’s have a conversation.

Of all the sources of prospect evaluation, Project Prospect is my favorite. Their new Digital Prospect Guide contains not only player blurbs, minor league batted ball splits, and scouting videos embedded right in the file, but they also get out there and talk to the prospects themselves and issue original reporting.

So pleased to be able to integrate some of their insights into Rotobase. Thanks Adam!

Prost!

• If a player has had 3 seasons of 500 AB or more, it will be noted under positives. This recognizes that staying healthy is a skill, and the player deserves some credit.
• If a player has an increasing or decreasing Line Drive trend that is now noted as well. LD% is highly variable, but there is skill associated with it and is the primary driver of BABIP.

I’m still working with the best way to implement the converse of the positive AB note. I’ll post when it goes live.

Enjoy.