Monday, March 14, 2011

How'd we do?

The main goals (and the main strengths) of this site are (1) to provide accurate predictions of end-of-season RPI, and (2) to provide up to the minute current RPI. A secondary goal is to provide predictions of at-large bids and seeds. Up until this season, we were not predicting seeds, just bids. This season, we decided to use the "Dance Card" formulas for seeds in addition to the at-large formulas. In terms of at-large bids, we did a bit worse than seasons past - missing 3 (VCU, USC, and Florida State). However, only one major bracketologist got even 67 out of 68 (Fox Sports). As far as I have seen, nobody got 68. Based on the Bracket Project website, only a handful got 66 or better. Most were 65 or less, so 3 is nothing to be ashamed of - especially considering the fact that this is all done using a simple formula. Typically the Dance Card formula misses no less than 1 or 2, so this season was unusual.

As for seeds, there IS something to be ashamed of. Quite simply, the seed predictions stunk! In the name of objectivity, we wanted to avoid tweaking the seeds and let the formulas do all the work. However, something obviously needs to be done. The plan for next season is probably to abandon the formulas for seeding - or at least to modify them to allow for some "human analysis". We'll stick with the at-large formulas and figure out a better way to seed the teams. At this point, the seeding decisions seem to be more complicated than can be explained by a simple formula.

The big benefit from projecting bids/seeds relative to other bracketologists ought to come EARLIER in the season, rather than LATER. In fact, RPIFORECAST is one of the best at predicting seeds in December/January and February. We have archived all of the major bracketologists' predictions this season and will be determining who were the best and worst EARLY in the season.

Anyway, thanks for visiting, enjoy the tournament, and look for some more exciting things next season!

2 comments:

Evan said...

Thanks. Great job this season. I enjoy the site.

I was wondering about a couple teams on which the Dance Card formula seemed "way off" compared to conventional wisdom for a good chunk of the year: Central Florida stuck around well after they had been written off, and Florida State never got much respect from the DC formula.

Do you think conventional wisdom undervalued UCF b/c of their horrible losing streak relative to their full resume. Or was this just a fluke/quirk in the formula?

Ryan Israelsen said...

Thanks Evan. Central Florida is obviously a different team now than the one that beat Florida way back in Early December. They have the 8th highest non-conference RPI! I think it's a combination of the formula and the streak you mention. It doesn't really distinguish between a team that was hot early and one that has been consistent. it does take into account record over the last 10 games, but it's not a big factor.

Also, remember that these projections are based on running simulations using Jeff Sagarin's PREDICTOR to come up with probabilities of winning future games. If a team like UCF wins a bunch of games early, they are going to be considered more likely to win the games going forward. If that doesn't happen, the probabilities will come down, but only slowly. By the end of the season, it should be fairly accurate, but in the meantime, it may not jive with conventional wisdom.

As for Florida State, They were close. In fact, if you look at Jay Coleman's site, he has them a bit closer than I did. He's using a slightly different version which includes conference affiliation and representation on the committee, but that version also missed 3. I may switch to that formula for at-large bids, but am definitely going to switch up the seeding strategy.

Thanks for visiting!