One topic that seems to bubble up around MMAPredictions among the users is the issue of MMA fans here on the site who predict just UFC events and those who predict on a wide range of MMA events across different MMA promoters. As the owner of MMAPredictions, I wanted to give my thoughts on the matter and hopefully shed some light on why the site is set up the way it is.
How do things currently work?
Just to review, we should talk about what it takes to be ranked on the site. Currently to be ranked and be eligible to be listed among the twenty best predictors on the Top Predictors page, you must satisfy just two requirements:
- You must have predicted at least four events
- You must have predicted on one event in the last two months.
The idea behind these is very simple. The first criteria is used to prevent someone from coming on the site and getting 100% on their first event and being ranked as a number one predictor because their overall accuracy for all time is 100%. That’d be lame and I think we’d have a ton of upset users. So we had to set a threshold that would show a little effort put into predicting MMA events over time, and reduce the likelihood that someone gets lucky once (twice, and even less likely, three times) and dominates the rankings. So we decided after four events folks could be considered worthy of being compared to their peers, who have also done the same. To be honest it was an arbitrary decision we made after trying to balance what new users would have the patience to wait for before being ranked, as well as what would introduce a decent shade of difficulty. So we settled on having to predict on four events before being ranked.
The second condition - that you’ve predicted in the last two months - is to avoid another unpleasant scenario. For the sake of argument, imagine if JoeUser comes along and predicts perfectly for a while, getting 100% of all his MMA fight predictions correct, and captures the number one spot in the rankings. Then for whatever reason imagine JoeUser never comes back to the site again. It would be disheartening for the group to know that the number one spot would never be available. So we put a “freshness” rule on the conditions to be ranked, ensuring that those who are ranked have been around and actively predicting recently.
Taking things back a step, it is a good time to mention that one of our guiding principles on the site is simplicity. We try our best to make it intuitive to use. We try to avoid imposing rules. No crazy point systems to learn. The simpler the better.
Who comes to MMAPredictions?
In general, we can talk about two basic categories of mixed martial arts fan and, by extension, two categories of MMAPredictions users. There are those who are just learning about the sport of mixed martial arts. They enjoy MMA, have probably been exposed to it recently from friends or television, and more than likely their knowledge extends to the most popular MMA promotion today, the Ultimate Fighting Championship.
Then there are the MMA fans who have been watching the sport for years. In varying degrees, they are familiar with hundreds of MMA fighters and handfuls of MMA promotions. Yes, the term couldn’t be avoided for long: we’re talking about the “hard core” fight fans.
Why do people come to MMAPredictions?
I think people come to MMAPredictions for a variety of reasons. Some like to test themselves and see how much they know about MMA, and they like that the site allows them to track how well they predict fight cards over time. Those folks could care less about their peers and have fun tracking their successes and failures with tools like the graphs which display one’s prediction accuracy average on their user profiles. (For mine, click here: Caleb.)
Another set of folks enjoy competition. These folks want to know how they rank against other users here on MMAPredictions to show that they know more than the others, which is what the topic of this post is about: the system used to compare MMAPredictions’ users.
Let me show you a pair of emails I received recently from a user on the site, richc.
September 25, 2007, at 4:01 PM
I have a suggestion for the rankings system. Could we go by a system for people who only want to, for example just want to predict UFC fights, have an over all leader in UFC predictions category or an over all in any other organizations. Then have a category ranking system for hard core fans, who want to predict all the fight cards. Kick the idea around some, maybe you can add to it or come up with something better. Thanks again for your time.
richc
On Oct 16, 2007, at 4:15 PM
caleb,
several weeks ago i e-mailed you a suggestion to make the ranking systems a little more fair. you never replied to this or have not even made the subject itself “public” on the web site. why not ? please clarify this to me. i understand you are very busy, but this subject SHOULD BE adressed to all members, not just between you and i. thanks for your time. please reply.
richc
First things first: I apologize for not responding immediately to your email, richc. I am usually very good about that.
These emails embody a sentiment that I suspect a number of folks have here on the site. Hard core fans who predict on lots of fight cards outside the UFC have it harder on their stats. There is no doubt about it. Since the site’s inception, the average prediction accuracy of users on UFC events tends to be higher than other events. The data shows that users do not do as well promoting events from organizations like Strikeforce, Cage Rage, K-1, and EliteXC.
[There are a couple of reasons that UFC events are easier to predict correctly (or appear to be when looking at the data collected in our brief time online). Perhaps fans know more about the fighters because they’ve been exposed to them more and know more about their backgrounds and fighting styles. Perhaps the UFC’s matchmaking is not as rigorous as other promotions, making lopsided fights where one fighter is easy to predict to be the victor. Whatever the reason, we will attempt to examine it in more detail in a piece we plan on putting together next July after MMAPredictions’ first anniversary and we have a year’s worth of data to examine. Keep an eye out for it.]
So now that we have a little background, let’s answer the question:
Why is there no UFC-only prediction system on MMAPredictions?
It’s true: we have folks who “take a hit” on their prediction accuracy because they enjoy a wide range of MMA and predict on lots of cards. The more one predicts, the closer your average accuracy will resemble the entire population’s average accuracy (see the Law of Large Numbers). Ironically the people who are the most adamant about the sport and put in the most time on MMAPredictions have it the hardest if they want to be compared to their peers. If we create a new “division” of users who predict on UFC only events, it would mean a few things. Here are a few reasons I can think of to keep the current system which compares all MMA fans together.
- Creating separate prediction divisions or leagues would create a culture of “us” and “them”, and it wouldn’t take long for the climate of antagonism to develop between “the noobs” who elect to only predict UFC events and the “self righteous know-it-alls” who predict on a broader spectrum of events. See? I’m name-calling already! And I am the nice guy around here
- What if I have only predicted on UFC events for my entire time on MMAPredictions and then I predict on an IFL event like the upcoming IFL 2007 Grand Prix Round 1? Can I ever go back to being a UFC-only predictor? Do I suddenly develop a UFC prediction accuracy average and a non-UFC prediction accuracy average? Operationally the mechanics of defining and managing two groups of predictors is a nightmare.
- I know this is hard to believe, but the possibility exists that someday the UFC goes away. I doubt Japanese fans - or many fans anywhere for that matter - would ever have predicted that Pride FC, a company which packed more than 67,000 fans into an arena for mixed martial arts fights, would ever go away. These things happen. No, I am not saying the UFC is on it’s way downhill. It could very well become like the NFL and be the only major entity which embodies its respective sport. However as we’ve learned from Pride, it is still very early in MMA history and even the largest pillars of the sport are vulnerable to collapse (let’s not forget the long, ever-growing list of UFC’s competitors which would love to see nothing else!). In that scenario in which the UFC fails - albeit admittedly hard to imagine from where we sit today - does MMAPredictions then do away with its UFC-only division and create a new one built around its successor? And how would that affect the historical rankings of the previous UFC-only division? Are the two comparable?
Lastly, we know that it is possible to do well under the current MMAPredictions ranking system even when you predict outside the UFC. Check out this table, which shows the current top five predictors on the site, and displays how many of their last five predictions came from outside the UFC. (If it’s too small to read, click here or the table itself for a bigger version)
What we see here is that even the people who are doing the best on the site venture into promotions outside the UFC and manage to do well. All of our top five predictors have at some point decided to go outside the UFC to predict.
I think at the moment the system works very well most of the time. I am open to suggestions on different ways to do things, so please keep them coming. Please recall that whatever it is, it mustn’t be elaborate. If there were a Steve Ballmer-like video of me going crazy, I’d be saying, “Simplicity simplicity simplicity simplicity”. The more things there are to learn, the more time someone has to working to understand the site instead of enjoying it.
Remember, all hard core fans started somewhere. In 18 months the folks who today only predict UFC fight outcomes because it’s all they know will be looking at the next crop of new MMA fans and some will sneer at how little the new fans know.
And for those who think the UFC is always predictable, don’t forget September’s UFC 76: Knockout, which caught users by surprise faster than Anderson Silva’s hands work. According to its detailed prediction results the average of all our users’ prediction accuracies was just 42.6%! It’s safe to say that even if a predictor were to stick to an only-UFC strategy, sooner or later they would get caught as well.
Most of all, remember that this is just for fun. At the moment we have no plans to give away anything of super high value in our contests, so there’s nothing huge at stake. Relax, enjoy MMA, and enjoy predicting MMA.
- Caleb 
Got comments? Stick them in our MMA Forum or contact me directly!
One last thing: 