Elite 1v1 skill rating system

[Lulgrim]

This has changed somewhat so I'm replacing the original message. Discussion of Monthly Rumble / generic ranking and skill rating stuff has been going on here.

The rating system is now in place and used for tournament matches (automatically) and any other matches as necessary (submitted as described below). We use the Glicko rating system (the simpler Glicko-1 version), about which you can read below:

• A short explanation on our website
• A longer explanation by Michalis Kaloumenos
• The original explanation by Mark Glickman (creator)

There is no automatch, of course, and the rating doesn't update immediately as it requires manual processing. On the other hand, we can expand the available data and statistics a lot (see player profile/history pages below).

What matches will be rated?

1. Tournament matches (starting from Monthly Rumble #1)
2. Any 1v1 standard settings* game between registered players where both players agree in the in-game chat to make it a ranked match
3. Replay must be submitted for a match to be included

* Standard resources, VP victory, 500 VPs

How do I submit matches?

1. Start a thread* on our "Replays" forum and upload the replay as an attachment OR
2. Upload the replay to GameReplays, start a thread on our "Replays" forum and post a link to it OR
3. E-mail the replay (or the link) to me (my username @ this website)

* Include at least Elite version number and player names in the title

Then what?

We get mathematical skill estimate for each player; the system allows keeping a match history that shows all your ranked games with date, opponent, their rating, heroes, map, the scoreboard data, your career hero-specific stats, global hero/faction stats, and all sorts of shit.

See it yourself here:

Skill ratings list: http://www.dawnofwar.info/index.php?page=ratings
Player profile page: http://www.dawnofwar.info/index.php?pag ... &player=90
MoarStats™ extended stats page: http://www.dawnofwar.info/index.php?pag ... stats=moar

[Lulgrim]

The rating system is updated with Monthly Rumble #4 games. I have also added visual win ratio graphs and a skill rating history graph.

[Lulgrim]

Global stats now have some faction economy thingies: http://www.dawnofwar.info/index.php?pag ... ats=global

[Bahamut]

pretty nice stats. Feed it 1k+ games and you'll be able to actually judge balance by those stats

[Lulgrim]

Gonna take a year's Monthly Rumbles (even assuming 100% coverage, as about 25% of replays have been missing in the first four) to hit 1000, so, again, non-tournament "srs" games would give us data faster.

[Lulgrim]

Player details now show some extra hero-specific numbers (I'm still considering what to put there) when you click on portraits (I'll pretty it up in the future but for now I just put a subtitle saying you can click them).

http://www.dawnofwar.info/index.php?pag ... stats=moar

[Tex]

I will start contributing games to this, sounds cool!

[Lulgrim]

Updated my links in this thread.

[Lulgrim]

With the Emperor's guidance I have made many litanies to the machine spirit and it has agreed to prepare the chapter's skill ratings automatically.

In other words, as long as I don't just lose the list with names and winners, I can rebuild the whole match history from the beginning with Glicko ratings and RDs, accounting for idle periods, with the figurative push of a button. Or I can make it auto-update as well, which is not relevant quite yet but might be later (more below). At the moment, it allows us to tweak calculation parameters and get the sort of skill estimation best suited to our player population and match frequency.

Thomas has said he can look into the possibility of auto-uploading match results from the game, can't really say how this will end up but at least it's a possibility. I'm sure you agree this would make the auto-updating stats system srs pr0.

If there are players interested in 1v1 skill rating systems and/or statistics (I know fuck all about statistics, really) your input is welcome.

Edit: As it no longer requires much manual work, I am reconsidering COCA games to be included in the match history as well.

[Uncle Milty]

This is awesome! All of this.

[Tex]

Okay, sorry if its right there in front of me, but I am doing ranked games now and I just need to know what email to send them to or w/e.

[Lulgrim]

I should write something about that again... Well, you can 1) post on the Replays forum here and upload attachment, 2) upload on GR and post a link on the Replays forum, or 3) try emailing it to lulgrim @ this website.

Putting reps/links on the forum has the merit of making them available for discussion & casters. If you do, maybe title threads like

[2.2 b7] Mario (Plumber) vs Luigi (Plumber) insane mirror 45 mins omfg1111

or whatever, the version numbers are important anyway.

[Asmon]

[2.2 b7] Mario (Plumber) vs Luigi (Plumber) insane mirror 45 mins omfg1111

This match-up favors Mario so much it's not even funny.

[xerrol nanoha]

[2.2 b7] Mario (Plumber) vs Luigi (Plumber) insane mirror 45 mins omfg1111

This match-up favors Mario so much it's not even funny.

Luigi is sleeper OP, better watch out!

[Lulgrim]

For those interested, here is a calculation run of Rumble 1-4 games. Feel free to comment on the RD lability and whether the ratings seem reasonable to you (you'll have to read at least the short description of the system in advance).

https://www.dropbox.com/s/58uuapzc8t8ea ... 0c15.8.txt

[Lulgrim]

The match log now includes games from COCA (the ones that had replays available) and the "ranked weekend" games from Nov 23-24.

https://dl.dropboxusercontent.com/u/709 ... 24-c15.txt

[Ace of Swords]

I was thinking about a little thing, perhaps here on forums could you specify that the rating under the names of the users is for 1v1s?

[Lulgrim]

Well we have no other kind...

[Ace of Swords]

Yeah I guess, but I was wondering it might look like somekind of forum using good rep rating or something around these lines rather than in-game elo to a user that doesn't know whats going on, that's all, nothing needed really just a clichè thing.

[ben1993]

How can I get myself added onto the 1v1 ranking table?

[Indrid]

Play ranked 1v1 games and submit the replays to the replay forum. Try to seek out other people on the ladder to play. You can use this tool to choose maps, but you don't have to.

Entering the MRT would also help since they are ranked games.

[Templar$Ghost]

How can I get myself added onto the 1v1 ranking table?

I would defiantly recommend that you play people on the ladder. I already had four of my games disbanded

[SirSid]

Very cool i will try to use this as much as i can

[Lulgrim]

Additions:

(1) Players must be registered on the website so that we have some sort of fixed ID for them.

(2) I have reordered the ranking based on credible rating (CR in short) which places you according to the lower 67% credible limit (CR = rating - RD) instead of raw, unadjusted rating (which allowed players with high RD silly high on the list). We can change this to the 95% credible limit (2 * RD) if it still seems too easy.

[Ace of Swords]

Well, well, this is shaping to be better and better everyday, good job lulgrim.

[Lulgrim]

Are there more stats I should put on the main rating/ranking list? I thought the win % was a bit redundant, and I ditched the raw ratings, but now it does look a bit empty.

[Dark Riku]

I actually liked the win percentage.

[Lulgrim]

I will redesign the list as it looks lame anyway.

In other news, I have been thinking about a way to determine best hero. I threw around some formulas but the they seem a bit unsatisfactory... Now I'm considering the skill rating angle, could the strongest hero be the one who has defeated the highest rated (CR) opponent, or the average of 3 highest opponent ratings? Would there be issues in this?

[Dark Riku]

Some examples might help here. It's too abstract for me atm
What happens if you beat your highest ranked opponent with more than 1 hero for example?

[Torpid]

Or that those opponents just happen to main races that are weak against you - Orks and CS are weak to CL, but both very prominent high level 1v1 heroes.