I did a 100+ games tournament and calculated the AIs Elo.
These ratings are relative. Please dont expect these Elos to match LiChess Players elo ratings. It just to compare the differences between the engine Ais.
Level 1 AI = ~850
Level 2 AI = ~950
Level 3 AI = ~1050
Level 4 AI = ~1250
Level 5 AI = ~1700
Level 6 AI = ~1900
Level 7 AI = ~2000
Level 8 AI = ~2250
In the near future we will try to collect data for another rating list which will try to compare LiChess members/elo to the AIs elo.
nice work!
(why can't I beat level 1 :[ )
Well, this is sort of a fascinating relief. Even though my speed chess jumps my ELO between 1300 and 1500, I was able to beat a 2250-rated opponent on a few occasions.
But, I spent an infinitely long time thinking on almost every move to win those games. Besides, it's only relative. :P
How did you do the tournament? I don't know much about Elo's system, but did you like, use variously rated chess engines against the software here?
Yes I had engine opponents of various range in rating play at least 10 games against the CraftyAI here.
The calculations were done through formulas.
Here is my list of ratings for the lichess AI compared to lichess players. The lower and mid levels are more accurate.
I would need a higher rated player to get more accurate ratings for the higher levels...
Level 1 AI = ~800
Level 2 AI = ~1000
Level 3 AI = ~1400
Level 4 AI = ~1600
Level 5 AI = ~1700
Level 6 AI = ~1900
Level 7 AI = ~2000
Level 8 AI = ~2200
Now I'm even more curious what method you used here. My only guess is that you're able to filter the user database with Elo charts, for games played against Crafty level [x]. (Well, or it could be
you were just carefully recording recent games against Crafty from all of those anon dudes.)
Good thing thibault made the difficulty level effect more dynamic, or that table would be very different. :D I think at the time it just used (x) Ply search depth, where (x) is the difficulty level
chosen. Because from what I understand, just 3 Ply is enough to depress an expert with modern engines (due to the advances in hash table technology).
The first list.... I played at least 10 games with various AI personalities from Chessmaster Grandmaster Edition against each CraftyAI level. The program suggests opponents and calculates the ratings.
Its fairly accurate.
Repeat for every level. Some of the levels I played more than 10 games to get more accurate results.
The second list... is just my observations vs the players I have played here.
Oh, I see. Heh I assumed when you said "engine opponents" that you meant different computers (well technically...but more agreeably, different personalities of the same computer). Crafty wording.
=D
Never heard of a feature like that, though. The closest thing to that I have is just the Chessmaster LIVE! Xbox 360 arcade game (gives you a rating and points to the personality closest to your Elo).
(But that thing sucks. When I try to play 15-minute games vs. even some 1700 dude, just sits there for the last 10 seconds and times out. Crappy emulation.)
Based on all that data though (and from what I remember switching between the levels), those ratings look to be in just about the right ballpark.
What happened with lichess AI? All levels play like the 8th level.
can someone confirm that?