POPA Reconso far, the best way to measure whether agents are capable of logical thinking, solving puzzles from a to z and fixing themselves is…
so far, the best way to measure whether agents are capable of logical thinking, solving puzzles from a to z and fixing themselves is through ai coding challenges in our opinion, the best benchmark for now is DeepSWE https://deepswe.datacurve.ai/ especially interesting is: ‘tokens generated’ metric as an output, it’s literally an ai-slop meter *astra shows the best results there
The excerpt is quoted from a public Telegram channel, which owns its text.