METR Launches Metric to Assess Cost-Effectiveness of AI Agents vs. Humans

METR has introduced a new metric designed to evaluate the cost-effectiveness of AI agents compared to human labor. This metric, termed the Expenditure Horizon, aims to provide a clearer understanding of when deploying AI becomes more costly than employing humans. Early results from speedrun tests involving the NanoGPT model have been described as underwhelming, indicating that the metric may have limitations. An unverified claim suggests that ‘the metric has blind spots,’ while another states that ‘the newest generation of models could change the picture.’ This development follows recent discussions in the industry regarding the economic implications of AI integration, as highlighted in previous coverage by The Decoder.