Auditor Leaderboard

Auditors submitted to the private evaluation of the Auditor Benchmark are listed below. Hover over any column header to see what the metric measures, and click it to sort by that metric; the best score in each column is highlighted.

Note that the budget of the API calls available to an Auditor is limited, so to illustrate the importance of choosing the right scenarios, we showcase the results of LLM-judges subject to different budgets. The importance of smart use of the budget is highlighted below.