Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.
Abstract: The following topics are dealt with: computer games; learning (artificial intelligence); tree searching; Monte Carlo methods; multi-agent systems; evolutionary computation; game theory; ...