👋 Welcome to RefineBench — a comprehensive evaluation library for testing refinement capabilities of language models across multiple settings and domains. To reproduce the full results reported in ...

CATArena (Code Agent Tournament Arena) is an open-ended environment where LLMs write executable code agents to battle each other and then learn from each other. CATArena is an engineering-level ...

Abstract: Programming language source code vulnerability mining is crucial to improving the security of software systems, but current research is mostly focused on the C language field, with little ...

Abstract: The quality of modern software relies heavily on the effective use of static code analysis tools. To improve their usefulness, these tools should be evaluated using a framework that ...

Andrej Karpathy stripped down the LLM architecture and loss function to basic mathematical operations. Andrej Karpathy, a former researcher at OpenAI and the founder of AI-native education company ...