67 lines (49 loc) · 5.45 KB

LLM for codegen paper list

Papers

Contributed by Xing Ma and Yangjie Zhou

Codegen Benchmark

Conf.	Title	Key Words
ICML'25	KernelBench: Can LLMs Write Efficient GPU Kernels?	Cuda code
ACL'25	Tritonbench: Benchmarking large language model capabilities for generating triton operators	Triton code
arXiv'21	Evaluating Large Language Models Trained on Code	Fisrt codegen benchmark, python, pass@k
COLM'24	Evaluating Language Models for Efficient Code Generation	Code efficiency
ICLR'23	CODEGEN: AN OPEN LARGE LANGUAGE MODEL FOR CODE WITH MULTI-TURN PROGRAM SYNTHESIS	Multi-turn codgen
ICLR'25	BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions	Calling external libraries, external tools
ICLR'24	SWE-bench: Can Language Models Resolve Real-World GitHub Issues?	Real-World
arXiv'23	VerilogEval: Evaluating Large Language Models for Verilog Code Generation	Verilog
ICSE '24	CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Models
Nips'24	Mercury: A Code Efficiency Benchmark for Code Large Language Models	New metric
Nips'24	EffiBench: Benchmarking the Efficiency of Automatically Generated Code	Efficiency-critical coding problems

Prompt-based codgen

Conf.	Title	Key Words
DAC'25	MAGE: A Multi-Agent Engine for Automated RTL Code Generation	RTL, multi-agent
ACL-finding'24	Iterative Refinement of Project-Level Code Context for Precise Code Generation with Compiler Feedback	Project-Level Code
AAAI'25	QiMeng-GEMM: Automatically Generating High-Performance Matrix Multiplication Code by Exploiting Large Language Models	Performance hint
ACL-finding'25	QiMeng-Attention: SOTA Attention Operator is generated by SOTA Attention Algorithm	LLM-friendly IR
DAC'25	ReChisel: Effective Automatic Chisel Code Generation by LLM with Reflection	Chisel Code

Training-based codegen

Conf.	Title	Key Words
arXiv'25	Kevin: Multi-Turn RL for Generating CUDA Kernels	RL
arXiv'25	CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning

Code optimization

Conf.	Title	Key Words
ICLR'24	LEARNING PERFORMANCE-IMPROVING CODE EDITS	high-level optimizations
arXiv'25	Astra: A Multi-Agent System for GPU Kernel Performance Optimization	Multi-agent, SGL code
NIPS'24	Don’t Transform the Code, Code the Transforms: Towards Precise Code Rewriting using LLMs	Ast rewrite

LLM for compiler

Conf.	Title	Key Words
CGO'25	VEGA: Automatically Generating Compiler Backends using a Pre-trained Transformer Model	AI-Generated Compilers, function template
NIPS'25	ComBack: A Versatile Dataset for Enhancing Compiler Backend Development Efficiency	Dataset, Compiler backends
OSDI'25	QiMeng-Xpiler: Transcompiling Tensor Programs for Deep Learning Systems with a Neural-Symbolic Approach	Neural-Symbolic

Other

Conf.	Title	Key Words
HPDC '25	Can Large Language Models Predict Parallel Code Performance?

CodeLLMPaper(github仓库) https://github.com/PurCL/CodeLLMPaper?tab=readme-ov-file#a-venues