IPEval Benchmark

About IPEval

IPEval is a pioneering bilingual Intellectual Property (IP) agency consultation evaluation benchmark, meticulously crafted to assess the competencies of Large Language Models (LLMs) in the intricate domain of intellectual property. This benchmark is the first of its kind, encompassing a diverse spectrum of 2,657 multiple-choice questions that are intricately divided across four major capability dimensions: creation, application, protection, and management. More details can be found in our paper.

Data

Our data can be directly downloaded on Huggingface datasets. Please refer to our github instructions for how to read and use the data.

Citation

@misc{wang2024ipeval,
      title={IPEval: A Bilingual Intellectual Property Agency Consultation Evaluation Benchmark for Large Language Models}, 
      author={Qiyao Wang and Jianguo Huang and Shule Lu and Yuan Lin and Kan Xu and Liang Yang and Hongfei Lin},
      year={2024},
      eprint={2406.12386},
      archivePrefix={arXiv}
}

Contact Us

Have any questions about IPEval? Please contact us at wangqiyao@mail.dlut.edu.cn or create an issue on out github.