Benchmarking of machine learning systems and artificial intelligence systems provide a reliable, transparent and standardised way to compare performance of different parameters to handle a task. Benchmarks is an important ingredient in catalysing research and innovation by allowing comparison of different ideas and approaches.

Innovation in Indian legal NLP has largely happened in commercial contexts and in closed ecosystem. There are currently no NLP benchmarks for Indian legal system unlike other countries such as China, US and other EU countries. Since India follows the Common Law system i.e. the case laws or the precedents determine the applicable law. This limits the utility of the currently available legal dataset from other countries following Civil laws.

This initiative focuses on building benchmarks for Indian legal NLP on several key AI tasks such as Named Entity Recognition, Judgement Summarisation and Question Answering etc.

A key feature of this initiative is building open datasets by engaging students. There is a micro course that has been created for students to learn about the working of legal AI systems while building open datasets by annotating the judgements from Supreme Court, High Court and District Courts in India. If you are keen on contributing to this initiative please click on this link to express interest.

Events Archive