ABSTRACT

Artificial Intelligence (AI) has been revolutionizing judicial systems around the world, and the Indian judicial system is no exception. There have been many attempts in the Indian courts to utilize technology for case management, transcription, research, and analysis. While there is great promise in using technologies such as AI for judicial efficiency, transparency, and pendency reduction, several significant legal and ethical questions remain. Issues related to algorithmic bias, accountability, data privacy, and automated decision-making can potentially threaten basic aspects of the rule of law and justice. In a very recent case before the Hon’ble Delhi High Court, Greenopolis Welfare Association vs Narendar Singh (2025)¹ – a case that involved a real situation in terms of a petition with citations that did not actually exist. It was found out that the citations were entirely fictitious – most probably generated using AI technology.

INTRODUCTION

The Indian judiciary system is amongst the biggest in the world, but it faces certain challenges such as pendency, accessibility, and delay in procedure. With the advent of artificial intelligence, AI-based systems have emerged as possible solutions to such challenges by facilitating transcription, research, and analytics. The e-committee of the Supreme Court has taken an active role in the adoption of technology by launching AI-based transcription during Constitution bench proceedings in 2023.

Incorporation of AI into the judicial process, therefore, brings certain issues into perspective, such as: Is the algorithm biased? Is it in line with the fundamental rights enshrined under Article 14², 19³ and 21⁴ of the Constitution of India?

The increasing use of artificial intelligence in courtrooms raises worries that technology may aggravate bias and structural distortions in the legal system. The Supreme Court of India addressed this in the case of Gummadi Usha Rani & Anr. v. Sure Mallikarjuna Rao & Anr.⁵ In this matter originating from Vijayawada, Andhra Pradesh, the apex court ruled that a court relying on AI-generated fake judgments is not simply making an error or mistake but is committing judicial misconduct with serious legal consequences.

Similarly, in international disputes, like the State v. Loomis (2016)⁶, the case of Loomis involved the use of COMPAS, an algorithm-based risk assessment tool developed by a privately owned corporation. Even though the Court upheld its use, it highlighted several issues regarding due process that may arise as a result of the opacity of the algorithms. The case of Loomis remains globally relevant in discussions on black box justice.

In this paper, I analyse some of these issues using the lens of algorithmic justice.

CONCEPT OF ALGORITHMIC JUSTICE

Algorithmic Justice is a term that refers to the use of artificial intelligence and machine learning in committing discrimination and abuse of human rights. It can be argued that the basis of Algorithmic Justice is that there must be an invention that will ensure no discrimination happens to any particular group of people. Algorithmic Justice revolves around the concepts of fairness, transparency, and accountability in regard to automation in the lawmaking process.

JUDICIAL INTERPRETATION AND EMERGING CASE LAW

Indian courts have begun addressing the risks and responsibilities associated with AI.

1. Gummadi Usha Rani v. Sure Mallikarjuna Rao – The Supreme Court (Justice P.S. Narasimha and Justice Alok Aradhe) stayed the trial court’s proceedings. They declared that delivering a judicial decision based on hallucinated AI case law is not merely an error, but constitutes a serious breach of judicial duty
2. Deepak v. Heart & Soul Entertainment Ltd⁷. – the Bombay High Court did not strike down the use of AI entirely, but it strongly condemned and penalised by imposing Rs 50,000 costs on the litigant the submission of unverified, AI-generated legal arguments that cited completely non-existent.
3. Subhash Desai v. Principal Secretary, Governor of Maharashtra⁸ -The Supreme Court of India officially launched its AI-based live transcription for Constitution Bench hearings.
4. Mahesh Kumar Alias Mahesh Bansal v State Of Haryana⁹ – The Punjab and Haryana High Court firmly rejected the use of predictive algorithms for bail hearings, declaring that personal liberty is too fundamental to be subjected to statistical models. The Court emphasized that bail decisions require a careful, individualised evaluation of facts rather than relying on automated probabilities.
5. State v. Loomis (Wisconsin, USA, 2016) -The Court upheld the use of the COMPAS algorithm but warned that opaque systems cannot be the sole basis for sentencing.

STATUTORY AND CONSTITUTIONAL FRAMEWORK

There currently exists no specific law concerning AI in India. The regulatory framework for AI in India consists of an eclectic mix of techno-legal laws wherein digital and criminal laws, as well as the Constitution and policies like the India AI Governance Guidelines, are included.

CONSTITUTIONAL FRAMEWORK

The Indian Constitution indirectly governs AI deployment by mandating that all technologies respect fundamental rights:

• Right to Privacy (Article 21): The Supreme Court of India declared privacy a fundamental right in K.S. Puttaswamy v. Union of India¹⁰. AI models scraping public or private data must operate in compliance with this right.
• Right to Equality (Article 14): Ensures that AI-driven automated decision-making (e.g., in hiring, banking, or governance) is non-biased, fair, and objective.
• Freedom of Speech (Article 19(1)(a)): Balances the creation and dissemination of AI-generated content (including deepfakes) with reasonable restrictions on public order and decency.

STATUTORY FRAMEWORK

Because there is no dedicated AI law, AI offences and liabilities are prosecuted under existing foundational legislation:

1. Digital Personal Data Protection (DPDP) Act, 2023: Regulates the processing of digital personal data. It requires AI developers to obtain explicit user consent, implement data minimisation, and ensure data breach accountability.
2. Information Technology (IT) Act, 2000: The primary cyber law. It imposes penalties for offences such as identity theft, data theft, and the publication of obscene electronic material. It is frequently invoked to address AI-generated deepfakes, morphing, and synthetic misinformation.
3. The Bharatiya Nyaya Sanhita (BNS), 2023: India’s criminal code addresses harms like fraud, cheating, and defamation committed using AI algorithms.

POLICYMAKING & REGULATORY STRUCTURE

The government adopts a “whole-of-government” strategy to allow for rapid innovation in AI while addressing the risks posed by AI to its multi-faceted population:

• India AI Governance Guidelines: This policy by the Ministry of Electronics & IT (MeitY) includes ethical principles or Sutras and guidelines for transparent & accountable deployment of AI.
• Sectoral Guidelines: AI governance is tailored to fit specific sectors by regulators such as the RBI¹¹, SEBI¹², and ICMR¹³.
• Intermediary Liabilities: In accordance with IT rules, intermediary platforms using AI technology need to identify misinformation and take down any unlawful deepfake videos generated using AI.

FUTURE LEGISLATION

• Digital India Act/Bill (DIA): This proposed legislation is intended to replace the IT Act 2000 and introduce a strict, comprehensive regulatory framework for emerging technologies. It aims to explicitly define boundaries and penalties for internet intermediaries and companies deploying consumer-facing AI systems.

AI IN INDIAN COURTROOMS: APPLICATIONS AND CHALLENGES.

Initiatives in India

To date, there is no indication from the Supreme Court of India that it is ready to embrace AI technology in the judicial process. Nevertheless, the Supreme Court took the initiative and created SUPACE (Supreme Court Portal for Assistance in Courts Efficiency), which was designed as a tool for helping judges to scan through the case filings and pull out pertinent facts from the filing. As Justice L Nageswara Rao stated, SUPACE aimed at designing a system wherein the machine would analyze the filing and give responses to factual queries that the judge might have regarding the case. Additionally, the Supreme Court has introduced another technology called SUVAS (Supreme Court Vidhik Anuvaad Software), which aids in translating case documents in English into Indian languages and vice versa. Nevertheless, only 31 cases were translated in the period from March 2020 to December 2021, according to the public records available. Translators should still verify the machine-generated translation and, as Justice Rao stated, it is quite difficult for the court to find such translators.

Since the extent of influence of SUPACE and SUVAS in dealing with the problem of pendency of cases is very little, it is appropriate on our part to focus on other ways, which may include the process of predictive analysis as one of the alternatives, through which the courts in India can successfully tackle this problem of pendency of cases. The graph below is a presentation of the problem of pendency of cases in India.

Predictive Analytics

AI tools may predict:

• case outcomes
• bail risks
• sentencing patterns

However, predictive models risk embedding systemic biases, especially against marginalized communities.

ADVANTAGES AND RISKS OF USING AI IN LEGAL RESEARCH

There are several artificial intelligence-based applications like ChatGPT, legal research websites, and proprietary judicial support software that lawyers and judges employ to conduct legal research more efficiently and derive meaningful insights from large data volumes within a brief timeframe. However, while these applications offer numerous benefits, there are many risks associated with their use as well.

Firstly, one such risk pertains to the problem of hallucination, i.e., the generation of fictitious cases, non-existent laws/statutes/citations, or even false information. For example, according to reports published in 2023, some lawyers were using fictitious cases generated by ChatGPT in US courts, e.g., in the lawsuit of Mata v. Avianca Inc.¹⁴ Secondly, an AI system may misinterpret legal rules/provisions and overlook important details related to the historical, constitutional, and socio-economic context in which particular laws emerged and operate. Therefore, since legal reasoning involves complex interpretations, the outputs produced by artificial intelligence should be viewed merely as research assistance.

Understanding these risks, courts of different countries, including India, repeatedly stressed that all AI-generated materials require verification on the part of professionals. In other words, while technology could help conduct research, it cannot replace the critical judgment, legal reasoning, and contextual analysis of lawyers and judges.

ETHICAL ISSUES OF USING ARTIFICIAL INTELLIGENCE IN COURTROOMS

Prejudices and Discriminations.

AI systems trained on historical data can inherit and replicate existing societal biases. In societies with long-standing inequalities, such systems risk reinforcing discrimination based on caste, gender, religion, race, or socio-economic status, thereby perpetuating unfair outcomes in judicial processes.

Opaqueness and “The Black Box.”

A lot of AI systems, and particularly proprietary algorithms created by private entities, are considered “black box.” The working principle is difficult to understand and explain. In the case of applying such algorithms in a courtroom, opaqueness could pose a problem, violating the principle of natural justice, as the party opposing the use of AI would have no opportunity to challenge and explain how the result has been derived.

The Mechanization of Justice.

Excessive use of AI may lead to the mechanisation of justice, where the decisions are made based on the algorithm’s calculations, not based on the discretion of the judges. As judgments involve moral consideration, human emotions, and understanding the situations, too much use of AI may dehumanize justice.

CONTEMPORARY ISSUES AND PRACTICAL IMPLICATIONS

Due Process and Right to a Fair Trial.

However, the major problem faced by the utilization of AI in court cases is the matter of a fair trial and due process. For instance, AI could be used by the judge in order to analyze large volumes of information, find a precedent, or predict the results of the trial, but still, there should be human beings who make judgments. The judgment of the judge will affect the lives of people, so it is vital.

One of the main concerns of using AI is the fact that AI makes judgments without providing any justification.

Data Protection and Surveillance Risks.

For AI to be used within court proceedings, there is a need for compliance with the requirements of the Digital Personal Data Protection Act, 2023 (DPDP Act). In relation to this, there is a need for courts to observe the requirement of limited data gathering as they process only that information which is required for particular purposes and ensure its safekeeping through encryption and other cybersecurity practices. It is imperative to observe such practices to protect against any unauthorized access to judicial data.

Digital Divide and Justice Accessibility.

Even though there is a likelihood that AI will make the processes in the justice sector more efficient, it might also cause an increase in disparities. Many people living in rural areas, some small courthouses have insufficient access to technologies and internet connectivity. Besides, the implementation of advanced AI solutions can turn out to be costly, which would make them unavailable for the majority of lawyers and legal practitioners.

Accountability and Liability.

Moreover, the incorporation of AI in the legal process gives rise to accountability issues as well. It is still unknown whether the faulty AI software developers are accountable or the court officials using the software are liable for making mistakes while deciding cases. There is no such liability framework in India yet, which makes the courts liable for mistakes made due to AI errors.

CONCLUSION

The use of Artificial Intelligence in judicial proceedings is one of the most significant innovations in 21st-century justice administration. AI has strong potential to improve judicial efficiency, streamline procedures, support legal research, and enhance access to justice, particularly in countries like India, where courts face heavy caseloads and limited resources.

Although the use of AI can help improve judicial efficiencies, its application in the courtrooms poses several issues regarding the legality, constitutionality, and ethics involved. Issues like algorithmic discrimination, invasion of privacy, and the absence of sufficient human intervention make it even more complicated to implement. Hence, it should be used merely as an assisting mechanism, guided by humans and conducted within a proper framework that protects judicial integrity and personal rights.

REFERENCES

• Digital Personal Data Protection Act, 2023
• Information Technology Act, 2000
• Supreme Court e Committee Reports (2023–2024)
• Subhash Desai v. Principal Secretary, Governor of Maharashtra
• Punjab & Haryana High Court Bail Order (2024)
• State v. Loomis, 881 N.W.2d 749 (Wisconsin, 2016)
• https://legalwiki.co/article/supreme-court-citing-fake-ai-generated-case-laws-amounts-to-judicial-misconduct/
• https://www.scconline.com/blog/post/2026/01/19/bom-hc-cost-unverified-ai-generated-submissions/
• https://www.pib.gov.in/PressReleasePage.aspx?PRID=2228315&reg=3&lang=2
• https://cms.law/en/int/expert-guides/ai-regulation-scanner/india
• Mata v. Avianca Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023).
• Supreme Court of India observations in Jaswinder Singh v. State of Punjab (2023), where ChatGPT was referred to only as a tool for broader understanding and not as a substitute for judicial determination.
• https://www.unesco.org/en/articles/recommendation-ethics-artificial-intelligence.
• https://www.niti.gov.in/sites/d efault/files/2024-06/document.pdf NITI Aayog, Responsible AI for All: Approach Document for India (2021).
• https://vidhilegalpolicy.in/blog/explained-the-digital-india-act-2023/

FOOTNOTES

1. Greenopolis Welfare Association (GWA) v. Narender Singh and Ors, CM
   (M) 1909/2025, CM APPL. 61372/2025. ↩︎
2. Article 14 of the Indian Constitution – Right to Equality. ↩︎
3. Article 19 of the Indian Constitution – Right to Freedom ↩︎
4. Article 21 – Right to Life. ↩︎
5. Gummadi Usha Rani & Anr. v. Sure Mallikarjuna Rao & Anr., 2026 SCC Online SC 341 / Special Leave to Appeal (C) No. 7575/2026 ↩︎
6. State v. Loomis 881 N.W.2d 749 (Wis. 2016) ↩︎
7. Deepak v. Heart & Soul Entertainment Ltd .,- 2026 SCC OnLine Bom 209, decided on 7-1-2026 ↩︎
8. Subhash Desai v. Principal Secretary, Governor of Maharashtra , (2023) 5 SCC 1. ↩︎
9. Mahesh Kumar Alias Mahesh Bansal vs State Of Haryana CRM-M-22107-2024. ↩︎
10. K.S. Puttaswamy v. Union of India (2017) 10 SCC 1 (also cited as AIR 2017 SC 4161. ↩︎
11. Reserve Bank of India. ↩︎
12. Securities and Exchange Board of India. ↩︎
13. Indian Council of Medical Research ↩︎
14. Mata v. Avianca Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023). ↩︎