Virtual companions, real consequences

It is imperative to set up legal frameworks and ethical guidelines that safeguard users, especially those most vulnerable, from unintended consequences of AI

- ROHIT KUMAR SINGH

However, existing laws such as the Information Technology Act (IT Act), 2000, and the Consumer Protection Act could potentially be invoked in cases where harm is caused by AI-based applications.

Under the IT Act, intermediaries (which could include platforms hosting AI chatbots) are granted certain protections from liability as long as they do not knowingly allow harmful content or interactions. However, this protection becomes murky when dealing with AI systems that can engage in personalized and emotionally charged conversations. If an AI chatbot were found to have contributed to a user's mental distress or suicide, could the platform be held liable under Indian law? The answer is far from clear.

Negligence and duty of care: Can developers be held accountable?

In India, negligence is typically defined as a breach of duty that results in harm to another person. To establish negligence, it must be proven that the defendant owed a duty of care to the plaintiff; the defendant breached that duty; the breach caused harm or injury.