Future of Digital Welfare in India: Critical Reflections on Agentic AI and Welfare Delivery Governance

Suruchi Kumari & Osama Manzar

Abstract

The growing integration of agentic AI into Digital Public Infrastructure (DPI) is likely to reshape welfare governance in India, raising important questions about citizens' rights, accountability, transparency, and inclusion. This essay examines the implications of integrating agentic AI into welfare governance through a Just AI perspective, arguing that public service delivery should be designed to include rather than exclude people, and to centre the human in the loop. It conceptualises AI systems not merely as technical infrastructures but as socio-political systems whose design choices shape access to welfare, participation, and citizenship, and considers how data justice, accountability, and participation can inform future AI policy in India.

—

In India, a recent digital audit in Maharashtra of the Ladki Bahin Yojana removed 90 lakh beneficiaries, reducing the applicant base from 2.6 crore to 1.53 crore. The authorities claimed that the exercise was undertaken to eliminate “duplications and ineligibility” among beneficiaries. The application process itself involves e-KYC, which raises an important question about the digitalisation of welfare: verification is conducted digitally, and rejection is also done digitally, with little or no scope for alternative forms of verification or human intervention.

In another study, Hera Rizwan (2026) found that nearly half of the eligible pregnant women and nursing mothers among beneficiaries in India have been unable to access their rations because their identities are not authenticated through facial scans in the Indian government’s POSHAN tracker app. The POSHAN tracker uses Google’s ML Kit, a tool designed for face recognition rather than long-term identification. 

The two examples above illustrate how the integration of digital AI technologies within Digital Public Infrastructure (DPI) can affect people's rights. In India, DPI and the increasing use of AI for tracking, identifying, and verifying beneficiaries of welfare schemes are largely opaque, and citizens have repeatedly contested official claims of efficiency and accuracy. These examples raise important questions about the use of AI and DPI for welfare delivery in India, particularly in relation to accountability, citizens’ rights, and the future of welfare governance.

In this essay, we advance a Just AI perspective that centres justice in both the design and application of AI systems. Here, justice is realised through accountability, participation, privacy protection, and the safeguarding of citizens’ rights. As DPI and agentic AI become increasingly integrated, understanding the relationship between them becomes crucial. Agentic AI systems and DPI are likely to rely heavily on each other through the exchange of digital IDs and beneficiary databases to identify, verify, and determine eligibility for welfare provisions. This essay therefore argues that the design of the DPI will have consequences on the functioning of the agentic AI systems.

Importantly, marginalised communities experience exclusion differently, facing a range of barriers, including unstable or absent connectivity, limited digital literacy, inadequate infrastructure, lack of meaningful access, shifting identities, and barriers due to discrepancies and gaps in documentation. When data infrastructures do not account for these lived realities, the voices of those most affected by AI-driven systems remain invisible, with no representation in their design, development, or auditing processes.

Now imagine a near future in which welfare functions such as beneficiary identification, tracking, verification, and service delivery are undertaken by agentic AI systems. For example, an AI agent tasked with processing supplementary nutrition benefits for pregnant women under the POSHAN Abhiyaan may autonomously follow an “Observe–Orient–Decide–Act” framework. It could cross-reference a beneficiary’s pregnancy status, medical records, income category, and other data points to assess eligibility. Here, the agentic AI will do the task of “observing” the state of the application, verify eligibility against the database at Anganwadi centres or hospitals (for which no permission will be taken) and then “decide” on the verification. In the event of data mismatches – arising from surname change after marriage, unclear signatures, or mismatches between photos and signatures – the AI agent rejects or “acts” on the application. 

Policy discussions around AI are therefore increasingly focusing on AI agents that are not simple bots responding to prompts but more advanced versions that act as goal-driven entities capable of planning, reasoning, and explaining. Chowdhury (2024), studying automated decision-making (ADM) in Australian welfare delivery, shows that the Robodebt programme failed to meet principles of accountability and transparency and was found to be inconsistent with human rights standards. Similarly, the Dutch Systeem Risico Indicatie (SyRI), the first automated welfare fraud detection system, was criticised by the UN Special Rapporteur on extreme poverty and human rights in 2018 as an example of the risks posed by automated welfare governance. The Rapporteur warned that such systems were “zombie-like” and could create a “digital welfare dystopia.”

Agentic AI systems operate through multi-step agentic actions; therefore, there is no cross-checking of tasks from the back as the data pipeline and tasks flow downstream. A recent working paper by Amlan Mohanty (2026), Governing AI Agents: Cascading Risks, Coordinated Action, published by the Centre for Responsible AI (CeRAI), IIT-Madras, outlines the unique features of agentic AI systems, which raise important governance issues. The convergence of five unique features of agentic systems – autonomy, tool use, multi-step action, influence, and adaptability – poses governance challenges that current policy frameworks in India will need to address. AI agents don’t simply generate content; they act, adapt, and influence digital and physical environments across extended time horizons, with compounding and potentially irreversible consequences. This working paper predicts that, from 2027, agentic systems will be deeply integrated into the digital economy, performing a vast array of specialised tasks across sectors, interacting with one another in unpredictable ways, generating synthetic content at a massive scale, and potentially being weaponised for cyber-offensive operations. The pace of deployment is accelerating faster than existing governance frameworks can keep up with, exacerbating the barriers in digitalised systems. In many cases, existing regulations will address emerging risks, while in other cases, more concerted and coordinated action will be necessary. Donta et al. (2025) similarly characterise AI systems as socio-technical systems capable of autonomous perception, reasoning, and action, enabling operation with limited or no direct human intervention. Such autonomy represents a substantive shift from conventional automation and decision-support systems and has direct implications for how these technologies interact with human values, institutions, and social structures. The capacity for independent planning introduces moral dilemmas; large-scale autonomous action amplifies the risks of bias and danger; and integration into human environments raises adoption and design challenges.

This essay raises important questions for the future of AI policy in India, where regimes of citizenship will be reconfigured and debates are likely to centre on safeguarding citizens’ agency by regulating AI agents to reduce risks across various socio-political contexts. Rather than treating interaction with citizens and techno-political implications as a static policy outcome (e.g., users’ experience of exclusion), we conceptualise it as a dynamic process shaped by the design and deployment of agentic AI systems. This requires attention to the upstream dimensions of AI governance, including questions such as: Who develops the AI agents? What kind of data is used to feed the AI data? Who decides the selection of the AI agents and the purpose of the AI agents? Who regulates them? 

However, the design of AI agents also evolves through everyday use in downstream activities. This can be better understood by asking questions such as: How are the everyday lives of citizens impacted by AI-based governance? Are some groups more vulnerable to algorithmic biases and exclusion, such as nomadic and other migrant populations whose residential addresses vary across databases? An upstream-downstream analysis of data pipelines in AI agentic systems should be guided by the principle of data and information justice, in which representative data from all communities is collected. Another critical dimension concerns the distribution of accountability across institutions, including governments, private companies, and public institutions. A data-justice approach to the upstream and downstream analyses of AI agents’ design and implementation will therefore help determine which AI policies are required to regulate these systems in the near future.

Current Approaches and Gaps 

Currently, India’s approach to digital governance is gradually shifting from e-governance to AI-enabled governance, posing critical challenges at the levels of principles, design, and implementation. The National Strategy for Artificial Intelligence, launched by NITI Aayog in June 2018, identifies AI as a transformative tool to address India’s development challenges by improving the access, affordability, and quality of essential services. It prioritises inclusive and socially oriented growth, particularly in underserved sectors and regions, with rural India recognised as a key focus area due to persistent service and infrastructure gaps. In sectors such as agriculture, healthcare, and education, AI-enabled decision-support systems and data-driven platforms are envisaged as mechanisms to strengthen frontline workers and local institutions, extending services to remote populations without extensive physical infrastructure expansion.

The strategy emphasises augmentation rather than displacement of human labour, positioning AI as a support system for farmers, health workers, teachers, and administrators. It also highlights AI’s role in promoting inclusive economic participation through decentralised skilling, digital work opportunities, and technology-aligned training. Under the AI for All framework, AI is framed as a catalyst for inclusive rural growth, strengthened governance, and enhanced human capacity. 

However, existing approaches often lack a participatory design framework and do not adequately engage with lived, contextual realities, thus failing to capture local knowledge systems and the crucial role of community-centred local institutions. The limited participation of community and local governance bodies, such as Panchayati Raj Institutions, also constrains opportunities for local cross-checking and verification of whether data is representative and valid. 

The next phase envisions digital infrastructure that not only facilitates transactions but also learns, predicts, and adapts. This AI-DPI convergence atop DPI rails unlocks three capabilities and strengthens AI transformation in India through the following areas:

• Predictive governance: Predictive governance uses AI to identify risks early, such as forecasting crop losses for timely insurance payouts, anticipating regional power demand, and flagging MSMEs at risk of closure for early support. This shifts governance from reactive to anticipatory and enhances the implementation of AI within public systems.
• Smarter regulation and service delivery: Fraud detection in Direct Benefit Transfer (DBT) schemes, high-risk inspections in food and labour departments, and automated grievance routing are early examples of how AI-driven inspections, automated approvals, and algorithmic risk scoring can improve efficiency and compliance.
• Localised and federated AI: Given India’s linguistic and cultural diversity, AI models must adapt to local contexts, and federated architectures allow states to train models without centralising sensitive data.

The current national strategy has critical gaps relating to the principles of data justice, explainability, and transparency. Its approach remains technical, with limited focus on societal, environmental, and political dimensions of AI governance.

A community-centred approach to data governance, on the other hand, requires recognising data rights and treating data as a collective resource that communities can co-create, access, verify, trust, and validate. The Just AI approach calls for a shift from viewing individuals as “beneficiaries” or “passive subjects” to recognising them as rightful stakeholders who understand the “who”, “how”, and “why” of data collection and use.

Current policy frameworks do not adequately incorporate data and information justice into the design of AI-enabled welfare systems. As the examples discussed above illustrate, the data and information available to AI and agentic AI systems is often complex and susceptible to misinterpretation. In many Global South contexts, vulnerable communities often have limited trust in governments and other institutions that collect and process their data. As a result, digital data systems may be perceived as increasing vulnerability, particularly where personal information is viewed as being open to misuse, exploitation, or forms of surveillance that threaten livelihoods and survival. For example, if policymakers want to map families in slum areas using agentic AI and subsequently decide to proceed with slum demolition, the technology could directly contribute to the displacement of vulnerable residents. Signal Foundation President Meredith Whittaker argues that agentic AI systems will gain “root access” to all of our digital data, giving AI pervasive access to data and information without obtaining the permissions/consents of humans/citizens. In computing, this is referred to as “root permission” or complete system control – akin to having a master key to every door. Agentic AI systems often receive far more access to data than any single human would need.

The challenges posed by agentic AI for welfare governance arise from the core features of these systems – autonomy, tool use, influence, and adaptability. These characteristics have serious implications for security, privacy, and individuals’ control over data. In India, concerns extend beyond data injustice and privacy violations to include design-level issues such as mandatory facial recognition, the centralisation of data, and the absence of explainability, all of which can render the system opaque to citizens. The inability to explain why an application is rejected or why certain communities are excluded from the design forms a part of the “black box” problem, in which decision-making processes of AI systems remain difficult to understand. 

As AI systems gain greater autonomy in decision-making, their impact extends beyond technical performance to encompass ethical norms, governance, and public trust. This has prompted the development of new frameworks that seek to capture the intertwined relationship between technological autonomy and its societal consequences. Donta et al. (2025) propose the MAD–BAD–SAD framework, which organises the socio-technical dimensions of agentic AI into three categories. MAD (Motivations, Applications, Dilemmas) explores the drivers, uses, and moral tensions of agentic AI; BAD (Biases, Accountability, Dangers) addresses issues of fairness, responsibility, and safety; and SAD (Societal, Adoption, Design) analyses social acceptance, integration, and human-centered design.

The Just AI approach treats agentic AI not merely as a technical question but as a socio-political and ethical one, placing communities at the heart of digital design and advocating for a participatory design framework, in which communities act as co-creators and co-governers of the algorithmic data and digital systems. In doing so, it prioritises social and economic inclusion over purely commercial objectives.

Recommendations

Scoping the Necessity of Agentic AI at the National Policy Level 

Before strategically adopting agentic AI for public service delivery, policymakers should assess where such systems are genuinely needed within the current socio-economic situation. This assessment should consider the implications of deployment for citizens’ rights and potential exclusion, as well as the accountability mechanisms required to address them.

Explainability of the AI Agents’ Work 

The opacity of agentic AI decisions needs to be addressed through simple and accessible explanations of data pipelines and decision-making. Such information should be made publicly available in plain language both digitally and in print, on the various departments’ websites and offices. Data pipelines should also specify which authority collects the data and for what purposes the data will be used.

Ensuring Data Justice

Data justice must be upheld across representation, privacy, and security at the input level, and the right to information and redressal of grievances at the output level. Governments should support the building of community internet infrastructures at the local level. Unlike traditional data infrastructure, typically built and owned by governments and corporations, community-led data infrastructure is designed, owned, and operated by the community it serves. The Association for Progressive Communication defines community internet infrastructure as “the combination of agreements and relationships that, together with hardware, software and applications, are used in order to provide one or more internet-based services, and that are designed, owned/co-owned, managed and operated by an organisation or community. The resulting services are conceived and operated with a people-centred approach, taking into account equitable and rights-respecting guidelines”.

Awareness and AI Literacy Training

Sustained AI literacy training modules should be implemented at the national level to educate citizens on how to safeguard their data and understand AI data pipelines in their everyday lives. 

AI Agents’ Audit Boards 

Independent audit boards should be established to assess the principles and performance of AI agents in relation to risk assessments, technical standards, social inclusion, users’ rights, and legal frameworks.

Regulations and legal frameworks 

Robust regulatory and legal frameworks should be developed before the large-scale deployment of agentic AI in public services. Such frameworks should prioritise the protection of citizens’ rights and establish clear standards for accountability, transparency, and oversight throughout the implementation process.