Scaling Big Data in Agriculture


Small and marginal farmers account for more than 86.2 % of all farmers in India. With fragmented land holdings, depleting natural resources and climatic variations, the marginal farmers have experienced prolonged low productivity and inefficient production. Actors including the state, civil society, nonprofits are working in their own capacity and in tandem to address the livelihood problems of small and marginal farmers. The common thread of solution that resonates across the ecosystem is the need for innovation. Big data could be the innovation that would trigger the paradigm change in achieving efficient agricultural production in the context of developing countries. There are existing platforms that have already leveraged agricultural production with the aid of big data around the world. Picture Based Insurance (PBI) by International Food and Policy Research Institute (IFPRI) is one such innovation which aims to deliver comprehensive crop insurance to the farmers with the use of their smartphones. Farmers capture pictures of their crop cycle from sowing to harvest. The extent of damages is then measured and the insurance payments are made directly to their bank accounts. This in turn creates a platform for leveraging the data from the farmers to develop sustainable cropping practices.

Big data are large sets of structured and unstructured data from various sources ranging from administrative data, survey data, satellite data and user data from various platforms which can be computed and made sense statistically to understand patterns. For example, patterns from large sets of weather data can be used to predict natural disasters. Through a combination of information analytics and affordable technology, insights from big data can enable farmers to reduce the information asymmetry on efficient agricultural practices. In the Indian context, the past decade has seen a spur in growth of AgriTech innovations. Agri tech solutions leverage data from a wide range of data sources and provide precise advisory to farmers in cropping pattern, soil usage, fertilizer applications and harvesting cycles. Big Data is one of the key factors to sustainable farming, for instance Geographical Information Systems (GIS) enabled technical support enables farmers to increase productivity of the land without expanding agricultural land. This article tries to focus on key challenges and approaches to address those challenges at the ecosystem level in order to enable scaling of Artificial Intelligence for the benefit of smallholder farmers.

While big data cannot be considered as a silver bullet to solve the agricultural crisis in India. Application of big data in tandem with the existing agricultural interventions might prove fruitful and ensure sustainability of the solution in the long run. One approach could be a model in which the AgriTech companies could work with the farmer cooperatives. Farmer cooperatives are one of the most successful interventions in the Indian agriculture space that harnessed the strength of the communities. Through collaboration with the cooperatives, AgriTech companies can achieve reach to gather quality farm level data from individual farmers. The cooperatives in turn can create a value chain where the farmer’s information and data are translated to income.

There are also considerable challenges in leveraging big data in the Indian agricultural context. The question of ownership, targeting beneficiaries for advisory and technological infrastructure in the rural areas seem to pose a big challenge in the context of India. Most of these problems can be addressed and the solutions can be further enabled through a systems approach. With contributions from different stakeholders in the ecosystem, the problem of collection of quality data, institutions for regulation on ownership of data and implementation of advisory solutions can be addressed in the long run.

Big data and AI technologies have sparsely made inroads in rural India. With an ecosystem approach big data and related technology can be adopted, enabled and scaled up to reach a large population of marginal farmers. Indian nonprofits have proved time and again on their capability to scale up innovative solutions to achieve impact. Scaling up big data and AI in the agricultural space will not only enable marginal farmers in providing them backward linkages through inputs and advisory but also enabling them in forward linkages in identifying consumption gaps and patterns that exist in the market to produce efficiently and increase their income levels. The impact of big data in other fields such as health, urban infrastructure, education have already created ripples across their sector. Big data is both an opportunity and challenge that can’t be looked away in the agricultural sector. Collaboration and partnerships are the key in leveraging big data and AI technology for the marginal farmers in the Indian agricultural space.

The author, Nelson Mathews is a Research Analyst with Catalyst Management Service


Inclusive Business Model



In India, around 70 percent to 90 percent of the rural households depend on agriculture and allied sector as their principal means of livelihood. Smallholder farmers constitute 80 percent of total farm households, forming the backbone of agricultural production. However, despite their contributions, they remain poor and vulnerable, plagued by a serious lack of resources ( land, water, energy, and credit), appropriate technologies, opportunities to develop skills, functional and fair markets for products and inputs, health care and sanitation, and education and social services. Declining productivity and incomes, and concerns about the overall viability of smallholder farming systems itself, pose a huge barrier to achieving poverty reduction, food security, and economic empowerment of this marginalised section. Notwithstanding years of investment by government, NGOs and the farmers themselves, and despite improved markets and the emerging new opportunities, the smallholder agricultural production system has continued to operate inefficiently, failing to yield scalable impact.

We believe that there is a need to refocus attention on the farming systems as an ‘enterprise’ and re-strategise services through a demand-led market-based mechanism. An integrated approach that leverages market opportunities, provides comprehensive end-to-end solution, and builds sustainable business-supply-chains to integrate farmers, can revitalise the smallholder farming systems. This requires a long-term engagement with farmers, and, importantly, looking at them as partners in the chain, rather than as recipients of benefits.

Our work with various market agents have proven that there are scalable and lasting solutions for improving productivity and reducing poverty for a farm. What is required is the transformation of smallholder farming systems from subsistence to profit-making. This requires an inclusive business model that goes beyond CSR to look at integrated inclusive business chains, invested in by companies/individuals who can sustain them through established channels and ensured markets.

It is important to note that for a model that creates social value, it is crucial to periodically monitor and evaluate impacts/social values, develop ‘performance metrics’ for the staff, channel partners and the farmers, extract learning from these, and use them in the action plans to implement effective interventions ensuring both business and social impacts.