TCA Anant, secretary, ministry of statistics and programme implementation (MOSPI), explains how data is produced and processed in India, and the role technology plays in it. Excerpts from an interview with Shivangi Narayan:

Data collection in India has mostly been a paper-pen process. How much technology is in use now?
In general we have been automating small and repetitive processes. Recently, the rural development ministry as well as housing and poverty alleviation ministry used tablets for collecting data for the socio-economic and caste census. The union government in partnership with states has been promoting various e-governance initiatives which lead to digitising the records of a number of government programmes. All these forms of technology adoption are at various stages of implementation and penetration.

(Also read: In hi-tech days, Indian data collection still in paper age)

Critics say government data is often not relevant. Why is it so?
The question of relevance does not apply to data collected through census or surveys. In each of these cases the data that is collected is based on carefully developed questionnaire and is very limited and focussed in its coverage. A lot of the loose statements about relevance are typically in connection with administrative data. For example, we have data about road coverage, but this data does not indicate the quality of these roads. Because of this, citizens often observe that data is not relevant. The issue of relevance is one that is dealt with by modernising and updating official record-keeping systems to include necessary parameters, which reflect the needs of the policy.

What is the future of big data analytics in India?
There are three kinds of big data analytics currently in progress in the world. One, analysing large data sets collected through censuses and survey. Second, analysing data collected from e-governance platforms. Both these kinds of analytics are currently in place in India. There is a third kind, analytics of data collected from the internet, which is what the West is talking about, and that is not happening much in India due to a variety of reasons including low internet penetration.

One example of such big data analysis is that at MOSPI we are analysing the data of the ministry of corporate affairs collected through its e-governance programme, MCA21. The adoption of XBRL, a business reporting language, in MCA21 implies that data can be easily analysed as it is in a workable digital format. 

Other examples of big data analytics is in the work of credit rating agencies. If you have been active in the banking and insurance sectors then your credit rating may be available online. The income-tax department analyses data about your expenditure. In other places data from voter lists and other public records is analysed to provide services.

Are the authorities sensitive to the ethics for data collection and storage?
We need to have ethics of data in place because we are moving towards a world where data will have a central position. Today we have digital tools that make it easy to analyse the kind of data for which earlier we needed an entire army of support staff. There are a number of organisations that have rules of safeguarding data in place. The Reserve Bank of India and the central boards of direct and indirect taxes have established rules for data safeguarding. However, in newer areas related to social programmes, we still need to evolve standards.

Why is there such a severe shortage of human resource at MOSPI?
There are many reasons. One of them is that the private sector market has high demands for people with the kind of skill-sets that we need. In addition, appointments in the government take time, and by the time the final candidates are appointed, they often find placement in the private sector. We experienced a shortage of people when the private sector market was booming. With the recent slowdown, we are now getting a healthy response.

It is said that India lags far behind China in data collection and processing. Why is it so?
I am not sure about the origin of your observation. Compared to most developing countries our data collection is very comprehensive. We, however, lag behind developed countries because we are yet to completely digitise our systems. In developed countries, citizens provide data electronically to their governments. While our government has to conduct data collection drive, they do not even need to conduct population census because they already have that data with them.

(The interview appeared in the August 1 to 15 issue of the magazine)