Making public data public: Sri Lanka misses its own targets as India rushes – Editorial

In Sri Lanka, the Department of Census and Statistics (DCS) is the main government agency responsible for collecting and accessing data that can be used for statistical analysis. The stated vision of DCS in Sri Lanka is to be the “leader in the region” in producing timely statistical information to achieve the country’s development goals.

This overview finds that DCS in Sri Lanka falls far short of its own stated commitments to achieve this vision. The analysis also reveals that India, despite a much later start and a more complex data collection environment, has rushed to become the regional leader that Sri Lanka aspires to be, providing access to public data. .

The comparative analysis with India covers three dimensions: (i) ease of access, (ii) speed of access and (iii) cost of access.

Importance of up-to-date and widely accessible data
Data is one of a country’s most important public goods. High accessibility of data leads to multiple positive outcomes: it enables academics and professionals to generate better analysis, helps government formulate better policies, facilitates better economic and investment decisions and can also provide better visibility to the company on actual results and government policy progress.

The term “data” is often used to refer to summarized aggregate statistics. We use the term to refer to the actual data used to generate these statistics.

The availability of data, and not just summary statistics, is essential for decisions about a country’s development goals. Take, for example, the unemployment rate reported in the Annual Labor Force Survey. This estimate is obtained by taking the average number of unemployed in the total sample; it only provides an overall macro picture of unemployment in Sri Lanka. But these summary statistics do not reveal important details such as the geographic and age dispersion of unemployment and the extent of unemployment disparities by other demographic factors such as ethnicity, language or gender.

Data sets with observation units at the individual, household or firm level can provide information on aspects such as gender, age, education level, income level and the district of residence, etc. while being appropriately anonymized to ensure the confidentiality of the reporting unit. These data pave the way for the formulation of better analyzed policies, more likely to be effective and sustainable. Therefore, in order to understand the details and make the right decisions “to achieve the country’s development goals” (as stated in the DCS policy), there must first be open access to real data.

Three dimensions of data accessibility: SL misses and lags behind India
For data to be an effective decision-making tool, both for policy makers and the public, it is important to assess three dimensions of data accessibility: (1) ease, (2) speed and ( 3) the cost. The analysis of this Insight shows that Sri Lanka is missing its own targets and falling behind India in these three important dimensions.

1. Ease of access: Sri Lanka does not make any datasets available online
In October 2014, the DCS implemented a data dissemination policy. This policy responds to the need for the public to access data and provides guidelines on how to access it. He clarified that “public” datasets are available on their National Data Archive (NADA) for public use and can be downloaded from the DCS website.

However, more than six and a half years later, DCS has failed to deliver on this key commitment as part of its own data release policy. Even as of June 2021, no dataset is available online on NADA. The research team made several attempts to obtain these “public” datasets and found that they were not available for download as stated in the policy. DCS officials also confirmed this status quo.

India is ahead: In India, the function of data collection and statistics at the national level is divided between the Ministry of Statistics and Program Implementation (MoSPI) and the Ministry of Home Affairs (MoHA) . MoSPI conducts periodic national economic censuses and large-scale national sample surveys, while MoHA conducts the decennial census.

India’s current data release policy, which also spells out its commitment to making data available online, was not adopted until April 2019 (five years after Sri Lanka). However, unlike Sri Lanka, India has fully operationalized its data dissemination policy. Users, public or private, inside and outside India, can access the public datasets through the national repository hosted on the MoSPI website. The website has a step-by-step guide on how to access and download these datasets. The data can be downloaded and viewed on Nesstar, a free software which can be used for data analysis or imported to other software such as STATA and can also be converted to Excel format.

2. Speed ​​of access: Sri Lanka has a protracted process
In addition to not making public datasets available online, DCS also failed to meet announced deadlines for providing offline datasets. According to DCS policy, once a request for access to certain datasets is made, the request must be assessed within two weeks and if approved, the datasets must be provided, upon receipt of payment. stipulated.

Two case studies by Verité Research were integral to the speed assessment. The case studies indicated that DCS is not meeting the timelines set out in its policy for providing access to datasets. The first case study involves a request to obtain a 5% sample from the 2011 Census of Population and Housing. It took several follow-ups by the research team for about a year and a half before the data became available. are received. The initial request was made on December 12, 2017; the final dataset was not received until June 1, 2019.

In the second case study, a similar request to obtain the Labor Force Survey and the Household Income and Expenditure Surveys of 2012 and 2016 also required several follow-ups and almost two months before the data was received ( see exhibit 1).

India is ahead: As India allows online accessibility and downloading of data, the time required to access data can be as short as a few hours rather than months (and sometimes over a year) as is the case in Sri Lanka.
3. Access cost: Sri Lanka charges a fee for most datasets; India provides them for free
In addition to the lack of ease and speed, data access in Sri Lanka faces the additional hurdle of costs. With the exception of Sri Lankan government institutions and students engaged in higher education, all users have to pay a fee to access the data. The standard cost for a local user to access 50KB of data is Rs.100.
Since these datasets are quite large, the total cost can be quite large. For example, in the first case study in 2019, it cost 427,406 rupees to access the 5% sample of the 2012 Population and Housing Census (see Exhibit 2).

The stipulated fees, again, are not in accordance with the stated policy. Data is collected using public funds. DCS policy states that pricing is designed only to “cover the cost of providing microdata and is not intended to cover all of the costs at all. [sic] activities, including the cost of data collection ”. Thus, apart from the fixed transaction costs associated with delivering the data to the requestor, the marginal cost of an additional quantum of data is close to zero. The linear tariff structure of Rs 100 for each 50KB unit, rather than a fixed transaction / delivery cost, is therefore not in line with the objective and principle of pricing set out in the stated policy.

The data release policy also states that it is designed “to encourage wider use of its products by making them affordable for users.” Exhibit 2 shows the actual cost incurred by Verité Research in obtaining selected datasets from DCS, ranging from Rs 30,000 to Rs 450,000. These prices, exceeding half of Sri Lanka’s annual per capita income for just 5 percent of the largest data sets, are more likely to discourage than encourage most users to use them by making them unaffordable rather than affordable. affordable.
The price is therefore twice contrary to the stated policy. It is not limited to the “supply cost” of the transaction and it is dauntingly expensive (rather than encouragingly affordable).

India is ahead: in the case of India, the datasets are available for free. In April 2019, the Indian government recognized official statistics as a public good and issued an official memorandum specifying that access to all datasets should be provided free of charge with one-stop online access to conduct research and research. both public and private purposes.

On data, India has progressed to become a model for Sri Lanka
By providing access to the data, India has taken more progressive positions and moved much faster to implement them. Sri Lanka, on the other hand, has adopted less progressive policies and has fallen behind in their implementation.

To quote the position of the Indian bureaucracy (as stated in the office memorandum issued by the Ministry of Statistics and Program Implementation, Government of India, on providing free online access to data) : “Official statistics are key elements for decision-making and political intervention and become public goods for conducting research in the public and private spheres. Recognizing the potential of the data, the Ministry of Statistics and Program Implementation of the Government of India has decided to provide free, one-stop access and support to micro-data from censuses and surveys conducted by the Ministry… ”

This overview assesses Sri Lanka’s performance in terms of its own stated policies as well as its aspirations to be a regional leader providing access to timely statistical information. The assessment against stated DCS policies suggests that Sri Lanka is currently failing to meet the targets that have been defined in its own policies. Comparison with India reveals that, despite recognition in both countries of the importance of data to public users, India’s recognition has resulted in policies that are more progressive than Sri Lanka’s and better implemented. implemented in terms of improving the ease, speed and cost of accessing data.

(Verité Research is an independent Colombo-based think tank that provides strategic analysis to high-level decision-makers in economics, law, politics and media. Comments are welcome. Send email to [email protected])