Improving the quality of COVID-19 data published to IATI: Initial findings and next steps for publishers

As part of ongoing efforts to address data quality issues and to support publishers in following the COVID-19 publishing guidance, the IATI Secretariat analysed the COVID-19 data published using the IATI Standard (read the full analysis in COVID-19 data published to IATI: what do we know and what’s next). The Secretariat then used this analysis to conduct outreach to publishers (specifically IATI members who are publishers) to share identified data quality issues and to collect information on their approach to publishing COVID-19 data, including challenges encountered, and also to reach out to IATI members who have not yet published COVID-19 data to offer support and learn if/when they plan to do so.

By following the guidance set out in this note, publishers can make their COVID-19 data more useful for partner countries and other development partners, allowing for better coordination and planning around COVID-19-related activities.

The purpose of this post is to share what has been learned from this exercise and to review tangible steps publishers can take to improve their IATI COVID-19 data. By following the guidance set out in this note, publishers can make their COVID-19 data more useful for partner countries and other development partners, allowing for better coordination and planning around COVID-19-related activities.

How are publishers identifying COVID-19-related activities within their IATI data?

As detailed in the COVID-19 publishing guidance, organisations can include COVID-19-specific values in multiple different data fields. If organisations include ‘COVID-19’ in the Activity Title, Activity Description, Tag, or Transaction Description (in one or more transactions), or codes specific to COVID-19 in the Humanitarian Scope, this enables the activity to be clearly identified as COVID-19-related. And if organizations include ‘COVID-19” in the Transaction Description, this enables the identification of specific transactions as COVID-19-related.

Overall, 53 publishers have published COVID-19-specific values in 1376 activities. The majority of publishers are identifying their activities as COVID-19-related by including ‘COVID-19’ in the Activity Title (34) or the Activity Description (42). Feedback from publishers suggests that this is the most commonly used option as it is the easiest to implement and allows them to quickly identify activities as COVID-19-related.

Overall, 53 publishers have published COVID-19-specific values in 1376 activities.

Currently, nine publishers are including ‘COVID-19’ in the Transaction Description of 1768 transactions. Feedback from publishers indicates that there is interest in being able to precisely indicate the amount of funds allocated to COVID-19 within an activity, something that is possible when transactions are identified as COVID-19-related. However, some publishers indicated that this is difficult to implement as it would require time-intensive changes to their internal reporting systems.

Amongst publishers that are not yet including COVID-19-specific values (of those who responded to the Secretariat’s query), most indicated that they are planning to publish in the near future, but haven’t yet done so as they are internally deciding how to categorize these activities or because they were yet to publish according to their regular publishing timelines.

Table 1: Number of publishers including COVID-19-specific values in activities and transactions *

*The data was retrieved from the IATI Registry on 17 June 2020.

What kind of data quality issues have been identified?

During the analysis of COVID-19-related activities and subsequent communication with publishers, a number of data quality issues were identified, that if addressed, would improve the quality and availability of COVID-19 data.

Most publishers including COVID-19-specific values according to the COVID-19 publishing guidance are doing so relatively consistently, but there are still some errors and instances of the guidance not being consistently applied across activities. Identifying these issues and working with publishers to address them was the focus of this initial round of engagement on data quality. Some of the issues identified include:

• Inconsistent use of ‘COVID-19’ in activity titles and descriptions: instead of including ‘COVID-19’, some activities included ‘COVID 19’, ‘COVID’, ‘Coronavirus’, etc.
• Errors in the use of Humanitarian Scope codes such as misspelling, omitting a number or inclusion of ‘COVID-19’ instead of actual codes.

As a result of sharing these issues, some publishers have already taken steps to address them, but publishers are asked to regularly quality check their data to ensure that COVID-19-specific values are accurately and consistently included across all COVID-19-related activities and/or transactions. This ensures that data users can identify and find all COVID-19-related activities.

In addition to these COVID-19 specific data quality issues, the following issues are also important for improving the quality of data published to IATI overall, including data on COVID-19:

Lack of reporting of recipient countries both at the activity and transaction levels. Many COVID-19-related activities do not specify the country where the activity is taking place, which makes it particularly difficult for data users in these countries to be able to use IATI data to determine the COVID-19-related activities that organisations are funding and implementing in their countries, and for data users more broadly to understand the geographical distribution of these activities.

Not identifying all participating organisations taking part in the COVID-19-related activity. Some funding organisations are not including the implementing organisation name in their data. Likewise, some publishers are not specifying the IATI Organisation Identifier for participating organizations. This makes it difficult to identify the flow of funds through the delivery chain.

Limited inclusion of COVID-19-specific values at transaction level, together with lack of use of provider and receiver organisations at transaction level. As one cannot assume that all transactions within an activity are COVID-19-related, not including a COVID-19-specific value at the transaction level makes it difficult to assess the resources allocated to COVID-19 within an activity. And not including the provider and receiver organizations means that it is not possible to identify the source and recipient of these resources.

While a few organisations are updating their data daily or weekly, the majority of organisations publishing data related to COVID-19 are only updating their data quarterly or monthly. The lack of real-time reporting on resources committed to and spent on the response to the pandemic makes it difficult for government officials to use this data to coordinate and make decisions on the allocation of national resources.

How to make your COVID-19 data more useful to data users

If activities do not include COVID-19-specific values according to the COVID-19 publishing guidance, these activities cannot be identified as COVID-19-related activities and found by data users. As such, all organisations who have published, or plan to publish, COVID-19-related data should follow the guidance correctly and consistently. The data quality issues raised in the section above can be addressed in the following ways:

• Consistently referring to ‘COVID-19’ in activity titles and descriptions and/or ensuring that the COVID-19-specific codes are accurate per the COVID-19 publishing guidance. This will enable data users to be able to identify these activities as COVID-19-related and for users to easily find these activities when searching IATI data in d-portal or other tools.
• Including recipient countries and participating organisation(s) along with their role(s), name(s), and IATI organisation identifier(s). This will enable data users to know where the activities are taking place and to trace the flows of funds.
• Publishing timely updates to your data, weekly if possible. This will better enable data users to use IATI COVID-19 data for real time coordination and planning.
• Identifying COVID-19-related transactions by including ‘COVID-19’ in the transaction description. This will enable data users to precisely identify the amount of resources that have been committed to or spent on the COVID-19 response. While this is the option suggested in the current COVID-19 publishing guidance, as feedback from publishers suggests, that this is not an option for all organisations. As such, publishers may wish to consider what options are available for identifying COVID-19-related transactions in their IATI data. Some alternative options have been proposed and are being explored (see discussion, which is still open for comments).

What’s next?

While this effort was an initial analysis of the COVID-19 data (as of 17 June), the Secretariat will continue to monitor progress and assess data quality of IATI COVID-19 data, and will share the findings through regular posts. The Secretariat will also continue to proactively engage with publishers to further improve data quality. If you require support, please do not hesitate to get in touch at [email protected].

The Secretariat is also actively engaging with the IATI community to assess whether improvements are needed to the COVID-19 publishing guidance so do get involved on IATI Discuss or directly at [email protected].

And the Secretariat wants to hear your stories - whether you are a publisher and you want to share your journey to prepare or improve your published data, or if you are a data user and you would like to share any findings or lessons learned in analysing the COVID-19 data. Please email us at [email protected] and share your story on Twitter or LinkedIn.