Integrating IATI into HDX: bridging the gap between development and humanitarian data

We spoke to David Megginson, Standards Lead at UN OCHA’s Centre for Humanitarian Data about their pilot to include IATI data in the Humanitarian Data Exchange.

What is the Humanitarian Data Exchange?

The Humanitarian Data Exchange (HDX) is an open platform for sharing data across crises and organisations. Managed by UN OCHA’s Centre for Humanitarian Data, HDX makes humanitarian data easy to find and use for analysis. Users are part of a community that is connecting data from a range of organisations working in the humanitarian sector.

What data can be found on the Humanitarian Data Exchange?

HDX includes a wide range of humanitarian operational and reporting data, from geographical boundaries and needs assessments, to information on biomass and road networks. The site’s goal is to bring together different types of data and make it easy for members of the humanitarian community to access and use it when responding to crises. For example, a user can quickly access all data in HDX relating to a specific country on dedicated country pages, and for select countries, our new Data Grid feature shows at a glance what essential data is available, and what still needs to be shared.

To-date almost 250 organisations have shared information on their humanitarian work, contributing nearly 9000 datasets. Users can access data through interactive maps, download individual datasets in popular machine-readable formats and find useful links to gain further information.

Why did you decide to include IATI data in HDX?

we wanted to improve the visibility and discoverability of IATI data by allowing it to be accessed by HDX users. We know that currently there is a huge amount of aid data published to IATI that can be valuable for humanitarian responders

Firstly, we wanted to improve the visibility and discoverability of IATI data by allowing it to be accessed by HDX users. We know that currently there is a huge amount of aid data published to IATI that can be valuable for humanitarian responders. This is includes data on organisations who are delivering development projects in close proximity to an area affected by a humanitarian crisis. Much of this important development information may only exist within IATI, however, so most responders are not aware it exists, how to find or how to use it. Roderick Besseling from the UK Department for International Development originally approached the Centre with this idea, and has been supportive throughout the collaboration.

Secondly, the Centre for Humanitarian Data wanted to promote the IATI Standard as part of our core mission to support and promote aid data standards. We have always recognised the importance of IATI for transparency and coordination in the broader aid sector, so we decided to pilot the use of IATI data with the Humanitarian Exchange Language (HXL). This simpler data standard for humanitarian data is currently used on the HDX platform and other organisations to improve the speed and interoperability of general humanitarian data. We’ve been happy with the results so far and look forward to exploring deeper collaborations between the two standards.

Who do you want to use the data?

We want those working in humanitarian crises to use IATI data to become more aware of the development projects happening in the countries where they are working, sometimes only minutes away. For example, in the outbreak of a virus, responders can understand what health professionals and supplies are already in place as part of development-aid projects being delivered in the affected countries and use this information to better coordinate their responses with others already on the ground.

We want those working in humanitarian crises to use IATI data to become more aware of the development projects happening in the countries where they are working, sometimes only minutes away

Multilaterals and NGOs working within the humanitarian sector already coordinate their work through “cluster” systems that focus on specific sectors such as education or health. Unfortunately, this coordination rarely extends to those outside of the humanitarian sector. It is possible that the closest thing the development-aid sector has to coordination is sharing and using data through IATI.

Though the line between where humanitarian aid ends and development aid starts can be blurry, the communication and coordination division is very real. Putting IATI data in front of humanitarians in a format they’re used to (spreadsheets), is a first, small step to bridging that gap.

Specifically how did you make IATI data compatible to use in HDX?

We put a simplified version of IATI data into HDX, side-by-side with other aid data for the same country (baseline figures, needs assessments, activity reports, facilities lists, food prices, etc). To connect IATI data with the existing data on the platform, we added interoperable hashtags from HXL for basic information categories (such as dates, sectors, locations and organisations). To allow users to find the full datasets, we’ve provided links to IATI’s online search platform, d-portal.

We initially piloted with IATI data relating to Nepal and the Democratic Republic of Congo. After demonstrating success, we expanded to 244 countries and country-like entities.

We initially piloted with IATI data relating to Nepal and the Democratic Republic of Congo. After demonstrating success, we expanded to 244 countries and country-like entities

How can users access the data?

Users can view a summary of the data through Quick Charts, which are small preview visualisations at the top of each dataset page.

Many humanitarian responders, however, will want to download the datasets so that they can explore them in a spreadsheet program, or combine them with other humanitarian data. Because spreadsheet formats don’t have the rich expression power of IATI’s original XML markup, we provide two different kinds of snapshots:

• a CSV file with one row for each activity, e.g. IATI activities in Nepal (no location information)
• a CSV file with one row for each activity/location combination (so the same activity may be repeated in more than one row). For example see IATI activity locations in Nepal

Both options use live data from the IATI search platform, d-portal, so they are always up to date.

To ensure the data is useful, we also add tags to the datasets that are familiar to humanitarians (not to be confused with the HXL hashtags inside the spreadsheets). For example, the humanitarian community refers to a list of aid activities as a “3W” (who-what-where) or “4W” (who-what-where-when), so we’ve tagged all of the IATI datasets with these terms to make them show up in searches.

Do you have future plans on working with IATI data?

we want to find more ways to bridge the humanitarian-development chasm so that people in need will benefit from a more-coordinated and less-wasteful international aid effort

We plan to update Quick Charts on HDX to show what proportion of the activities are humanitarian versus development and will include that data in the simplified spreadsheets. This is now possible as d-portal displays where publishers have used the ‘humanitarian flag’ on their activities. We are also looking forward to the new IATI Datastore being released and intend to be one of its first users.

More broadly, we want to find more ways to bridge the humanitarian-development chasm so that people in need will benefit from a more-coordinated and less-wasteful international aid effort. We look forward to a time when, using IATI identifiers, we can associate HXL-hashtagged operational field datasets with the activities that funded them, providing much-improved transparency for everyone involved.

While it does not directly involve HDX, the Centre for Humanitarian Data is working on a pilot to directly ingest IATI data into OCHA’s Financial Tracking Service (FTS). FTS is responsible for tracking progress in funding requirements for humanitarian responses. Working directly from IATI data will allow FTS to include funding data from more organisations in its tracking, while reducing the burden of humanitarian organisations reporting their data to an additional channel.