Transitioning from iati.cloud API to IATI Datastore API version 3

IATI’s Datastore version 3 uses the same backend technology as the iati.cloud SOLR API. However, various optimisations have been made allowing all available valid IATI data to be returned and ensuring that the outputs are aligned with the IATI Schema. A comparison between the two APIs can be found on this page.

Please note, iati.cloud is no longer managed by IATI and may not have the same services as previously listed.

Step-by-step transition

To replicate a query on iati.cloud, without adding any advanced features, the following guidance should be followed:

Create an API Gateway login

All users of the IATI Datastore API v3 are required to register on IATI’s API Gateway and gain a free API key. This step allows the IATI Secretariat to better track how IATI APIs are being used, and ensure a secure and robust service for users. Once you are logged into the API Gateway you can choose between two subscriptions:

Name	Description
Exploratory	Subscribers will be able to run 5 calls/minute up to a maximum of 100 calls/week. No approval is required
Full Access	Subscribers have unlimited access to Public

Once setup you will be able to use the API Keys provided either from within the API Gateway webpage or your local development tools. Further guidance is provided in API Gateway Instructions.

Change the base URL

iati.cloud	IATI Datastore version 3
https://datastore.iati.cloud/api/v2/activity?	https://api.iatistandard.org/datastore/activity/select?
https://datastore.iati.cloud/api/v2/transaction?	https://api.iatistandard.org/datastore/transaction/select?
https://datastore.iati.cloud/api/v2/budget?	https://api.iatistandard.org/datastore/budget/select?

Select data fields to return

The transaction and budget endpoints in the iati.cloud API only return a subset of the data fields available in IATI. In contrast, the IATI Datastore v3 API returns all available fields. The fl= parameter can be used to limit the fields returned to match the iati.cloud output.

iati.cloud	IATI Datastore version 3 field selection
https://datastore.iati.cloud/api/v2/transaction?	&fl=iati_identifier,reporting_org_ref,title_narrative,description_narrative,recipient_country_code,recipient_region_code,sector_code,transaction_ref,transaction_humanitarian,transaction_transaction_type_code,transaction_transaction_date_iso_date,transaction_value_currency,transaction_value_value_date,transaction_value,transaction_provider_org_ref,transaction_provider_org_provider_activity_id,transaction_provider_org_type,transaction_provider_org_narrative,transaction_receiver_org_ref,transaction_receiver_org_receiver_activity_id,transaction_receiver_org_type,transaction_receiver_org_narrative,transaction_disbursement_channel_code,transaction_sector_vocabulary,transaction_sector_code,transaction_recipient_country_code,transaction_recipient_region_code,transaction_flow_type_code,transaction_finance_type_code,transaction_aid_type_code,transaction_aid_type_vocabulary,transaction_tied_status_code,transaction_description_narrative,default_currency,xml_lang,default_flow_type_code,default_aid_type_code,default_tied_status_code,humanitarian
https://datastore.iati.cloud/api/v2/budget?	&fl=iati_identifier,reporting_org_ref,reporting_org_type,recipient_country_code,recipient_country_narrative,budget_type,budget_status,budget_period_start_iso_date,budget_period_end_iso_date,budget_value_currency,budget_value_value_date,budget_value,default_currency,xml_lang,default_flow_type_code,default_tied_status_code,default_aid_type_code,humanitarian

Change field names in query parameter

Names of elements and attributes are based upon the names and path provided in the IATI Schema. Each sequence of non-alpha characters is replaced with a ‘_’.

For example:

IATI names	IATI Datastore version 3 names
iati-identifier	iati_identifier
transaction/value	transaction_value
transaction/value/@currency	transaction_value_currency
transaction/description/narrative/@xml:lang	transaction_description_narrative_xml_lang

A full list of the names and paths, plus the data type of each element and attribute can be found in the Activity Summary Table.

Fields with name changes between iati.cloud and IATI Datastore v3 include:

iati.cloud names	IATI Datastore version 3 names
default_lang	xml_lang
reporting_org_type_code	reporting_org_type
location_id_code	location_location_id_code
location_id_vocabulary	location_location_id_vocabulary
location_reach_code	location_location_reach_code
location_class_code	location_location_class_code
country_budget_items_budget_description_narrative_lang	country_budget_items_budget_item_description_narrative_xml_lang
country_budget_items_budget_description_narrative_text	country_budget_items_budget_item_description_narrative
planned_disbursement_value_date	planned_disbursement_value_value_date
planned_disbursement_provider_org_narrative_lang	planned_disbursement_provider_org_narrative_xml_lang
planned_disbursement_provider_org_narrative_text	planned_disbursement_receiver_org_narrative
transaction_type	transaction_transaction_type_code
transaction_date_iso_date	transaction_transaction_date_iso_date
transaction_value_date	transaction_value_value_date
transaction_provider_org_narrative_lang	transaction_provider_org_narrative_xml_lang
transaction_provider_org_narrative_text	transaction_provider_org_narrative
location_id_code	location_location_id_code
transaction_disburstment_channel_code	transaction_disbursement_channel_code
result_indicator_title_narrative_lang	result_indicator_title_narrative_xml_lang
result_indicator_title_narrative_text	result_indicator_title_narrative
result_indicator_description_narrative_lang	result_indicator_description_narrative_xml_lang
result_indicator_description_narrative_text	result_indicator_description_narrative
result_indicator_reference_vocabulary_uri	result_indicator_reference_indicator_uri
result_indicator_baseline_comment_narrative_lang	result_indicator_baseline_comment_narrative_xml_lang
result_indicator_baseline_comment_narrative_text	result_indicator_baseline_comment_narrative
result_indicator_period_target_comment_narrative_lang	result_indicator_period_target_comment_narrative_xml_lang
result_indicator_period_target_comment_narrative_text	result_indicator_period_target_comment_narrative
result_indicator_period_actual_comment_narrative_lang	result_indicator_period_actual_comment_narrative_xml_lang
result_indicator_period_actual_comment_narrative_text	result_indicator_period_actual_comment_narrative
result_document_link_title_narrative_lang	result_document_link_title_narrative_xml_lang
result_document_link_title_narrative_text	result_document_link_title_narrative
result_document_link_description_narrative_lang	result_document_link_description_narrative_xml_lang
result_document_link_description_narrative_text	result_document_link_description_narrative
result_indicator_document_link_title_narrative_lang	result_indicator_document_link_title_narrative_xml_lang
result_indicator_document_link_title_narrative_text	result_indicator_document_link_title_narrative_text

Select results to return

The Datastore v3 API has a limit of 1000 documents returned at once. This is to ensure data is of a size that can be handled by most browsers and will be returned in a reasonable timescale.

As with iati.cloud, the number of documents to return is set by the rows parameter. To return all documents in a query, users will need to repeatedly call the API and loop through the available pages by using the start parameter e.g.

• To get the first 1000 documents: &rows=1000&start=1
• To get the second 1000 documents: &rows=1000&start=1001

Results that cannot be returned

The IATI Datastore v3 does not have inbuilt currency conversion to USD. Users should take the value date of the financial value and choose their own method for converting the currency. IATI.cloud was based on the average IMF conversion rate for a given month.

The following endpoints, with CSV output only, do not have an equivalent in IATI Datastore v3:

• https://datastore.iati.cloud/api/v2/activity-sector?
• https://datastore.iati.cloud/api/v2/activity-country?

Comparison of IATI Datastore version 3 API to iati.cloud API

The two APIs are set up in very similar ways. However, the IATI Datastore API v3 is built to allow users greater flexibility when filtering data. Notably, all IATI data can be searched and returned using the activity, transaction and budget endpoints.

The IATI Datastore version 3 also delivers more timely updates, has lower costs and has an API Contract.

Do note that the IATI Datastore does not process IATI data. Any post-processing work, such as currency conversion and country/region/sector expansion, will sit down-stream of the Datastore API and will be considered in IATI’s upcoming work.

IATI Datastore version 3	iati.cloud
API subscription key is needed	No subscription key needed
All IATI data can be searched and returned for each endpoint	A subset of IATI data can be searched for each endpoint
All IATI data can be returned for each endpoint	A subset of IATI data can be returned for each endpoint
All field names match the IATI Standard	Not all field names match the IATI Standard
Narrative searching available on all narrative fields	Narrative searching available on some narrative fields
Paginated results	No pagination of results
No USD currency conversion	Some USD currency conversion
No CSV activity sector and country expansion	CSV activity output can return one country or one sector per row

Further support

The IATI Datastore has been developed by the IATI Secretariat. For additional support on using (or transitioning over) to the new API, please contact the IATI Helpdesk: [email protected]. IATI’s Secretariat will happily set up a call to help get you started.