How to use the Datastore API
Learn how to form Datastore version 3 API queries and return IATI data for further analysis.
This guide provides information on how to form Datastore version 3 API queries. A subset of the Datastore API’s features and options is provided below. A full list of features that IATI is committed to maintain is outlined in the API contract. All features in the SOLR API documentation can be used as part of the Datastore v3 API, however those not in IATI’s API contract may be changed and updated over time.
- Datastore v3 API contract - The contract contains the full list of search and filtering options IATI is committed to maintain in the Datastore v3 API. Key features are highlighted below.
- SOLR documentation - This contains all of the features available via SOLR. The list of SOLR features is larger than those IATI is committed to maintain in the Datastore v3 API.
The Datastore API can be accessed through the IATI’s API Gateway. See the API Gateway guidance document for details.
Examples of complex API calls can be found here.
Each user is required to have a free account and include their subscription key when making queries. The details below show how to form the API query, as well as how to do so using the API Gateway’s Query Field.
Please note, users of the IATI Datastore will need to have technical competency and understand the basic structure and elements of the IATI Standard. Please see Tools and resources for data use for other ways of accessing IATI data.
Available endpoints:
Return data formatted as either one activity, transaction or budget per row. Each endpoint returns all activity data related to that activity, transaction or budget.
- https://api.iatistandard.org/datastore/activity/select?
- https://api.iatistandard.org/datastore/transaction/select?
- https://api.iatistandard.org/datastore/budget/select?
In the Query Field use the collection parameter, choose a drop down option of activity, transaction or budget:
| Parameter | Value |
|---|---|
| collection | activity |
How to add activity filters
The first filter is added using the sequence `q=`, following this, further filters can be added using `AND`, `OR`, `NOT` and + (+ means AND/OR but cannot be expressed using text).
Attribute Code Searches
- Search for a specific attribute code: q=sector_code:11110
- Search for multiple attribute codes: q=recipient_country_code:(UG TZ)
- Search for one code OR another: q=sector_code:11110 OR transaction_sector_code:11110
- Search for one code AND another: q=recipient_country_code:UG AND sector_code:11110
- Search for one code AND/OR another: q=recipient_country_code:UG + sector_code:11110
- Search for one code but NOT another: q=recipient_country_code:UG NOT sector_code:11110
Narrative Searches
- Freetext search in a specific narrative: q=description_narrative:rabbit
- Freetext search in multiples narratives: q=title_narrative:rabbit OR description_narrative:rabbit OR iati_identifier:rabbit OR transaction_description_narrative:rabbit
- Freetext search for two words in a specific narrative: q=title_narrative:(rabbit AND production)
- Freetext search for a phrase in a specific narrative: q=title_narrative:“rabbit production”
Note, the iati-identifier is counted as a narrative field in the IATI Datastore. Use quotes to find a specific activity e.g. q=iati_identifier:“US-GOV-1-AID-OAA-TO-14-00001”
Further narrative search options
- * = wildcard, returns matches with zero or more letters
E.g. apple* returns apple, apples and appleseeds
- ? = single character, returns matches which include a single character
E.g. te?t returns test and text
Note: to use * and ? as wildcards in narrative searches, the search term MUST NOT be quoted. If quoted, * and ? symbols will be searched for in the narrative text.
How to search dates and financial values
The API allows users to select date and financial values within a certain range. This is added in the format [* TO *].
- q=transaction_value:[* TO 10000] finds all field values less than or equal to 10000.
- q=activity_date_iso_date:[2021-01-01T00:00:00Z TO *] finds all activities that have an activity date later 1st Jan 2021.
Note: The SOLR API cannot search for both the type and iso-date within a single activity-date element. This means you cannot select only start or end dates to search by; this needs to be done by the user or programme once the JSON/XML/CSV output has been retrieved.
In the Query Field use the q parameter, then add the combination of filters required using the guidance above e.g.
| Parameter | Value |
|---|---|
| q | title_narrative:(rabbit AND production) |
Return specific columns
- &fl=description_narrative,sector_code
In the Query Field use the fl parameter, then add the list of data fields to return e.g.
| Parameter | Value |
|---|---|
| fl | description_narrative,sector_code |
How to select the format:
- &wt=json
- &wt=xml
- &wt=csv
In the Query Field use the wt parameter, choose one of the three options: json, xml or csv
| Parameter | Value |
|---|---|
| wt | json |
How to select number of rows
If not used, a default of 10 results will be returned.
A maximum of 1000 rows can be returned per API call. Use the &start parameter to select which row to start from.
- &rows=50
- &start=0
In the Query Field use the rows and start parameter, then specify the number required e.g.
| Parameter | Value |
|---|---|
| rows | 50 |
| start | 0 |
Names of elements/attributes:
Names of elements and attributes are based upon the names and path provided in the IATI Schema. Each sequence of non-alpha characters is replaced with a ‘_’.
For example:
| IATI Name | API Name |
|---|---|
| iati-identifier | iati_identifier |
| transaction/value | transaction_value |
| transaction/value/@currency | transaction_value_currency |
| transaction/description/narrative/@xml:lang | transaction_description_narrative_xml_lang |
A full list of the names and paths, plus the data type of each element and attribute can be found in the Activity Summary Table.
Character encoding
The API will encode certain characters. Either the character or the encoding can be used when typing/pasting in an API. For example: