The International Organization for Migration (IOM) is the leading inter-governmental organization in the field of migration and works closely with governmental, intergovernmental and non-governmental partners. It is dedicated to promoting humane and orderly migration for the benefit of all and works to help ensure the orderly and humane management of migration, to promote international cooperation on migration issues, to assist in the search for practical solutions to migration problems and to provide humanitarian assistance to migrants in need, including refugees and internally displaced people.
The Displacement Tracking Matrix (DTM) continually tracks and monitors displacement across countries allowing IOM to identify the locations to which IDPs have chosen to settle. The location and population of these IDPs are recorded and further in-depth assessments are conducted to identify the multi-sectorial needs of the displaced.
IOM is currently working on a global DTM out of their office in Geneva. The goal is to produce a monthly combined DTM for all countries, starting with camp status (the type of DTM usually shared on HDX), and about 25 columns (which they refer to as "indicators"). The global DTM will be HXL tagged. We've also begun discussions between UNHCR (Laurent Pitoiset) and IOM (Muhammad Rizki) to ensure that the HXL tags they use for refugee and migrant data are well-aligned, and each is interested in becoming a consumer of the other's data.
The initial overview and first analysis of the data structure of datasets have been moved into separate pages, so that this page contains the latest analysis (below).
Jump to Initial Overview
Jump to Data Structure - First Analysis
Jump to Data Structure - Second Analysis
Data Structure - Third Analysis
IOM suggested looking at baseline rather than site assessment data.
Yemen - IOM TFPM DTM Dataset (April 2016) - Baseline data
DTM Cameroon Round III - Baseline data
IOM DTM - Mosul crisis Baseline data
Libya - IOM DTM Dataset (June 2016) - Baseline data
Conclusion
Given the lack of any consistency between the DTM Master List spreadsheets, writing a one size fits all automated data checker and cleaner for all of them is challenging. It would involve placing a great deal of "intelligence" into the cleaning program with the possibility that errors are introduced during cleaning for example, by accidentally matching the wrong column heading when making an algorithm to match a very diverse range of names. It may be possible to write cleaners per country but the effort involved would be large.
A better approach is to try to encourage the different offices to use a similar template for their spreadsheets, as from this starting point, writing a cleaner would not be too onerous. If such a template were introduced, it could be HXLated from day one. How easy it would be to invent a template that covers the range of needs is debatable but it is likely to be much easier than trying to process greatly varying spreadsheet formats.
Appendix: IOM Global DTM information
The following files contain a data dictionary and partial sample for the in-progress global DTM, as supplied by IOM in Geneva.
This Google Sheet contains proposed HXL hashtags for the global DTM: https://docs.google.com/spreadsheets/d/1gifTnrz9A2fZ8Tuwg-EClsFvu4QEbC4OUtdgb61dXGs/edit?usp=sharing