Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

The International Organization for Migration (IOM) is the leading inter-governmental organization in the field of migration and works closely with governmental, intergovernmental and non-governmental partners. It is dedicated to promoting humane and orderly migration for the benefit of all and works to help ensure the orderly and humane management of migration, to promote international cooperation on migration issues, to assist in the search for practical solutions to migration problems and to provide humanitarian assistance to migrants in need, including refugees and internally displaced people.

The Displacement Tracking Matrix (DTM) continually tracks and monitors displacement across countries allowing IOM to identify the locations to which IDPs have chosen to settle. The location and population of these IDPs are recorded and further in-depth assessments are conducted to identify the multi-sectorial needs of the displaced.

IOM is currently working on a global DTM out of their office in Geneva. The goal is to produce a monthly combined DTM for all countries, starting with camp status (the type of DTM usually shared on HDX), and about 25 columns (which they refer to as "indicators").  The global DTM will be HXL tagged. We've also begun discussions between UNHCR (Laurent Pitoiset) and IOM (Muhammad Rizki) to ensure that the HXL tags they use for refugee and migrant data are well-aligned, and each is interested in becoming a consumer of the other's data.

Country DTMs

(Unless otherwise noted, all datasets are shared by IOM and consist of a single DTM round.)

Country DTMs are not well-standardised, though IOM in Geneva is starting an initiative to define a common core set of columns. Some countries use databases to manage their DTMs, while others are entirely spreadsheet based. Note that as of 2016-09-23, most of the DTMs on HDX are operationally out-of-date (shared once but not updated regularly).


HDX dataset

Date

Notes

DTM Cameroon Round III

2016-04-30


IOM DTM South Sudan Dataset

2015-10-31

Raw DTM (with some sensitive fields removed). Much-more granular than the other examples, with 175 columns, and structured as a survey. This lets us see what the DTM originally looks like, before it’s digested into the high-level reports that are more typically shared publicly. Includes ADM1, ADM2, “Payam”, and “Village”, as well as site id and lat/lon. Lists number of IDPs and households.

Nepal - IOM DTM Earthquake Dataset

2015-11-25


IOM DTM Mozambique Dataset Round 1

2015-04-30


Libya - IOM DTM Dataset Round 1

2016-01-10


IRAQ - IOM DTM Datasets

2016-01-20

Multiple rounds.

Haiti - IOM DTM Datasets

2015-12-30

Multiple rounds.

Nigeria - IOM DTM datasets

2016-01-28

Multiple rounds.

Cameroon - IOM DTM Datasets

2015-11-30


Libya - IOM DTM Round 2

2016-03-01


IOM DTM Nepal Dataset Round 5

2015-11-03


IOM DTM Iraq Dataset Round 33

2015-11-30


Iraq - Displacement Tracking Matrix (DTM) 14 Sept 2014

2014-09-14

Shared by OCHA Iraq.

Iraq - Affected Persons Locations (DTM) 7 August 2014

2014-08-07

Shared by OCHA Iraq.

Iraq - Affected Persons Locations(DTM) 24 August 2014

2014-08-24

Shared by OCHA Iraq.

IDPs Data by Location in Iraq

2014-09-01

Shared by HDX team. Multiple rounds.

Iraq - Affected Persons Locations(25 November 2014)

2014-11-25

Shared by OCHA Iraq.

Iraq - Affected Persons Locations(Jun to July 2014)

2014-07-02

Shared by OCHA Iraq.

Malawi - Displacement Tracking Matrix

2015-05-20

Shared by OCHA ROSA.

Malawi - Displacement Tracking Matrix

2015-03-06

Shared by OCHA ROSA.

Malawi - IDP Site

2015-03-06

Shared by OCHA ROSA.

IDPs Data by region in Mali

2015-05-31

Shared by OCHA Mali.


Data Structure - First Analysis

Some spreadsheets have "master list" in the title and some do not.

Taking the latest master list for a few countries, unfortunately we see that the DTMs do not have a fixed structure. 

Below are examples of different headings for various DTM spreadsheets:

Libya: rd4_DTM_Master_List_Jun2016.xlsx

This is at an aggregated rather than survey level, has several sheets and 2 rows of headers for the main sheet entitled "1-DTM Round 4 Dataset":

Shabiya_Name_ENBaladiya_IDBaladiya_Name_ENBaladiya_Name_ARLatLongis area assessed by DTM? (Y,N)IDP HouseholdsIDP IndividualsIDP households displaced in 2011Type of Displacement 2011Baladiya of Origin 2011IDP households displaced 2012- mid-2014Type of Displacement 2012- mid-2014Baladiya of Origin 2012- mid-2014IDP households displaced after mid-2014Type of Displacement after mid-2014Baladiya of Origin after mid-2014Migrant Individuals in BaladiyaMigrant Individuals in Detention Centers in BaladiyaMigrant Individuals crossing BaladiyaReturnee HouseholdsReturnee Individualshouseholds displaced by general violence reasonshouseholds displaced by special security reasonshouseholds displaced by economic ReasonsArea have IDPs in Rented_House_PaidArea have IDPs in Rented_House_NotPaidArea have IDPs with Host community - relativesArea have IDPs with Host community - non-relativesArea have IDPs in schoolsArea have IDPs in Public_BuildingArea have IDPs SquattingArea have IDPs in Unfinished_BuildingArea have IDPs in Abandoned_ResortsArea have IDPs in Collective_NonFormal settlementsArea have IDPs where shelter type is unknown
ADM2_Shabiya_Name_ENADM3_Baladiya_IDADM3_Baladiya_Name_ENADM3_Baladiya_Name_ARLatitudeLongitudeArea assessed by DTMIDPs In Baladiya_HHIDPs In Baladiya_INDIDPs In Baladiya HH_2011Origin Type 2011Origin 2011IDPs In Baladiya_HH 2011_2014Origin Type 2011_2014Origin 2011_2014IDPs_In_Baladiya_HH 2014+Origin Type 2014+Origin 2014+Migrants in BaladiyaMigrants in Detention CenterCrossing MigrantsReturnees HHReturnees IndDisplacement for violenceDisplacement for SecurityDisplacement for EconomicRented accommodation (self-pay)Rented accommodation (paid by others)Host families who are relativesHost families who are not relativesSchoolsOther public buildingsSquatting on other people’s properties (e.g. in farms, flats, houses)In unfinished buildingsIn deserted resortsIn Informal Settings (e.g. tents, caravans, makeshift shelters)Unknown


Nigeria: rd10_DTM_Master_List_Jun2016.xlsx 

This is at an aggregated level and has just one sheet "Sheet1" with a single line header:


LGAWARDSTATUSLATITUDELONGITUDESITE MANAGEMENT AGENCY (SMA)SMA TYPEREGISTRATION ACTIVITYSUPPORT WASHSUPPORT HEALTHSUPPORT SHELTER/NFISUPPORT FOODSUPPORT PROTECTIONSUPPORT EDUCATIONSUPPORT LIVELIHOODSITE CLASSIFICATIONSITE TYPELAND OWNERSHIPCOMMON SHELTER TYPENO OF HOUSEHOLDSINFANTS MALEINFANTS FEMALECHILDREN MALECHILDREN FEMALEYOUTH MALEYOUTH FEMALEADULT MALEADULT FEMALEELDERLY MALEELDERLY FEMALETOTAL NUMBER OF IDPSPREVIOUSLY BEEN DISPLACEDINTENDED RETURN AREAWHY NOT RETURNNO SHELTERTENTSMAKESHIFTINDOORSACCESS ELECTRICITYACCESS SAFE COOKINGHAVE PRIVATE AREASHAVE MOSQUITO NETSMOST NEEDED NFIWATER SOURCE LOCATIONDRINKING WATER SOURCEWATER CONSUMPTIONDRINKING WATER POTABLEDRINKING WATER QUALITY COMPLAINTSLATRINE CONDITIONFUNCTIONING TOILETGARBAGE DISPOSALSOLID WASTE PROBLEMHAND WASHING STATIONSHYGIENE PROMOTION CAMPAIGNOPEN DEFECATIONACCESS TO FOODACCESS TO MARKETDISTRIBUTION FREQUENCYOBTAINING FOODMALNUTRITION SCREENINGMOST PREVALENT HEALTH PROBLEMACCESS TO MEDICINEACCESS TO HEALTH FACILITYLOCATION HEALTH FACILITYHEALTH FACILITIES PROVIDERACCESS TO EDUCATIONEDUCATION LOCATIONATTENDING SCHOOLMAJORITY OCCUPATIONACCESS TO INCOMELIVESTOCKCULTIVATION

Iraq: rd_50_DTM_Master_List_July2016.xlsx

This is at an aggregated level and has two sheets. The main sheet is "DTM DATASET" with a two line header:


Location of Displacement

Governorate of originShelter typePeriod of displacementLink for Map
Place idGovernorateDistrictLocation name
in English
Location name
in Arabic
LatitudeLongitudeOCHA
admin 1
OCHA
admin 2
OCHA
PCode
FamiliesIndividualsAnbarBabylonBaghdadBasrahDahukDiyalaErbilKerbalaKirkukMissanMuthannaNajafNinewaQadissiyaSalah al-DinSulaymaniyahThi-QarWassitCampHost familiesHotel/MotelInformal settlementsOther shelter typeReligious buildingRented housesSchool buildingUnfinished/Abandoned buildingUnknown shelter typePre-June14 June-July14 August14Post September14 Post April15 Post March16Open Street MapGoogle MapBing Map


Yemen: r4_DTM_Master_List_Apr2016.xlsx

This has aggregated level data in different sheets. Sheet "Returnee hybrid data Locationlv" which has a single line header:

AreaAssessmentIDAssessmentRoundGovernorateGovernoratePCodeDistrictENDistrictPCodeRetIDPsFromDistrict_HHOfficialPlaceENPCodeReturnee_ConflictHHReturnee_DisasterHHReturnee_AccessibilityReturnee_WomenPerReturnee_MenPerReturnee_FemChildPerReturnee_MaleChildPerLatitudeLongitude

Sheet "IDPs Raw data" has this single line header:


AreaAssessmentIDAssessed GovernorateGovernorate PCodeAssessed DistrictDistrict PCodeInterview DateAssessment RoundSite NameSite Name ASite PCodeLatitudeLongitudeSite TypeHH IDPs in DistrictIndividual IDPs in DistrictArrival YearArrival Month in 2015Site Estimated Conflict HHsSite Estimated Disaster HHsSite Estimated IDPs HHsFamily SizeAccessibilityWomen %Men %Female Children %Male Children %Women Estimeted #Men Estimeted #Female Children Estimeted #Male Children Estimeted #CampsUsing rented accomodationWith host families who are relatives (no rent fee)With host families who are not relatives (no rent fee)Using schools, Health facilities, religious buildingUsing private or public buildingIn informal settlement (grouped families) in urban areasIn informal settlement (grouped families) in rural areasOut of settlement (isolated families)


Common Fields

From the headers above, there are certain fields which are common to all. These are:

  1. Latitude
  2. Longitude
  3. Location Name (ward, district etc.)
  4. Families/Households (most DTMs contain the number of individuals as well as the number of households; individuals and households are the key indicators for the DTM, and most of the other columns are disaggregation facets).
  5. Type of shelter (although this is represented in different ways eg. individual columns for each type or one column with the type)

Data Structure - Second Analysis

IOM provided new information - that their datasets are classified into 4 categories:
  1. Site assessment
  2. Baseline assessment
  3. Flow monitoring
  4. Survey

A second analysis was performed on a selection of datasets labelled as site assessment:

IOM DTM Ecuador - Site Assessment Data - Spontaneous Sites


File: Site_Assessments_Refugios_R4.xlsx

This appears to be at a survey level. There are 3 sheets with the sheet "uploaded_form_izr7j1" containing the data.

IOM DTM Somalia - Site Assessment Data


File: IOM_SOM_DTM_Master_Data_Site_Assessments_Final_Rou...XLS
 


Nigeria - IOM DTM Dataset (June 2016) - Site assessment data

File: rd10_DTM_Master_List_Jun2016.xlsx

This is at an aggregated level and has just one sheet "Sheet1" with a single line header:


LGAWARDSTATUSLATITUDELONGITUDESITE MANAGEMENT AGENCY (SMA)SMA TYPEREGISTRATION ACTIVITYSUPPORT WASHSUPPORT HEALTHSUPPORT SHELTER/NFISUPPORT FOODSUPPORT PROTECTIONSUPPORT EDUCATIONSUPPORT LIVELIHOODSITE CLASSIFICATIONSITE TYPELAND OWNERSHIPCOMMON SHELTER TYPENO OF HOUSEHOLDSINFANTS MALEINFANTS FEMALECHILDREN MALECHILDREN FEMALEYOUTH MALEYOUTH FEMALEADULT MALEADULT FEMALEELDERLY MALEELDERLY FEMALETOTAL NUMBER OF IDPSPREVIOUSLY BEEN DISPLACEDINTENDED RETURN AREAWHY NOT RETURNNO SHELTERTENTSMAKESHIFTINDOORSACCESS ELECTRICITYACCESS SAFE COOKINGHAVE PRIVATE AREASHAVE MOSQUITO NETSMOST NEEDED NFIWATER SOURCE LOCATIONDRINKING WATER SOURCEWATER CONSUMPTIONDRINKING WATER POTABLEDRINKING WATER QUALITY COMPLAINTSLATRINE CONDITIONFUNCTIONING TOILETGARBAGE DISPOSALSOLID WASTE PROBLEMHAND WASHING STATIONSHYGIENE PROMOTION CAMPAIGNOPEN DEFECATIONACCESS TO FOODACCESS TO MARKETDISTRIBUTION FREQUENCYOBTAINING FOODMALNUTRITION SCREENINGMOST PREVALENT HEALTH PROBLEMACCESS TO MEDICINEACCESS TO HEALTH FACILITYLOCATION HEALTH FACILITYHEALTH FACILITIES PROVIDERACCESS TO EDUCATIONEDUCATION LOCATIONATTENDING SCHOOLMAJORITY OCCUPATIONACCESS TO INCOMELIVESTOCKCULTIVATION



Haiti - IOM DTM Dataset (June 2016) - Site assessment data

File: rd25_DTM_Master_List_Mar2016.xlsx


Conclusion

Given the lack of any consistency between the DTM Master List spreadsheets, writing a one size fits all automated data checker and cleaner for all of them is challenging. It would involve placing a great deal of "intelligence" into the cleaning program with the possibility that errors are introduced during cleaning for example, by accidentally matching the wrong column heading when making an algorithm to match a very diverse range of names. It may be possible to write cleaners per country but the effort involved would be large.

A better approach is to try to encourage the different offices to use a similar template for their spreadsheets, as from this starting point, writing a cleaner would not be too onerous. If such a template were introduced, it could be HXLated from day one. How easy it would be to invent a template that covers the range of needs is debatable but it is likely to be much easier than trying to process greatly varying spreadsheet formats.

Appendix: IOM Global DTM information

The following files contain a data dictionary and partial sample for the in-progress global DTM, as supplied by IOM in Geneva.

This Google Sheet contains proposed HXL hashtags for the global DTM: https://docs.google.com/spreadsheets/d/1gifTnrz9A2fZ8Tuwg-EClsFvu4QEbC4OUtdgb61dXGs/edit?usp=sharing 

  • No labels