...
Jump to Data Structure - Second Analysis
Data Structure - Third Analysis
IOM suggested looking at baseline rather than site assessment data.
Yemen - IOM TFPM DTM Dataset (April 2016) - Baseline data
DTM Cameroon Round III - Baseline data
IOM DTM - Mosul crisis Baseline data
Libya - IOM DTM Dataset (June 2016) - Baseline data
| Shabiya_Name_EN | Baladiya_ID | Baladiya_Name_EN | Baladiya_Name_AR | Lat | Long | is area assessed by DTM? (Y,N) | IDP Households | IDP Individuals | IDP households displaced in 2011 | Type of Displacement 2011 | Baladiya of Origin 2011 | IDP households displaced 2012- mid-2014 | Type of Displacement 2012- mid-2014 | Baladiya of Origin 2012- mid-2014 | IDP households displaced after mid-2014 | Type of Displacement after mid-2014 | Baladiya of Origin after mid-2014 | Migrant Individuals in Baladiya | Migrant Individuals in Detention Centers in Baladiya | Migrant Individuals crossing Baladiya | Returnee Households | Returnee Individuals | households displaced by general violence reasons | households displaced by special security reasons | households displaced by economic Reasons | Area have IDPs in Rented_House_Paid | Area have IDPs in Rented_House_NotPaid | Area have IDPs with Host community - relatives | Area have IDPs with Host community - non-relatives | Area have IDPs in schools | Area have IDPs in Public_Building | Area have IDPs Squatting | Area have IDPs in Unfinished_Building | Area have IDPs in Abandoned_Resorts | Area have IDPs in Collective_NonFormal settlements | Area have IDPs where shelter type is unknown |
| ADM2_Shabiya_Name_EN | ADM3_Baladiya_ID | ADM3_Baladiya_Name_EN | ADM3_Baladiya_Name_AR | Latitude | Longitude | Area assessed by DTM | IDPs In Baladiya_HH | IDPs In Baladiya_IND | IDPs In Baladiya HH_2011 | Origin Type 2011 | Origin 2011 | IDPs In Baladiya_HH 2011_2014 | Origin Type 2011_2014 | Origin 2011_2014 | IDPs_In_Baladiya_HH 2014+ | Origin Type 2014+ | Origin 2014+ | Migrants in Baladiya | Migrants in Detention Center | Crossing Migrants | Returnees HH | Returnees Ind | Displacement for violence | Displacement for Security | Displacement for Economic | Rented accommodation (self-pay) | Rented accommodation (paid by others) | Host families who are relatives | Host families who are not relatives | Schools | Other public buildings | Squatting on other people’s properties (e.g. in farms, flats, houses) | In unfinished buildings | In deserted resorts | In Informal Settings (e.g. tents, caravans, makeshift shelters) | Unknown |
Conclusion
Given the lack of any consistency between the DTM Master List spreadsheets, writing a one size fits all automated data checker and cleaner for all of them is challenging. It would involve placing a great deal of "intelligence" into the cleaning program with the possibility that errors are introduced during cleaning for example, by accidentally matching the wrong column heading when making an algorithm to match a very diverse range of names. It may be possible to write cleaners per country but the effort involved would be large.
...