The COTZ Baseline Survey: Part 1

[A ColaLife Mini Blog Mini Series by Rohit Ramchandani]

I’m thrilled to be writing my first ColaLife Blog entry. It’s been quite some time since I’ve written in this style, so its nice to be back in the saddle as it were. Hopefully, I can keep you as engaged as Simon does.

We’ll be posting about 3 different blog entries over the next few days, which will focus on my most recent trip to Zambia and the collection of our baseline data. I hope to share a few of my many impressions, observations and lessons learned over the past few weeks, some of which we hope can serve as a source of information and inspiration for others working on evaluations in rural/remote areas. While I will cover elements of our methodology and some of the more academic aspects of our work, I also hope to address some of the more practical issues not generally covered in academic literature that may serve as food for thought for others facing similar issues we encountered in the field. We’d also welcome comments and suggestions on how your projects may have handled similar challenges and situations.

Keemba Kids Keemba Centre and Shops
Scenes from Keemba

I’ve started writing this first entry from the middle of a rural community called Keemba, located in Monze, one of our control districts. I’m surrounded by inspiration, so no better place to start.

As ColaLife’s Public Health Advisor, and the Principal Investigator for the project, I’ve spent the better part of the last two years working with Simon and Jane on project development and planning. My primary focus has been the evaluation. Now, as we enter the implementation phase of COTZ and put all of that planning into practice, we wanted to share some important parts of this transition phase with all of you.

Enumerator trainingUpon my arrival in Lusaka on 24 July 2012, we immediately began preparations for training our enumerators – 25 of them to be exact – as well as our supervisors – 1 for each of our 4 districts. These are the front-line soldiers of field-work. And I don’t use the term soldiers loosely. These guys are up at the crack of dawn every day ready for the field, most of them away from their families for the entire data collection period, often walking kilometers a day in the hot sun, over rough terrain, to carry out the household surveys. Their experiences on the ground are case studies in human anthropology and perhaps some of the most interesting stories you’re likely to hear. They are the ones who capture the raw information that allows us to gain insight, and attempt to advance societies by improving quality of life.

I cannot stress enough the importance of this team – the individuals that make it up, its dynamic, and how these individuals are trained, treated, and compensated. The enumerators should have a sense of buy-in, they should feel a sense of ownership and investment, and they should be committed. They have to be able to put aside all of their personal biases, execute the purposefully designed surveys, and record the answers wearing totally neutral lenses. The success of an evaluation can live or die based on how this team comprehends the survey tools, specific questions, purpose of the study, and operates in the field.

The baseline survey team Supervisor briefs his team
The full baseline survey team | The Kalomo Supervisor briefs his survey team in the field

Our training took place at the University of Zambia. We trained our enumerators and supervisors over the course of 1 week, which included one day of field-testing in Kafue, a semi-rural town on the north bank of the Kafue River. To give you a general sense of what was covered during our week of training, I have included our training schedule below. This may be useful as a sample or template for field training, and others should feel free to adapt it as necessary.

TRAINING SCHEDULE

ACTIVITY SUGGESTIONS/NOTES
DAY 1 Opening remarks
Ice-breaker & roundtable introductions Names, where they are from, previous related experience
Overview of core team
Introduction to the ColaLife model
Roles & responsibilities – an introduction to the partners It helps to have representatives from the partner organizations there; the team should sense that they are part of something bigger
Overview of research design Pre-post test design; Four districts – 2 intervention & 2 matched controls; etc.
Review informed consent and related procedures This is a good place to discuss research ethics more broadly
Introduction and review of the household survey (paper-based) To get everyone participating, go around the table and have each person read a page of the survey, pause and then open the floor for questions. Good data is based on asking good questions, so make sure everyone is crystal clear on these questions.
DAY 2 Finish reviewing the household survey & discussing questions (paper-based)
Introduction to the translated version of the tool into local languages Nyanja/Chewa and Tonga
Introduction to the Samsung Galaxy Tablet (general layout, functions, and care) Wait till everyone has reviewed the paper surveys and are comfortable with the questions before breaking out the technology!Power it on and off, show how to turn on/off GPS, text predictor, airplane mode, etc. Show them the home button, put it to sleep and wake it up, swipe the screen, short/long press, play with the keyboard. Fiddle with it in every possible way so that everyone is totally at ease.
Introduction to Open Data Kit (ODK) software & electronic survey Tablets were set up so that ODK was the only icon on the screen. This helped ensure no confusion, no distractions, and easy access to the survey.
Mock interviews using electronic surveys/tablets Pair up the enumerators and have them interview one another, giving each of them the opportunity to run through the entire surveyAnother option is to do a “sharing circle”, where one volunteer sits in the middle and plays the role of a respondent, while everyone else takes turns asking questions and recording the answers. At the end we reviewed the recorded answers together to see if everyone got the same thing. We did this activity outdoors and split the enumerators into two language groups to do this effectively.
Day 3 Review of challenges and troubleshooting Ask the enumerators to take notes of challenges (related to the use of the tablet or specific survey questions) as they are doing their mock interviews
Overview of program evaluation techniques and survey methodology
Tips for working in the field This should include protocols to be followed, including those related to security/safety in the field
Review logistics for field testing
DAY 4 Field Testing
DAY 5 Review challenges and insights based on field testing and discuss how to troubleshoot problems raised
Sampling methodology for baseline Overview of what sites were to be covered per day by district, how and how many households and retailers were to be selected, how interviewees were to be selected, etc.
Logistics for baseline Which teams will cover which districts; Encourage review of materials over the weekend; Administrative details (i.e. per diems, items to bring, contracts, etc.)

 

HIGHLIGHTS & TIPS FROM THE TRAINING

Tablets are just one of the many examples of how we are continuously striving to innovate. The use of tablets to collect survey data is still relatively new and offers a number of advantages:

  • The potential to substantially reduce errors from data entry and transfer
  • Eliminating the potential issues associated with skips (i.e. when a particular response eliminates the need for specific questions to be asked, or prompts another set of questions), as they are automated
  • There’s no need to carry around excessive amounts of paper based surveys while in the field
  • It gives us the ability to see data coming in on an almost real-time basis and intervene much faster in order to correct potential problems

Samsung tabletWhile training the enumerators on proper use of the tablets, being able to project the screen image on the wall so that everyone could follow along was very helpful. Also, securing a room with windows made a big difference. Everyone is cooped up from morning till evening for an entire week, so its nice to be able to open a window and see the sunshine.

Time management was essential. We would start at 8:30am sharp, and end between 5 and 5:30pm each day. We allocated time for two 15-minute breaks and a one-hour lunch. It was a good idea to have tea/coffee and some snacks set up close by during the breaks.

As you will see from the picture above, we sat in at a U-shaped table so that everyone could see and hear each other clearly. The use of name-plates (i.e. folded pieces of paper with everyone’s names marked) was also very useful, and helped create a personal rapport amongst the team. Everyone was going to be working together for quite some time, so it was important to create that team camaraderie early on.

The training should be used as an opportunity to strengthen the tools and methodology, because if you’ve done a good job recruiting your enumerators, there is a plethora of local knowledge and previous experience upon which to draw. It was actually a good feeling when things would go wrong, because we knew we had the time to address them before going into the field.

Things went wrong that we could never have anticipated (i.e. buttons not being pushed in the right way, text predictor function changing responses, GPS functionality draining the battery faster, etc.). All of these things strengthened the training and allowed us to ensure the team was prepared to address the issues that arose. This is the one time when you want things to go wrong!

Also, its worth keeping in mind that nothing should be considered too simple to be explained. With this in mind, questions were encouraged throughout the process. Make sure that each enumerator understands every question, is comfortable with probing, and knows that they should be recording a response. We generally only get 1 chance once in the field. We went all that way, off the beaten path, to a very remote village with the specific task of asking that question, so skipping it is not an option! (note: this is different from recording “no response”, which is data).

Overall, the training went really well and everyone was very engaged. One of the most important things to do during training was to instil a sense of confidence in our enumerators. It was important to remind them that they were there for a reason – because of their previous training and experience, attention to detail, listening skills, punctuality, language skills, adaptability and work ethic. These are skills that every enumerator should possess. It’s no easy task trekking along the countryside talking to complete strangers for weeks at a time. But indeed, it is fascinating and truly inspiring!

More from our time in the field in a few days.

Comments

  1. Thanks Rohit! Zambia has seen a lot of research, especially by US-based universities. Very infrequently do enumerators receive the respect, attention and praise you’ve given them here. They are indeed a vital part of public health research in this country, and are often overlooked, overworked and (shamefully) underpaid. Too often data outcomes take precedence over the people who actually collect it. Thank you so much for highlighting their important role and emphasizing how critical a well-trained, well-treated, enthusiastic team of enumerators is to public health research. I hope other Western-based researchers wishing to work in Zambia read this blog post and take notes!

Trackbacks

  1. Bill Brieger says:

    RT @colalife: The COTZ Baseline Survey: Part 1 #colalife http://t.co/DwcDeEPM

  2. RT @colalife: The COTZ Baseline Survey: Part 1 #colalife http://t.co/LrvqqXKt