Data Collection Methods: Top-Down vs. Bottom-Up

When applied to organizational management, the term “top-down” can suggest the image of an inflexible boss dictating policy and tasks to subordinates rather than motivating and encouraging team members to let their good ideas rise from the “bottom-up.” It has a very different meaning, however, we describing data collection methods.

When building and scaling a field data collection and analysis system, a “top-down” approach is no better or worse than a “bottom-up” strategy but rather one of two data collection methods used to accomplish the same goal – aggregating and analyzing field data to inform business decisions.

Both of these data collection methods have their pros and cons in terms of optimization and scalability so let’s look at how both were used by the BRAC Skills Development Programme (SDP) – highlighting the failures and successes of each approach and how TaroWorks’ field service management app facilitated the work.

Image: TaroWorks being used on a smartphone at an SDP learner’s house. Source: BRAC

Defining Data Collection Methods

• Top-down data collection methods involve creating an overarching system of data collection and analysis before detailing and fleshing out subsystems under it. These subsystems can then have even more detailed and refined subsystems beneath them – as appropriate. As an example, for information gathering – this may look like creating a system of reporting for the management team first and then detailing how the field workers get the data to the management afterwards.
• Bottom-up data collection methods involve piecing together systems at the field level to give rise to more complex systems further up the information ladder, in the process making these original systems subsystems within the final design. This requires the elementary or bottom rung systems to be fleshed out in great detail before connecting them together to form the larger system. Using the information gathering example, this would look like first creating a data gathering model for the field workers and then deciding how that data will be transferred through to the management team.

BRAC SDP’s Original System

At the beginning of SDP’s data collection and analysis effort, a simple Excel spreadsheet – using the Visual Basic for Applications programming language –  was designed by BRAC SDP for Programme Organizers (PO) – essentially the people working in the field – to gather and input data. POs used to collect data and record it by pen and paper before entering the information into the  spreadsheet. They would then send the compiled information file to their District Manager (DM). Each DM would supervise six POs and they in turn would be supervised by an Area Manager (AM) who handled three DMs at a time. DMs would compile six reports and send them to the AM who would then have to compile the eighteen reports coming in from various regional DMs into a final report that would be sent to the Head Office (HO).

Three Problems with the Original Data Collection Method

• Data Entry Errors: Manual data input by the POs was vulnerable to mistakes and gaps, corrupting the whole chain of reports with incorrect or missing data – such as misspelled names or missing demographic information.
• Time-Consuming Process: The process was time-consuming and not in real-time, creating delays in responses. For example, the six-month BRAC SDP had a 10% quota for people with disabilities, but as it took two months for the participants’ information to reach the Head Office, there was very little that could be done a third of the way into the program if the quota had been unfulfilled.
• No Follow-Up: POs were unable to follow up on program participants as a lot of time got wasted in just reporting.

Testing a Bottom-Up Approach

To mitigate problems of using spreadsheets, a new data collection and analysis method was conceived and deployed using a bottom-up approach to system design. Trainers from BRAC SDP trained POs in the use of TaroWorks’ mobile offline data collection app, which uploaded results to a Salesforce.com cloud database for analysis by District Managers, Area Managers above them and then finally by the Head Office. The data collection was possible without the internet using TaroWorks as it would store the data locally and later upload the information to the Salesforce cloud database when a mobile data or wifi connection was available. The whole system was designed  to be user-friendly for POs.

  Bottom-Up Data Collection Design. Source: BRAC

Three Problems with Bottom-Up Data Collection Method

• Technical Expertise: Many of the POs were technologically challenged, having either never used smartphones before or only those of the lowest quality. Thus, trainers had the duty of not only teaching the POs how to use TaroWorks but more basically how to use the mobile devices on which it was installed, which took many additional hours.
• Cost of Mobile Devices: POs either did not have tablets and smartphones or they were afraid of losing them in the field, which would lead to data loss. As a result, BRAC SDP provided all of them with tablets which was a huge financial burden.
• Training Issues: As POs were trained before their supervisors received training, for a time, they had to directly contact the Head Office whenever they encountered a problem, eroding the management chain and burdening the HO with specific technical support issues from the field.

We wondered if the first two problems, however, might be program-specific and concluded they were after completing a similar data collection and analysis project in Satkhira, Bangladesh. This separate and subsequent project was called “Livelihood Skills Training for Out of School Children in Satkhira  & Khulna” and was funded by UNICEF. It utilized the same bottom-up approach in 32 different branches in the region. As the project was concentrated on a particular area, the data was able to provide insight into trends and the model proved to be adequate for the purpose.

The POs were strictly instructed to not use any paper for data collection and recording for the Satkhira project. A survey among the POs found that 70% of them already had smartphones and the remaining 30% were encouraged to buy them by offering monthly stipends for the devices. As a result, the first two problems listed above as issues in the bottom-up approach were less prevalent in this group.

The third problem, however, involved the very nature of bottom-up approaches and that led the BRAC SDP to try out a second of two data collection methods – top-down approach – as a potential alternative design for data collection and analysis work.

Testing a Top-Down Approach

Here HO staff first trained AMs and then DMs (staff at the top of the organizational structure) who later went on and trained their POs – the front line field teams. This approach involved using Google Forms as the primary data gathering tool. Surveys were prepared using that software and sent out to all POs in 117 branches. Graphs and charts created from the data collected with Google Forms were introduced for DMs and AMs to help sort data and make reports. As the data involved less text-based responses reports were relatively easy to create.

Issues with Top-Down Data Collection Method

• Internet Access: A failing of this system was that it required an internet connection to collect and save data on mobile devices, something we couldn’t always count on in the remote areas where the POs worked.
• User Support: Whenever POs faced problems they went to their pre-trained DMs for advice. Usually the problems would get solved at that point and if not they would be forwarded directly to HO for resolution. This was required due to unavailability of proper management and an efficient system at all stages.

Data Collection Methods: Did Bottom-Up or Top-Down Win the Day?

After experimenting with both bottom-up and top-down data collection methods, BRAC SDP had to select the favored approach going forward as we expanded data collection and analysis efforts to an additional 135 new branches. As it was observed that the top-down approach was more beneficial when dealing with projects of larger size, BRAC SDP opted to go for that model as we scaled-up. A bottom-up approach has its merits but only when it is used for small or pilot projects.

Given its features like offline data collection and field force management, TaroWorks was adopted for the expanded top-down approach as well. The BRAC SDP staff who had used TaroWorks in the bottom-up test became a resource to help train BRAC team members who had previously used Google Forms.

Given what we’d learned along the way, improvements were also made to our design and implementation of the expanded top-down program.

• Training: Trainers from BRAC SDP were present when the AMs and DMs trained POs in the use of TaroWorks and Salesforce to ensure that the training was on the right track and there weren’t any mistakes.
• Troubleshooting: Troubleshooting and problem solving related to data collection and analysis in the field has been made easier by using the WhatsApp communications app  among POs, DMs and AMs. Whenever a PO faces any issues related to their use of TaroWorks, they post questions and screenshots to their WhatsApp group, which the DM can see and resolve accordingly. The benefit of this is the DM does not have to individually inform each PO of the solution and can preemptively clear up issues for all who report to him or her.

Using Collected Data

Image: Light engineering factory in Tongi. Source: BRAC/Farzana Trina Chowdhury

All the system design in the world won’t benefit your data collection and analysis efforts if you don’t also have a clear sense of what you’d like to use the data for and how it will offer both business insights and help drive a data culture in your organization.

Here are two examples of how the data we’ve collected using the top-down data collection method is being used at BRAC to foster continuous improvement of business operations and has helped us emphasize the importance of data in our organization.

• How BRAC Uses Field Data for Continuous Improvement
• Data-Driven Decision-Making: How BRAC Mines Answers From Metrics

Want To Demo TaroWorks?

Many TaroWorks customers previously used paper and spreadsheets to collect data, analyze metrics and run field operations. Let us show you how TaroWorks’ offline mobile field service app can help scale your business by digitizing sales and supply chain management, increasing agent network productivity and analyzing data in real-time for business insights.