Today we are happy to share that VBAT project has met all its objectives and we are now moving ahead in our planning for the next steps. We announced the project in January 2011. The project was exciting for us because of a promising new concept: voice-based Web access. As part of the pilot project our aim was to test user acceptance of technology and trust in information received. It factored on how to enable people without prior exposure to technology and IVR in particular to interact with a computer through (mobile) phone and voice services, and trust the delivered information.
The success of this project relied on choosing a test-bed environment, and in that aspect we narrowed down on One World South Asia’sLifeLines India Project. With their agriculture helpline project, today they are able to answer farmers’ questions that are placed in their system by a farmer calling through a simple mobile phone. They currently have a database of more than 400.000 Q&A which allows them to answer more than 80% of the requests, the other 20% being answered by experts (enriching the database over the time).
After analyzing the requirements for the VBAT application in consultation with OWSA, Etienne and Stephane set about finalizing the test case for the focus group. The project team conducted a focus group experiment in August, 2011. Based on the key recommendations of the focus group that we shared with you, the refinement of IVR application was completed by October, 2011 and the field workers were then trained in using the VBAT application by November, 2011. We ran the pilot till January, 2012 and today we are happy to share some of the key outcomes of the project.
- Overall, the respondents were happy with the new system. They were unanimous in their approval of getting the information in real time rather than the existing waiting time of twenty-four hours. ‘We don’t want to wait’ was the most common feedback.
- While they found the three selected FAQs interesting, they were of the view that these were inadequate and wanted more content options.
- As part of the test the respondents were also asked to compare human-recorded answer versus answers automatically generated by a Text-To-Speech (TTS) engine. There was no significant difference in acceptance and trust between the TTS version and the human-recorded version.
Apart from the survey questions, the farmers had interesting observations related to the helpline initiatives in general. Specifically, what stood out was:
- The farmers have a positive opinion about the information provided and even proposed alternate ways to disseminate the relevant information. For instance, they commented that it would be very helpful to have a dedicated radio broadcast with the FAQ’s.
- From information to action: Farmers specifically mentioned the need for a better supply chain management. ‘It is good to get the information but we also need the same to be available in our villages’ was the popular refrain.
From a quantitative perspective, compared to the total queries received the FAQ’s accounted for 9.5%. In the overall context the FAQ’s performed better than expected in the preceding years; with this small set of questions accounting for approximately 8% of all the successful queries.
As part of the pilot project, we also conducted a careful analysis of the existing free and open-source TTS available in Hindi. The original expectation was that such TTS may have an impact of acceptance and trust from farmers. While our team was satisfied with the TTS and was able to understand the prompts generated by the TTS, the vast majority of the farmers in the focus group were unable to understand the prompt. Despite several attempts to modify the output (playing with different speed settings, etc.), we were unable to make this option acceptable. This represents an important result that needs to be disseminated within the open source community. From an implementation perspective we narrowed down on a commercial product and were able to construct a TTS that was clearly accepted by the respondents. A key future activity is to disseminate our learning to the open source TTS community and lead efforts towards making available such offerings in the near future.
During the focus group experimentation and subsequently in the pilot study the team was constantly reminded, both by the implementation partner (OWSA) and the farmer community across the seventy villages, that while it is important to provide a question and answer service for this community, what is more important is to enable the community to access what they want, whenever they want without them being dependent on the traditional helplines structure.
As in the case of LifeLines, over a period of six years the model has matured and based on our analysis, reached a point where almost 85% of the content is in a reusable format. The FAQ’s cannot be added in a linear format anymore and a shift towards search is necessary to allow for a non-linear access to information.
Moving forward on these key takeaways from the VBAT project we have identified a specific roadmap towards unlocking the full potential of such helpline services that are not only critical for sustainability of such initiatives, but also equally important in meeting a constant demand of the underprivileged communities. Our endeavours will require further work specifically on:
- How to implement a search feature on a voice site (which has different requirements from the generic Web search that is offered commercially)
- How to build the appropriate speech resources, specifically speech recognition software, at low-cost and high quality, to enable this search feature for under-resourced languages
- How to design an audio user interface to enable users from underprivileged communities to use the search feature in an efficient way.
We will share the full report with you soon. Meanwhile you may be interested in the application developed by us for this project. The VBAT demo interface is connected to a voice / farmer interface and can be accessed and tested at http://public.webfoundation.org/vbat.
- US number: +1-646-475-3466
- Free Skype number: +990009369996102738
- SIP VoIP: sip:firstname.lastname@example.org
The software package (zip file) including both the Web interface and the Farmer voice interface is now available. Please refer to the configuration guide for details. OWSA has a dedicated media team and they captured the evolution of this project since 2010, in this short video.
The new pathways leading to the realization of the above are the next steps that we will undertake in the near future. In the end we wish to thank the entire team at OWSA, Dr. Naimur Rahman and Dr. Etienne Barnard for their excellent support and hard work throughout the project.