For long now there have been discussions around role of monitoring and evaluation towards an effective and efficient project implementation and more as donors, partners and communities have deepened their interest not just for progress and also the impact. The rationale is quite clear, every dollar spent should generate some return in terms of planned benefits or outcomes and to happen this, agencies have been more focused and definite in terms of time and resources dedicated for monitoring and evaluation of work done. Largely M&E is a considered and positioned as a key support role and one among key part of project life cycle. However, in case of IPLM (Indian Public Library Movement) it has come up as one among the four primary outcomes of the project. This may sound different position wise, has it actually been able to occupy the designated space ones gets to till the end of this article.
Beginning with baseline as has been accomplished across all partners from UP, J&K, Kerala, Telangana, Tamil Nadu in about fifty-four selected libraries, about 2800 user and non-user interviews were conducted. Beyond user perspective, understanding the need of data for initiation of new services vis-à-vis the capacity and infrastructure gaps, needs assessment was made part of baseline survey – complimenting each other. To make the data collection quick yet efficient (always simple to use) initially it was thought using a mobile application would be effective and certainly yes and we get to learn about pop-up survey tool – innovation from global libraries programme in Romania. This innovation may look simple, however architecture is as complex as any other innovative web based solutions. Before adopting it, seven issue areas of common impact measurement system and their measurement indicators were discussed and found not wholly fitting to local context. Therefore, we chose to have few of issues areas given in CIMS and others were selected by librarians considering the need, relevance and impact. While in Romania, use of public access computers didn’t require any checks (like asking for a photo ID etc.), we plan to do it in all participating libraries to put in place some checks and balances and there always remains risk of uploading any objectionable/controversial content (photo, video, post etc.). Moving from use of pop-up survey tool to typical M&E practices, so far we have been successful to complete and baseline survey, have selected M&E focal points representing all partners and have been trained for monthly data collection for libraries using the library checklist. Before the checklist is fully rolled-out, pre-test was done in random libraries across states to find what works as part of variables in terms of data sources, challenges to collect data and more importantly the relevance of reflecting back on performance of libraries (there could be a case where data collection becomes redundant and irrelevant given the desired performance or outcomes). To make the data collection quick and efficient, pop-up survey tool we be a great tool to use – data templates are available on web portal, focal points need to login, choose the library they collect data for and submit. This method is also true for trainings and other activities and largely eliminating the scope for paper form data. The purpose is simple, data is collected quickly (helps to quickly analyze and share), better is its relevance for performance measurement and decision making. As part of results framework, data sharing has been considered equally important, having nice charts and well written reports sound more meaningful when partners and librarians understand it’s relevance and start planning and taking decisions using the evidence. To do this, dashboard for dissemination data (output and outcomes), is shared with team, partner staff, other stakeholders to have an easy access to polished version of data – beyond narrative reports. The dashboard will include all major indicators (output and outcome) to reflect trends and pattern of various data sets.
One of the key considerations of M&E roll out for IPLM is involving librarians to understand use of pop-up survey tool and library performance measurement, building and sharing stories about libraries. Introducing pop-up survey tool with defined indicators and training for librarians would be a way forward to sustain the performance measurement in libraries. To summarize, M&E focal points (representing partners) are trained to collect data for most of indicators using library checklist, pre and post training evaluation data, feedback from community awareness and knowledge building events and librarians would be trained to keep track of pop-up survey tool in libraries, performance data will be shared to understand outcomes achieved and supported to use all data for library development plan. This doesn’t end here, these will also be trained in skills of storytelling to be able to share the stories of change with related stakeholders.