Evaluating Implementation Fidelity of “ Mother Supportive Group ” Intervention to Improve Childhood Nutrition in Monaragala District of Sri Lanka

A higher level of implementation fidelity of nutrition interventions is required to make a significant impact on nutrition promotion especially among children of age under five years. Mother Supportive Group is a community empowerment programme of the Ministry of Health, which has been operating in Sri Lanka since 2012 for promotion of nutrition among under five years in the community. The objective of this study was to assess the implementation fidelity of Mother Supportive Group intervention in a selected community. A series of key informant interviews were conducted with nine middle-level healthcare managers and six focus group discussions were conducted with programme supervisors and members of the Mother Supportive Groups. Implementation fidelity of this intervention was assessed giving emphasis to adherence, dose & exposure, quality of delivery, participants’ responsiveness and programme environment. According to the results, adherence to key components of the implementation fidelity of this intervention in Monaragala district was unsatisfactory. Activities within Mother Supportive Groups have been delivered to a lesser extent than expected and the quality of delivered components was also found to be poor. Acceptance of Mother Supportive Group intervention by its primary target population was also unsatisfactory because of various programme issues and unconducive environment existed in the community for its implementation. As reported in number of other studies, relationship between implementation fidelity and outcome of the intervention has a positive correlation. In this study it was clear that, the implementation fidelity of Mother Supportive Group intervention was low and it has led to less effective coverage of primary target population, giving rise to no tangible impact on the nutrition of children in the study group. Therefore, measures need to be taken to increase the implementation fidelity of Mother Supportive Group intervention to achieve the expected level of nutrition and growth promotion.


INTRODUCTION
Nutritional requirements vary with pregnancy, infancy and childhood, adolescence and adulthood, old age, disease, convalescence and during disasters. Nutritional status during first 1,000 days of a child and its consequences on his/her adult life have been well established (Ruel et al., 2013). For an example, being small at birth and at two years of age (specially the linear growth) is strongly associated with short adult height, less schooling, less economic productivity and for women, low offspring birthweight (Richter et al., 2012). In 2011, it was estimated that 26%, 16% and 8% of under-five year children in the world were stunted, underweight and wasted, respectively (Black et al., 2013), while 26% of world's stunted children and 70% of children with wasting live in Asia (UNICEF, WHO and The World Bank, 2012). In Sri Lanka, prevalence of stunting, underweight and wasting were 17.3%, 15.1% and 20.5%, respectively while comparable figures for Monaragala district were 15.9%, 25.4% and 24.2% (DHS, 2016). Annual deaths of over 10 million, under-five year children worldwide are due to preventable and treatable illnesses despite effective health interventions. At least half of these deaths are caused by undernutrition (Hien and Kam, 2008). It was further estimated that 2.8 million child deaths (28% of under-five deaths) and 114 million Disability Adjusted Life Years (DALYs)  have been attributable to childhood undernutrition while it constituted to 8.5% of total global disease burden (Black et al., 2008).
In Sri Lanka, supplementary and therapeutic foods are distributed among pregnant mothers, lactating mothers and children as public health interventions targeting nutrition as well as growth of children younger than five years of age. Multiple micronutrient supplementation for underfive children as well as for school children is also being implemented targeting Iron Deficiency Anaemia (IDA) while fortification of salt with iodine is in place for combating iodine deficiency. Capacity building on Infant and Young Child Feeding (IYCF) practices and nutrition knowledge for healthcare staff was also another intervention implemented by the Ministry of Health in Sri Lanka to strengthen under-five year nutrition promotion programmes in the country. Growth Monitoring and Promotion (GMP) programme is a well-established public health intervention in the community where weight and height of under-five year children are being regularly monitored and nutrition counselling is done for those who are identified to have growth related issues. Community mobilization programmes such as 'Health Volunteers' and 'Mother Supportive Groups' were also implemented in primary healthcare settings of the country, mainly for strengthening the ongoing public health interventions in the community (FHB, 2019). Mother Supportive Group (MSG) is one of such community empowerment programme which has been operating in Sri Lanka since 2012 (HPB, 2018). The main objective of MSG intervention was to promote nutrition among under-five children by establishing healthy behaviours among parents and children. This is done by providing opportunities for the MSG members to share their experiences on best practices of managing under five-year child nutrition through effective interactions.
Post-evaluation of any intervention is vital to determine its effectiveness in achieving expected results. Post-evaluation in terms of the degree to which the MSG intervention was implemented as intended, has not been done since its implementation in 2012 except for a formative review of MSGs in Northern and Eastern provinces of Sri Lanka (Tillekeratne et al., 2015). The objective of this study was to assess whether the MSGs have been effective in their core purposes for which they were set up, based on the formative review of process parameters of MSG intervention but not the outcome parameters. Tillekeratne et al. (2015) reported that nutrition and health messages delivered in MSG intervention were not specific to improve nutrition related best practices and lack in focus. Food demonstration sessions carried out for MSGs were similar to those in undernutrition rehabilitation programmes of World Vision Lanka (a non-government organization), conducted targeting meal preparations. But according to the objectives of MSG, it should improve the skill of the mother to prepare a child's meal with an appropriate nutritional composition.
Furthermore, training opportunities were not provided to all MSG members but mainly to office bearers of a MSG. Further, training sessions carried out lacked in depth of knowledge to provide required information on nutrition. Also, these were not conducted uniformly among MSGs resulting in unawareness of some MSG members of the purpose of certain messages and activities carried out by Public Health Midwives (PHMM). It was further revealed that the mandate of MSG intervention was very much towards service provision but not about the analysis of the problem to be addressed, participatory development and empowerment.
Establishing 'MSGs' as a community empowerment intervention aimed at underfive nutrition promotion was given higher priority during the last 5 -6 years in Sri Lanka among other public health interventions understanding its importance. However, results of this intervention related to behaviour of mothers and improvement of nutritional status of under five were hard to find after 5 -6 years of its implementation due to poor implementation fidelity. For example, Gamagedara et al. (2020) reported that the effective coverage of MSG intervention to be 2.6% in the Monaragala district. This study further revealed that, most of the expected behavioural changes among caregivers of under-five children have not taken place even after 5 -6 years of implementing 'MSG' intervention. However, neither Tillekeratne et al. (2015) nor Gamagedara et al. (2020) have looked into the deeper aspects of implementing MSG intervention which are important, in a postevaluation, to determine the underlying causes of the intervention hindering its intended results. Similar studies have reported that, in conducting such analysis, qualitative methods of inquiry are widely used in combination with quantitative methods to enrich the depth, scope and breadth of research (Sumathipala and Siribaddana, 2003). Encompassing qualitative methods also allow researchers to take a naturalistic approach to capture experiences of participants in its 'natural' form rather than evaluating them based on pre-determined protocols or plans. Therefore, it enables unanticipated variations and important idiosyncrasies of intervention programmes implemented in the community to be highlighted in a naturalistic inquiry unlike by a checklist or pre-determined responses in a questionnaire (Jayasuriya, 2012).
Implementation fidelity is widely used in qualitative evaluation of nutrition interventions and it is defined as the extent/degree to which elements of an intervention are implemented as designed and planned (Saunders et al., 2015;Carrol et al., 2017). Implementation fidelity is also described as a potential moderator between an intervention and its intended outcomes. There is a certain overlap among terminology being used in describing implementation fidelity (Carroll et al., 2007). However, adherence to intervention, exposure/dose, quality of delivery, participant responsiveness and programme environment have frequently been identified as most appropriate elements of evaluation of implementation fidelity of nutrition/health promotion programmes and their outcomes (Saunders et al., 2005). Therefore, this study was carried out to evaluate the implementation fidelity of MSG programme in Monaragala district. The findings of this qualitative exploration enrich the findings of the quantitative assessment of MSG by Gamagedara et al. (2020).

Study context
A community level in-depth explorative study to assess implementation fidelity of MSG intervention was carried out from November 2018 to February 2019 among Public Health Staff and MSG members in the Monaragala district. Adherence to intervention design, dosage and exposure, quality of delivery, participant responsiveness and programme environment were the common elements of implementation fidelity that were assessed in this study to decide on the success of the intervention.

Sampling and data collection
Key Informant Interviews (KII) were conducted to collect information from middle level healthcare managers of the health sector that includes the Medical Officer of Maternal and Child Health (MO-MCH), Medical Officers of Health (MOOH), Regional Supervising Public Health Nursing Officer (RSPHNO) and Health Education Officers (HEOO). All key informant Interviewees were purposively selected to represent different levels of monitoring, supervision and coordination of MSG intervention in the community. Focus Group Discussions (FGD) were carried out to collect information from Public Health Midwives, Supervising Public Health Midwives (SPHMM), Public Health Nursing Sisters (PHNNS) and members of MSG. All PHNNS and SPHMM who were responsible for supervising public health interventions such as MSGs and guiding PHMM in their respective MOOH areas were included in FGDs as there were only 10 PHNNS and 11 SPHMM in the district at the time of this study. PHMM were also purposively selected to represent different level of implementation of the intervention within and across MOH areas. Members of MSG were randomly selected from different MSGs to capture maximum variation in implementing MSGs at family level. Altogether nine KIIs (MO-MCH -1, Ex.MO-MCH -1, RSPHNO -1, HEO -1, MOOH -5) and six FGDs (PHNSS -1, SPHMM -1, PHMM -2, members of MSG -2) were conducted.
KII and FGD guides were prepared and they were pre-tested and validated by identifying a sample population from Badulla district to ensure the study population knew no information about the survey prior to its commencement. In here the funnel design was followed that begin with broader topics and end up with narrow topics in each item in conducting KII and FGDs. FGDs were conducted separately among PHNSS, SPHMM, PHMM and members of MSGs at the auditorium of Regional Directorate of Health Services office, Monaragala. Lighting, noise, space and availability of necessary furniture were considered before conducting each FGD and KII. The venue used for conducting FGDs was selected outside of participant's working station to provide them with an environment to actively participate in FGDs avoiding their reluctance in providing information in the presence of their supervisors. On the other hand, all KIIs were conducted in participant's respective working stations on a convenient date and time for each participant. All FGDs and KIIs were conducted by the principal investigator himself with two assistants in local Sinhala language and recorded on a tape with the consent of study participants. The KIIs and FGDs were audio recorded and tape recordings were transcribed verbatim manually for converting audio to text. Later these texts were translated into English following a standard procedure (Sumathipala and Murrey, 2000). To describe key themes, rich text quotes were extracted along with de-identified participant information.

Data analysis
Framework approach was used in data analysis of this study while concept driven coding was used for coding. The analysis of qualitative data was performed manually through the process of sequencing, capturing and coding continuously as the data were being collected. Similar responses were taken together and any response unique to one particular group was taken separately. Information that revealed identity of participants was deleted and removed from the analysis.

RESULTS AND DISCUSSION
This study was conducted to assess the implementation fidelity of MSG intervention in an identified district to recognize the success of this intervention. Monaragala district was selected for this study considering the feasibility and it was also a district having an average degree of implementation of MSG intervention among the districts in Sri Lanka.
Further, Monaragala is the second largest district in Sri Lanka with 485,812 of its total mid-year population (Ministry of Health, 2012), 11 Medical Officer of Health (MOOH) divisions and 206 Public Health Midwives (PHMM) areas. The prevalence of stunting, wasting and underweight among under-five children in the district of Monaragala as well as in Sri Lanka show little progress over last few decades irrespective of number of interventions being implemented to improve childhood nutrition. Findings from both KII and FGDs pertaining to different parameters described below suggest that the implementation fidelity of MSG in Monaragala district has severe drawbacks.

Adherence
The extent to which implementers adhered to elements of MSG intervention in Monaragala district at each level of its implementation as outlined by designers was assessed under this parameter. Four important categories identified in this study under 'adherence' were, conducting awareness programme, training public health staff, monitoring and evaluation, and establishing MSGs in the community.

Awareness for middle level healthcare managers on MSG
According to the information gathered, district as well as divisional middle level healthcare managers have hardly received formal awareness on MSG intervention before its implementation in the community. Instead, they have received instructions in the form of a letter at an early stage of MSG intervention. Monitoring and supervising officers at district level revealed that they were not formally made aware of the intervention at national level except one or two lectures on MSG delivered during their annual or bi-annual reviews. Ministry of health and nutrition with its national focal points designed MSG intervention and implemented it island-wide in 2012. Conducting an awareness programme of the intervention adequately for those who are responsible for its implementation, monitoring and evaluation is a key requirement for effective implementation of an intervention to improve health issues concerned. As MSG intervention was integrated into ongoing Maternal and Child Health (MCH) programme, having an inbuilt mechanism for healthcare providers to be aware of MSG intervention throughout its implementation was very vital. This could have been achieved without much effort and resources but through a formal training component in annual MCH training calendar of the district. Ensuring guidelines on MSG freely available at each level of its implementation was also vital to keep public health staff aware of the intervention. It was hard to find such awareness programmes being conducted for relevant public health staff at least before implementing MSG intervention or subsequently for those who assumed duties later. A handbook of guidelines on MSG has been distributed among Public Health Staff in 2015 even though a substantial number of public health staff did not know of its availability, and majority of those who were aware of its availability were not aware of its content.
Relevant parts of the transcribed verbatim from randomly selected respondents of this study clarify the lapses of MSG intervention in terms of providing awareness for middle level healthcare managers.

Respondent (KII1):
We were asked [through a letter]

Training public health staff at divisional level
According to the information gathered, no arrangements were made for public health staff to be formally trained either before implementing MSG intervention or during subsequent years. Divisional level health managers (MOOH) and their staff have got to know about MSG intervention through letter instructions as well as from Health Education Offices during their monthly conference. There were also a substantial number of public health staff who became aware of MSG intervention only through their immediate supervisors, hand book of guidelines or annual reviews. Before beginning of an intervention, it is important to train officers who are responsible for implementing and supervising such intervention. Having an effective training programme to train those who are responsible before implementing an intervention as well as for those who assume duties subsequently will determine effectiveness of the intervention concerned. In implementing MSG intervention, all public health staff (PHNNS, SPHMM, PHMM and PHII) at divisional level were required to be trained before its commencement as they were responsible for implementing and supervising public health interventions of this nature in their respective field areas. Including this training component in annual training calendar of the district would help to train existing public health staff as well as those who assume duties subsequently.

Monitoring and supervision
There were formats developed by national focal points for reporting progress of the intervention monthly by each MSG to PHM, from PHM to respective MOH office and for keeping records on progress of MSGs at each MOH office. However, it was hard to find field health staff or leaders of MSGs reporting progress of MSG intervention regularly using those formats. On the other hand, there were annual reviews being conducted regionally as well as nationally in which the progress were assessed subjectively through narratives presented by members of MSGs and , sometimes, public health midwives. Outcome indicators were hardly used in determining effectiveness of MSGs in such reviews. However, monitoring and supervision is another essential component in implementing any intervention as far as achieving its intended results are concerned and need to give priority.
Relevant parts of the transcribed verbatim from randomly selected respondents of this study clarify the lapses of MSG intervention in terms of monitoring and supervision.

Respondent (KII1): I am not satisfied at all with monitoring and supervision of this intervention. Officers outside made visits to the district without our knowledge even though we work as the focal point officers in the district. We have not even received a report of such visits.
Facilitator: Are you supposed to send a progress report on MSG intervention to your national programme?

Respondent (KII3): I am not happy at all about the way that the annual reviews are being conducted because I believe that you cannot actually find those who do well with MSG intervention by listening to their stories. How can you confirm these stories? It is just like a stage drama. Those who can act well usually win, but they may not be so good at MSG intervention and vice versa. Therefore, I propose to change this reviewing system if possible [A Medical Officer of Health].
The progress of MSG intervention was also identified to take up as an agenda item at the monthly conference of MOH office according to its guidelines. However, it was revealed that such discussions on MSGs were made at monthly conferences only when annual reviews were called. It was observed that due priority was not received for MSG intervention in annual supervision plan of the MCH programme nationally, regionally or divisionally. It was further revealed that coordination among relevant units at district office as well as between district office and MOH office was not satisfactory as far as the effective implementation of MSG intervention is concerned. In certain MOH areas, monitoring and supervision of MSG intervention was not made through PHNSs and SPHMM but it was done through other non-technical officers. Public Health Inspector was also identified as a key officer, in addition to PHM, in establishing and functioning MSGs. However, contribution received from PHI to implement MSG intervention was very minimal. A uniform method of monitoring and supervision for MSG intervention did not exist in the community which may have resulted great difficulties for programme managers to measure its progress.
Relevant parts of the transcribed verbatim from randomly selected respondents of this study clarify the lapses of MSG intervention in terms of monitoring and supervision.

Establishment of MSGs in the community
PHMM were expected to create at least one MSG (5 -20 members) per PHM area at the beginning of MSG intervention and, thereafter, to expand it further. However, only a few PHMM complied with these guidelines and established MSGs accordingly. In certain MOH areas, some of the PHMM had none of the MSG functioning in her field. When the activities within an MSG are considered, field heath officers (PHMM and PHII mainly) are expected to train at least one member from each MSG, at a time, on healthy behaviours. These trained members are then expected to share their knowledge with members of her/his MSG during monthly meetings in forms of discussions and activities related to healthy behaviours on which they were trained. However, it was hard to find a regular mechanism being established to train members of MSGs at least in PHMM areas where there were functioning MSGs even though PHMM delivered health talks to all members of a MSG during its monthly meeting. Therefore, adherence to essential components of MSG intervention was found to be inadequate in order to achieve its intended results.
Relevant parts of the transcribed verbatim from randomly selected respondents of this study clarify the lapses of MSG intervention in terms of establishing MSGs in the community.

Dosage and exposure
Extent to which expected components of the intervention received by the intended participants was assessed under 'Exposure/Dose'. Initially it was expected to establish and function at least one MSG per PHM area. However, only a few PHM areas of a given MOH division were having at least one functioning MSG. Not only PHM areas but several MOH areas also did not have a single functioning MSG established at the time of this study. In monthly meetings of established MSGs, even though it was expected to receive facilitation from Public Health Staff, only PHMM attended to facilitate discussions and activities carried out during such monthly meetings. PHII and MOOH attended rarely to such MSG meetings in their respective field areas. As the name implies, MSG is not a group of females but it is in fact supposed to be a group of supporters including both males and females. In this context, male participation was also expected to be the same as that of females in MSG activities. In agreement with the information reported by Tillekeratne et al. (2015) in their study by reviewing MSGs in Northern and Eastern provinces of Sri Lanka, this study also revealed the poor male participation in MSG intervention. Furthermore, active participation of mothers of undernourished children were found to be poor even though they were the most targeted group for this intervention. Therefore, these revealed majority of primary target population have not received essential components of MSG intervention as it was expected.
Relevant parts of the transcribed verbatim from randomly selected respondents of this study clarify the lapses of MSG intervention in terms of dosage and exposure.

Quality of delivery of the Intervention
How components of an intervention are delivered in a way appropriate to achieving what was intended from the intervention was assessed under 'Quality of delivery'. In certain MOH areas, MSGs were not established to the intended extent while in certain MOH areas structure and composition of established MSGs were different from what was recommended. In two MOH areas, it was considered only parents/guardians of children who were undernourished as primary target population for MSGs. In other MOH areas, small MSGs (5 -6 members) were formed at village level and a larger MSG was then created per PHM area having amalgamated those smaller MSGs formed at village level. Finally, a common MSG per MOH area was formed having included office bearers of all MSGs in that particular MOH area. These are considered as deviations from what was recommended in guidelines. In majority of functioning MSGs, participation rate of mothers when their children become two or more than two years of age was less in comparison to parents of children less than two years of age. The participation of mothers became very less when their children reach five or more than five years of age. Willingness as well as degree of participation among mothers with an undernourished child in MSG activities was poor compared to mothers of wellnourished children. It was often the mother of an under-five year child who participated in discussions and activities of MSGs compared to father, grandfather or male guardian even though active male participation is also equally important in nutrition promotion among under-five children. In nutrition promotion of an underfive child, it is expected to discuss and share experiences on nutrition promotion among members of the MSG. However, members of MSG paid more attention to prepare a meal with locally available food and distribute among children who are attending weighing session while giving less priority to discuss and share experiences on healthy dietary practices which help to keep the child wellnourished. Furthermore, monthly meeting of an MSG is expected to be held in a convenient venue, date and time for members of the MSG. However, in certain PHMM areas, MSGs were established by registering all the attendees who came to their weighing centres. Also monthly meetings of such MSGs were held on the same day, one to one and half hours before starting a weighing session. However, it was found that, in such situations, the primary purpose of the attendees was not to actively participate in MSG activities and discussions being conducted but to get their children weighed. All these highlight the fact that expected components of MSG intervention have not been delivered in the intended manner and giving rise to failure in achieving intended results of the intervention during the past 5 -6 years.
Relevant parts of the transcribed verbatim from randomly selected respondents of this study clarify the lapses of MSG intervention in terms of quality of delivery of the intervention.

Participants' responsiveness
The extent to which primary target population and public health staff accepted MSG intervention and responded accordingly were assessed under 'Participants Responsiveness'. Pregnant mothers, mothers and fathers of children of under-five years old are the primary target population for MSG intervention as far as nutrition promotion among under-five children is concerned. Due to various programme issues such as inadequate awareness being received to public health staff on MSG intervention, poor engagement of public health staff in required activities of MSG intervention resulted in poor acceptability of MSG intervention among primary target population. For example, there was a high variation in establishing and functioning MSGs among MOOH and PHMM areas in a given MOH area as described previously. This has resulted poor coverage of the target population by MSG intervention contributing to failure in achieving its intended results.
Relevant parts of the transcribed verbatim from randomly selected respondents of this study clarify the lapses of MSG intervention in terms of participants' responsiveness.

Programme environment
Different aspects of the environment where the MSG intervention was implemented in terms of its influence on its implementation and its outcomes are described under 'programme environment'. This study setting was mostly a rural in nature. Fathers/guardians of under-five children in this community were often involved in generating their family income from agriculture based or daily paid occupations. Mothers of such families were occupied with other family matters and sometimes involved in generating family income too. Together with programme issues, this may have led to poor active participation of parents especially males in community empowerment programmes. It was further observed that active participation of mothers in MSGs was less when the child become more than three years of age. Mothers revealed that they have to accompany their children to and from preschool or school and tuition classes. Furthermore, there was a poor support from other family members and from those of immediate surrounding, leading to an unfavourable environment for primary target population to sustain their active participation in MSG intervention.
Relationship between implementation fidelity and outcome of an intervention has been very clearly shown in number of studies. During evaluation of parenting training programmes, it was observed that parenting practices were significantly improved when it was implemented with high fidelity and vice versa (Forgatch et al., 2005). Similar relationship between outcome of an intervention and fidelity was also described by Thomas et al. (2007). Implementation fidelity of MSG intervention in Monaragala district was found to be very low according to the findings of this study. This has led to low effective coverage of primary target population by MSG intervention giving rise to no tangible impact on nutrition in the community.

CONCLUSION
Implementation fidelity of MSG intervention in Monaragala district was assessed based on the accomplishment of adherence to MSG intervention, dosage and exposure of intervention components, quality of required components being delivered, participant's responsiveness and programme environment was not satisfactory. Accordingly, the intervention was not able to achieve minimum required coverage of its primary target population to show intended results of MSG intervention, even 5 -6 years after its implementation in this district. Unless these identified gaps are addressed adequately, it is highly unlikely that the expected impact of this intervention at community level would be achieved at a scale. Therefore, it is recommended to improve fidelity components of MSG intervention and to make necessary adjustments to rectify its poor implementation.

Public health implications
Scientific evaluation of outcome and impact of public health interventions are important to understand the success of the implementation process, the conditions and factors influencing the process, and sheds light on how the program need to be improved to achieve its intended impact. This study revealed that poor implementation fidelity of MSG intervention has contributed to the failure in achieving nutrition and growth promotion of children under-five years as intended. Undernutrition among children of age under-five years is still a public health problem in Sri Lanka. Hence, implementing community based public health interventions with high implementation fidelity is required to further improve their nutritional status.

ACKNOWLEDGEMENT
Authors acknowledge Uva Provincial Ministry of Health for granting approval to implement this study and, Provincial as well as Regional Health Service and Medical Officers of Health with their staff for immense support extended towards successful completion of this research project. Provincial Department of Health Services is acknowledged for providing funds for this study.

Ethics approval and consent to participate
Ethical clearance for this study was granted by Ethics Review Committee of the faculty of Medicine, University of Peradeniya (ERC No: 2017/EC/87). Provincial Ministry of Health provided administrative clearance for this study. Informed consent was taken from study participants before data collection.