Development of an interactive questionnaire for the improvement of child protection measures in sports

Summary

Laureus Sport for Good and CorrelAid have developed a test to determine the level of child protection in (e.g. sports) organizations, which previously only existed in static PDF form. As part of this project, the test was published as a Web App, which presents the respective results for each organization in a visually tangible way, provides matching recommendations for child protection measures, and allows Laureus to store and evaluate the data. The app is available in four languages (German, English, Spanish, French).

The Challenge

Laureus Sport For Good Germany

Laureus Sport for Good supports more than 250 organizations in over 50 countries worldwide that use sport and physical activity to combat violence, discrimination and disadvantage among children and young people. Laureus Sport for Good Germany was founded in 2001 as the first national Laureus Foundation and uses the power of sport with the aim of supporting disadvantaged children and young people in Germany and Austria in their personal development and thereby making a positive impact on society. Since 2009, Austria has also been part of the foundation’s field of action. To date, the German and Austrian Laureus support programs have provided help, advice and encouragement to more than 50,000 children and young people from socially deprived areas. By signing the International Safeguards for Children in Sport, Laureus Sport for Good has committed itself to upholding, promoting and protecting them.

Website

There are significant deficiencies, lack of knowledge and much lack of will to introduce and implement effective concepts with regard to child protection measures in the sports sector in Germany and Austria, but also internationally. Laureus Sport for Good is a signatory and part of the Safe in Sport initiative, which advocates for stronger protection concepts in sport worldwide, offers guidelines and recommendations and supports the development of protection concepts.

A previously existing self-test in the form of a PDF document, in which points had to be added up manually and the results searched for, was not user friendly and as a result was little used to review and further develop child protection measures.

In addition, because the PDF was completed and scored manually by organizations offline, there was no way to target support to the organization based on the results. Finally, due to a lack of data, it was not possible to conduct analyses that could, for example, show differences between organizational forms or countries.

However, these findings are important on two levels: first, for the organization itself, to get a simple and quick assessment of the strength of its existing processes and concepts, and to get direct recommendations for improvement. On the other hand, for Laureus to develop offers to further support the organizations in the field and to further bring the topic of prevention of (sexual) violence/child protection to the public and to point out the gaps.

The Data

The data to be collected in the test in PDF form is a self-report by an organization on various child protection issues in that organization. These questions are divided into eight question blocks, each covering eight different areas of protective measures. The partial aspects of the protective measures queried per question are to be answered in each case with “not yet fulfilled”, “partially fulfilled” or “fulfilled”. Prior to the start of the CorrelAid project, the answers were neither systematically stored nor further analyzed, as there was no way to access the participants’ answers, especially not in anonymized form. In addition, there was no coordinated (and anonymized) way to provide feedback to organizations in the form of recommendations on individual protection measures after they had completed the questionnaire.

The now interactive web app, in addition to simplifying the use of the test and providing automated recommendations based on responses, also allows for the (anonymous) collection of data. This involves anonymously storing data on the type of organization, the size of the organization, and the age of the participants in a Google spreadsheet, as well as the coded responses to the individual self-assessment questions.

The completing organization receives a downloadable PDF of their responses. By downloading, the organization agrees that the generated data will be stored at Laureus. It is also possible to use the tool online without downloading/forwarding the data.

The generated data provides an overview of how far organizations are in their development of child protection measures and in which areas there is a need for improvement. From this, Laureus Sport for Good can, on the one hand, derive which offers the foundation needs to develop in order to further support organizations in the area and, on the other hand, further publicize the topic of prevention of (sexual) violence/child protection and identify the gaps.

The Solution

Laureus and the CorrelAid project team jointly developed the concept of an interactive and reactive Web App to replace the old PDF questionnaire and provide additional functionalities. Based on the existing Data-Maturity Evaluation Tool Elmo developed by CorrelAid, the project team then developed a concept of a Web App with these functionalities with ongoing content input from Laureus and wrote it mainly using the R Shiny Package in the R programming language.

The web app, like the PDF questionnaire before it, captures responses to child protection issues in organizations, bundled into eight protection measures and each with three response categories of “not yet met,” “partially met,” and “met.” These three response categories are listed at the beginning of the questionnaire in a legend with a traffic light system, which allows a fluid and intuitive response to the questions. The way the legend is visually represented also contributes to the accessibility of the questionnaire, as the individual answer categories in the legend are also recognizable for people with, for example, red-green impairment. Thanks to the implementation in the web app, the results of individual answers can now be viewed constantly in parallel while answering, including continuously updated result statistics of the protective measure areas, such as mean values and medians, as well as a continuously updating reactive result graphic.

Going beyond the questions previously available, the Laureus web app now offers the ability to ask general questions about organizational data for inclusion in potential analyses. By using general wording and avoiding free-text fields, the anonymity of respondents is always preserved. In addition, the web app does not store any technical data such as the IP address or browser configuration of the respondents at any time, so that conclusions about their identity are also ruled out in this way.

In contrast to the old questionnaire concept, two things now happen immediately when an online questionnaire is completed in the web app: On the one hand, respondents receive immediate feedback in the form of a downloadable PDF that provides recommendations on individual protective measures based on the answers they have just given. On the other hand, responses on organizational data and safeguards are automatically saved in parallel in a private Google Sheets file accessible only to Laureus. Potential GDPR-related problems regarding the storage of personality-related data by means of a Google service with server locations outside the EU do not exist, as no personality-related data of the respondents was collected in the first place. The biggest advantage of this web app designed by CorrelAid for Laureus compared to the previous offline questionnaire is therefore that all answers are stored in digital form and are available for further evaluation and analysis without much effort.

The Impact

The interactive Web App offers users the improvement over the static PDF document of receiving the results directly evaluated and visually appealing. For users, this means low-threshold access and significant time savings. The ease of use also increases the reach of the test. From Laureus’ point of view, the possibility to centrally evaluate the test results is advantageous. It is not yet possible to estimate the gain in knowledge, as the app first has to be used for some time to generate the required data.

To facilitate Laureus’ sustainable code-side usability of the app, the code was commented and certain critical aspects (renv package, local setup of the app, deployment on shinyapps.io) were explained in the GitHub ReadMe. In addition, the code was explained in detail in an explanation video.