Depending on the size of your youth-serving organization, the data you’ll need to collect and analyze—or even simply summarize—could be considerable. That’s why, no matter what your organization’s size, in the early stages of building your child safety framework it’s important to think about the kinds of questions that you’ll need to answer periodically, and the data that you’ll need to collect to answer those questions. 

How to Collect and Use Data in Self-Audit

Simple checklists with “yes” or “no” (quantitative) responses can tally the numbers of employees and volunteers who have attended the required training, signed the code of conduct, and complied with other requirements set by the leadership. This allows you to gather basic information about the implementation of the safety program, and then use those numbers to provide feedback to both leadership and staff about progress, and the level of compliance with the new program requirements.

You can also use these numbers to keep parents and others informed about what your organization does to ensure their child’s safety before enrolling them. Basic data collection and a summary of results allow you to say to parents, “We are committed to child safety. We have a set of published safety policies and procedures, and all our staff and volunteers are screened. All members have a signed code of conduct on file, and have been trained in child/youth safety, and in their abuse reporting responsibilities under Massachusetts law.”  Statements like these on your brochures, materials, and website can help to put parents at ease (See Child Sexual Abuse (CSA) Prevention Evaluation Tool for Organizations).

Changes in staff practices, attitudes, and behaviors (qualitative) are more complicated to measure, and rely more on ongoing observation and monitoring. A good rule to remember is that all improvement requires change, but not all changes lead to improvement. Unless you have an ongoing commitment to gauging staff attitudes about the desired changes, and gathering information about the behaviors that indicate whether those attitudes are being translated into the desired actions, you may be tempted to stop measurement altogether when the policies are published and everyone has been trained. This does little to determine if the changes have been adopted into the culture of the YSO.

Some basic introductory and advanced tools, available via the Child Welfare Information Gateway, can help you measure and evaluate your child abuse prevention programs. An annual data collection process or “self-audit” can help you understand (and explain) the basics of how the implementation is progressing. More comprehensive data collection can also enable you to measure organizational, cultural, and behavioral change, and whether what is being implemented supports your desired outcome: to make your environment safer.

A simpler variation of the Child Sexual Abuse Prevention Evaluation Tool can be found here.

Some organizations may only collect information about program implementation, and basic information about the number and types of “internal” allegations encountered during the year (i.e., the number of reports filed involving staff, volunteers, or other children/youth) and how they were handled.  Others may need to take a broader look to answer questions about the effectiveness of their programs from a wider variety of stakeholders like funders, insurers, or boards of trustees (See Self-Audit Form for YSOs).

For example, a broader perspective would consider the possibility that your efforts to increase awareness about child sexual abuse prevention may also create a greater sensitivity to situations that may be happening outside of your organization. By educating adults (and children/youth if appropriate) about safe environments, personal safety, appropriate and inappropriate touch, and how to recognize and report dangerous situations, reports of suspected abuse and neglect in the home, extended family, neighborhood, dating relationships, and other outside situations may also come to your attention.

To measure the effectiveness of what you’ve instituted, your initial assessments could focus on gathering data on the numbers and types of reports being filed; how (and through whom) they came to the attention of the organization (observed, suspected, or self-disclosed); how quickly they were reported to the your reporting authority and/or to DCF; whether those to whom the abuse was reported knew what to do; and any information about outcomes of cases. The data can be collected without using the names or other identifying information, but should be treated as confidential and kept in a secure location. Asking respondents to provide examples of observed “safe” behaviors among children and youth, or increased awareness of the safety rules, would underscore their response with additional anecdotal information (See Sample Child Safety Incident After-Action Report)

One could argue that by collecting data of this type on an annual basis, an organization like yours could influence behavior by promoting ongoing awareness of the numbers and types of abuse and neglect cases your children/youth are experiencing. It could also indicate whether children/youth are learning the skills outlined by your framework, whether they’ve used those skills to report to a trusted adult within your community, and whether the adult did what was expected, within the required timeframes, as trained. Again, the focus is on the behaviors and skills that we would expect to be outcomes of the safe environment efforts. Certainly, there are more sophisticated analytic and research-oriented methods for determining program effectiveness, but the data we’ve described can be collected with minimal effort. More complex program effectiveness analyses would require appropriate expertise and guidance.

Periodic Policy Updates

In addition to an annual self-audit, we strongly recommend that you evaluate your policies and procedures periodically—at least every 2-3 years. This comprehensive analysis is primarily qualitative. It’s designed to interview individuals and groups who have carried out the various responsibilities described in your policies, and get their feedback about how what is written—and intended—is actually working in practice. The product of this analysis is a document that assesses each functional area of your policies and makes recommendations for changes and updates to the policies and procedures that:

• Reflect the way a policy area or procedure has evolved in its practice that is contrary to what is written, but is working successfully
• Restructure the text to address weaknesses or failures in policy and/or in a procedural area, and make them more effective or efficient
• Strengthen and/or clarify policy sections or procedures
• Develop policies and procedures for new issues that emerge from the analysis

This way, you can identify your policy’s strengths and weaknesses, uncover issues that were not anticipated or addressed adequately, and change/update them accordingly. Once updated, the areas of recommended improvement can be followed and observed for a period of time (such as six months) to determine their effect—and associated quality improvement—on your overall policy implementation.

To sum up our measurement recommendations to sustain your child protection framework over time, your data collection should include:

• An annual self-audit to assess the current state of your implementation, including a checklist of the data to collect
• A comprehensive analysis of your organization’s policies and procedures every 2-3 years, to identify what’s working in practice, what needs improvement, or what needs to be added to the policies and procedures in order to strengthen them
• A “continuous quality improvement” initiative that takes a subset of any identified policy or practice improvements, addresses them, follows their implementation for a period of time, and issues a report

Save to My Library