Legal and Regulatory Framework Assessment
Legal and Regulatory frameworks are critical to understanding what surveillance covers and how data governance occurs including internal standards related to data collection, storage, analysis and sharing. Having a clear and narrow research question prior to starting any legal and regulatory framework assessment is essential as not all policies are relevant. Additionally, determining the importance of or weighting the policies when determining any proposed program changes is critical since not all policies are as important as others.
Key Questions to Consider:
- What are my specific research questions related to surveillance activities?
- How do certain legal and regulatory frameworks impact anticipated program changes?
- Who are the key actors that need to be consulted to understand our research question?
Surveillance System Survey
It is important to target your survey questions and collect only data that you need. The survey tool for the Assessment in Peru was long and there were many adaptations to meet different respondent roles. Although there may be times this is useful, the Assessment Team learned the value of creating a survey that is short and targeted to the key audience. Make sure questions map directly to your key assessment questions and do not be tempted to keep adding questions!
Key Questions to Consider:
- What will I do with this information?
- Is it directly related to answering key questions?
Minimize qualitative questions and use notes rather than audio record. Qualitative questions require significant more time and often different skills to analyze, relying on more than one person to conduct the analysis and increasing the analysis time substantially.
Key Questions to Consider:
- Can this information be collected quantitatively using drop downs or Likert scales?
- Is this question essential to answering a key assessment question?
- How and who will be able to analyze this data?
- Will our timeline allow for increased data analysis?
Virtual data collection methods worked for our data collection. Conducting interviews using virtual methods allowed us to include more respondents and was cost effective. The quality of online interviews was found to be similar to those conducted in person in our assessment.
Key Questions to Consider:
- Will this allow me to include people I would otherwise not be able to include but should?
- Are there limitations to people participating using virtual methods (for example, lack of internet, cost of data)?
Surveillance Data Assessment
Evaluate quality of databases to what is available in the country before finalizing pathogens of interest and data analysis plan. First, explore what information exists, especially open data. Use available open data whenever possible and then develop with governmental stakeholders pathogen priorities to see what surveillance datasets exist, what you will have access to, and whether they (if you are not part of government) would participate in data analysis. If no access to the full datasets is granted, consider requesting data aggregated to the lowest possible level. If no access to aggregated databases is available, use published bulletins and extract data from dashboards and other websites and/or work with governmental agents to conduct the analysis.
Key Questions to Consider:
- What open data exists?
- What is the quality of the databases for inclusion?
- What are data privacy considerations when working with patient data?
During the assessment’s data analysis, several key lessons emerged that serve as instructive points for future studies.
Lessons Learned:
- Data cleaning presented its own set of challenges, particularly when integrating multiple datasets; not all merging efforts were successful, which lead to the exclusion of certain data.
- Adapting available data to fit within established frameworks, like those provided by the CDC-Peru, necessitated compromise; not every surveillance assessment indicator could be measured given the limitations of data access permissions and data sharing agreements and data relationships, which in our case confined us to evaluating only timeliness indicators at the individual case report level.
- While working with datasets that have unique identifiers, it is crucial to weigh the benefits and drawbacks concerning data privacy; even anonymized patient-level data could, theoretically, be de-anonymized using advanced techniques.
- It is important to acknowledge that the analysis is only as reliable as the data sources it is derived from. While the datasets likely mirror the actual surveillance system data, it cannot be fully certain without having access to the raw data from NetLab, SINADEF, or SISCOVID. This introduces the possibility that the post-processed data could have introduced bias in the results.
Interoperability Assessment
Consider a variety of methods for collecting specific types of data. For example, when developing information flow maps, business process analysis, and other informatics information, consider hosting a consultative workshop with key stakeholders to work together to understand and map data flow rather than incorporating questions into the main survey. If you want to triangulate the data flow maps or informatics information, present the data flow maps to sub-national persons to identify any regional variations rather than trying to create a flow map from scratch each time.
Key Questions to Consider:
- Who is the best person(s) to answer these questions?
- What is the best way to collect information from them?
Conclusions
The assessment conducted in Peru provided a scaffolding to design programming to address gaps.