Evaluation of Hospital-based Sentinel Surveillance for Invasive Bacterial

Vaccine Preventable Diseases (IB-VPD)

Sentinel Site Evaluation Questionnaire

Country:

Sentinel site hospital:

Name and contact information of WHO country focal point for IB-VPD surveillance:

Name and contact information of Ministry of Health focal point for IB-VPD surveillance:

Names of evaluation team lead and members:

Evaluation date:

Revision: January 2012

2

Evaluation Objectives and Methodology

Thank you for performing this evaluation of Hospital-Based Sentinel Surveillance for Invasive Bacterial Vaccine Preventable Diseases (IB-VPD).

The objective of the IB-VPD sentinel surveillance network is to:

Document disease burden and describe the epidemiology of Invasive Bacterial Vaccine Preventable Diseases in a country or defined geographic area.

Data from the global IB-VPD surveillance network have been used to support introduction of vaccines against diseases caused by Haemophilus influenzae type b (Hib), Streptococcus pneumoniae (pneumococcus) and Neisseria meningitidis (meningococcus). On-going surveillance is needed to monitor the impact of vaccination on the epidemiology of these diseases.

The objectives of the evaluation of the IB-VPD sentinel site are to:

Evaluate sentinel sites using standardized tools and methodology

Determine the structure and management of the IB-VPD sentinel surveillance

Determine strengths and areas for improvement

Provide immediate on-site verbal feedback and training to improve quality

Provide a written report within two weeks to WHO HQ and RO using the standard template

Roles and responsibilities of the evaluation team

Prior to the evaluation

Confirm dates of the evaluation with WHO Regional and Country Offices

Ensure participation of Ministry of Health staff in the evaluation whenever possible

Confirm the time and place for an initial meeting with the Ministry of Health

Confirm the time and place of the evaluation with sentinel site. Confirm that an authority in charge of the hospital or laboratory is expecting the team

Request information and data prior to the evaluation from WHO Regional and Country Offices (see page 3)

Prepare country folder with materials for each site to be visited (see page 3)

Review with the WHO focal point for IB-VPD surveillance the data collected

During the evaluation

Conduct the evaluation with Ministry of Health staff and WHO country focal point

The team lead should ensure that observations are recorded on this evaluation form

Provide verbal feedback, and training as needed; for the sentinel site authorities (see page 21)

Following the evaluation

Send a written report plus this completed evaluation questionnaire, following review and agreement by team members, to WHO RO and HQ within two weeks of evaluation visit (see Part VII, page 23)

3

Methodology of the evaluation

Prior to the evaluation 1. Collect information in advance via WHO Regional and Country Offices:

From the Ministry of Health surveillance department: Request data to compare with global WHO data, for example, number of paediatric (<5 years) hospital admissions with suspected diagnosis of meningitis (Tier 1) or meningitis, pneumonia, and sepsis/septicaemia (Tier 2), for each of the preceding 12 months

Number of hospitalizations among children <5 years of age by clinical diagnosis, Sentinel Site: _________

Clinical diagnosis Months

1 2 3 4 5 6 7 8 9 10 11 12

Tier 1: Meningitis

Tier 2: Pneumonia

Sepsis/septicemia

From the Ministry of Health Expanded Programme on Immunization: Request national and sub-national data on vaccine coverage/usage to compare indicators in the region where the sentinel site is located to other regions

From the sentinel site– Request copies of most recent reporting forms sent to MoH

2. Prepare packets for evaluators to review prior to arrival in country and to give to MoH, WHO and sentinel site once in country. Include:

Visual Aids Identifying Bacteria Causing Vaccine-Preventable Meningitis and Suspect Meningitis: Process for Collecting CSF Fluid [Posters and Colour Handouts—bring several copies]

Surveillance Reports Most recent Global Invasive Bacterial Vaccine Preventable Diseases (IB VPD) Information and Surveillance Bulletin and Global Rotavirus Information and Surveillance Bulletin http://www.who.int/nuvi/surveillance/resources/en/index.html

Immunizations Most recent GAVI information available for the country http://www.gavialliance.org/country/ [Note: Accepted proposals and reports are available on the site. Recent applications may be available from WHO] Immunization profile http://www.who.int/immunization_monitoring/data/en/index.html

Disease Burden Mortality and other statistics--Global Health Observatory: http://www.who.int/gho/countries/en/index.html Global burden of disease estimates http://www.who.int/topics/global_burden_of_disease/en/ Country/region rotavirus disease burden estimate http://www.who.int/immunization_monitoring/burden/rotavirus_estimates/en/index.html Country/region pneumococcal disease burden estimate http://www.who.int/immunization_monitoring/burden/Pneumo_hib_estimates/en/index.html

4

Laboratory Manual World Health Organization. Laboratory Methods for the Diagnosis of Meningitis caused by Neisseria meningitidis, Streptococcus pneumoniae and Haemophilus influenzae, 2nd edition. WHO/IVB/11.12. Geneva: WHO, 2011. http://whqlibdoc.who.int/hq/2011/WHO_IVB_11.09_eng.pdf

During the assessment

Bring a hand-held calculator

1. Explain to those interviewed that information is being gathered to strengthen a global surveillance system, not to evaluate the work of individuals.

2. First meet with the Ministry of Health Prior to the sentinel site evaluation, hold an interview with the Ministry of Health focal point for surveillance. If the Ministry does not have a focal point, take the opportunity to provide information about the surveillance system to the Ministry of Health officials. All interviews should be arranged and conducted through and with WHO IB-VPD focal point.

3. Conduct the evaluation in 3-4 hours. It is preferable to conduct the evaluation in the morning, so that additional activities, such as review of data or laboratory supervision, may be scheduled for the afternoon. If the evaluation is performed by a team composed of surveillance and laboratory persons, the team should stay together.

4. Conduct the sentinel site evaluation in this order: 1. Initial meeting with hospital sentinel site IB-VPD surveillance management team 2. Patient intake and physical examination areas 3. Specimen collection area 4. Specimen handling and transport to the laboratory 5. Laboratory specimen processing 6. Data management, results and reporting During the evaluation, try to obtain copies of all relevant forms, logbooks, etc.

5. Verbal debrief with hospital sentinel site management team, include Ministry of Health, WHO country focal person

6. With permission of hospital management, conduct on-site training, especially in the laboratory, as needed

7. Complete this evaluation questionnaire Questions to ask are written in regular font. Instructions for the interviewer are written in italics. Spaces are provided to record observations. Focus on information that may be useful for training and improving data quality or that requires follow-up (such as supplies of essential reagents or maintenance of equipment). Use blank pages for additional notes.

After leaving the sentinel site 1. Hold a meeting of team members to discuss findings, and complete an initial written report

using the template in this document 2. Provide the final written report and this completed evaluation form, reviewed by team

members, to WHO within 2 weeks

Observe first and then ask the evaluation questions

Questionnaire

Part I. Interview with Ministry of Health focal point for IB-VPD surveillance Instructions to interviewer: Thank the individual for participating and ask for his/her name, title, department and contact information using form on page 28. Explain the objectives of IB-VPD sentinel surveillance and the evaluation. Estimated time for interview: 30 minutes. A) Background Information

1. Does the Ministry contribute data to the WHO global IB-VPD surveillance network? Yes No [If the response is “No” or “I don’t know” use the opportunity to give additional information.]

2. Does the MoH receive surveillance bulletins from the WHO? Yes No If no, identify who in MoH should receive bulletins: If yes, does the MoH distribute bulletins nationally? Yes No

3. How many sentinel sites in the country perform IB-VPD surveillance? __________ a. List the names of the participating hospitals:

4. How many of these hospitals also perform surveillance for rotavirus diarrhoea? __________

B) Sentinel Hospital Selection 5. Comment on the selection of IB-VPD sentinel site (s) - why and how were these sites chosen?

6. For each site, complete the table below:

Sentinel Hospital name

Catchment geographic area*

Number of children <5 years of age in geographic area

Estimated % of children <5 yrs of age living in catchment geographic area

who would be taken to sentinel hospital (vs. other hospitals) for care

*Catchment geographic area is the sub-national level(s) where patients reside who are admitted to the hospital. This could be a city, district(s) or a combination of these.

C) Surveillance Data Availability and Use 7. Does the MoH receive data or reports regularly according to the agreed schedule (i.e. monthly or

quarterly) from the sentinel sites? Yes No [If yes, ask to review data] 8. Have data from the surveillance been used by the Ministry of Health? Yes No

If yes, how? [i.e., GAVI applications, presentations, publications or special studies] D) Ministry of Health Oversight and Vision of Surveillance

6

9. In the past 6 months, has MoH conducted supervision activities for IB-VPD sentinel sites? Yes No 10. If yes, what type of supervision activities has the MoH conducted for IB-VPD sites?

11. Does the MoH plan to continue surveillance for IB-VPD? Yes No If no, why not?

12. What are MoH objectives regarding surveillance for vaccine-preventable diseases?

13. What are the two (2) most important strengths and two (2) most important weakness of IB-VPD surveillance?

Thank the individual again for participating and ask if he/she has any questions. After the interview, complete the following, to be used in any verbal or written report. 1) The most important strength identified:

2) The most important weakness identified:

7

Part II. Interview with IB-VPD hospital sentinel site management team

Instructions: Thank hospital authorities for participating. Complete form on page 28 with names, titles, departments and contact information of key people. Explain objectives of IB-VPD surveillance and evaluation:

To observe how the sentinel site is functioning and provide guidance if needed

To identify ways to improve the data quality for invasive bacterial disease surveillance Describe how the visit will be conducted (e.g. this initial interview, followed by walk through of patient admission and specimen collection areas, then the laboratory, and finally to the data collection area.) Explain that the assessment will last 3-4 hours, and ask for permission to provide on-site corrections, if required, in the event that major problems are identified. Schedule at least one hour at the end of the assessment to provide preliminary verbal feedback for the hospital authorities. This initial interview should take 30 minutes. A) Background Information

1. What type of hospital is this? Public Hospital Private Hospital 2. Is this a: General Hospital Paediatric Hospital Other ___________________________ 3. Complete the information below for the sentinel site:

Catchment geographic area Population <5 years of age in geographic area

Estimated % of children <5 yrs of age living in catchment geographic area

who would be taken to sentinel hospital (vs. other hospitals) for care

1. What type of IB-VPD surveillance does this hospital conduct?

Tier 1 (meningitis) surveillance only

Tier 2 (meningitis-pneumonia-sepsis) surveillance

Tier 3 (population-based) surveillance

B) Surveillance Coordination

2. Is there a focal point or coordinator for IB-VPD surveillance in this hospital? Yes No

3. Is there a focal point for the laboratory component of the surveillance? Yes No

4. Is there a data manager focal point at the hospital? Yes No

5. Describe how these individuals coordinate activities:

8

C) Standard Operating Procedures: Obtain copies and review after interview

6. Are there IB-VPD surveillance protocols, SOPs, manuals or guidelines at this site? Yes No

a. If yes, ask to observe the document(s) and obtain a copy. Observed? Yes No Copy obtained? Yes No

7. If the above document/s is/are available, indicate which of the surveillance aspects below are: Described? Consistent with WHO criteria?

Case definitions Yes No Yes No Case detection Yes No Yes No Case registration Yes No Yes No Completion of case report forms Yes No Yes No Specimen collection Yes No Yes No Specimen transportation to lab Yes No Yes No Laboratory processing Yes No Yes No Data management and analysis Yes No Yes No

8. Do SOPs include how many tubes or how much CSF should be obtained Yes No

If yes, how many tubes are stated? One Two Three More than Three

If yes, how much CSF should be taken in total in children < 5 yrs: _____________ mL

9. Do SOPs include criteria for the minimum volume of blood to be taken for cultures? Yes No Unknown

If yes, what is the minimum volume of blood to be collected from: Children <5 years of age ______________ mL Others _________________ mL

10. Do SOPs include criteria/standards for the maximum time interval between specimen collection and arrival in the laboratory? Yes No

If yes, what is the recommended maximum time interval? <1 hour >1 hour

D) Training, Supervision, and Use of Data

11. Is there a plan for continuing education/training for staff involved in IB-VPD surveillance?

Yes (describe below) No Unknown

12. During the past 12 months, has the hospital received supervisory visits for surveillance? Yes No Unknown

If yes, for whom? For clinical staff: Yes No Unknown

9

For laboratory staff: Yes No Unknown

Please indicate month and year of two (2) last supervisory visits received from national level?

(month/year)______________ (month/year)______________

13. Do you receive feedback from the national level on data you have reported, e.g. about data quality, information on duplicated records, etc.? Yes No Unknown

14. Do you receive any of the following information about specimens taken from your site?

a. Serotypes Yes No If Yes, how often? ___________

b. Antimicrobial resistance Yes No If Yes, how often? _____________

c. Contamination Yes No If Yes, how often? _____________

15. Do physicians use individual patient laboratory results to guide antibiotic use?

Yes No Unknown

If No, please describe why not i.e. provide the limiting factor(s)_______________________

________________________________________________________________________

16. Has data from the surveillance system been used in the hospital? Yes No Unknown

a. If yes, please describe how:

14. What are the two (2) most important strengths and two (2) most important weakness of IB-VPD surveillance?

Thank the individuals for participating and ask if there are questions. After the interview, complete the following, to be used in the verbal feedback and written report. 1) The most important strength:

10

2) The most important weakness:

Move on to the hospital admissions area.

11

Part III: Sentinel site assessment of clinical aspects (case ascertainment and

specimen collection) and specimen transport to the hospital laboratory Walk through the hospital and follow the flow of surveillance information. Begin with observing admission procedures for children <5 years and then the specimen collection area. Depending upon the facility, initial patient evaluation may take place in an emergency department, outpatient clinic or urgent care unit. Obtain copies of forms used (case report/investigation form, lab request form, etc.) to complete the table on page 14 and data management sections of this questionnaire.

Record your observations. Ask questions but do not interfere with normal activities. The time required for this part of the assessment will depend on the facility, but should not take more than about 45 minutes to observe the flow of patients and collection of data and specimens.

A) Case Detection, Registration, and Reporting 1. Do the staff follow a standard protocol or algorithm for patient evaluation?

[i.e. Integrated Management of Childhood Illness] Yes No

2. Are visual aids displayed for the staff to follow a protocol? Yes No

3. Which of the following clinical presentations in a child would result in investigation of possible invasive bacterial disease and specimen collection (Check all that apply.)

Fever Rapid breathing Impaired consciousness Difficulty in breathing Refusal to feed Other (Specify:) Fitting (seizures) Bulging fontanelle Excessive crying Stiff neck Prostration (inability to sit upright in a child normally able to do so, or to drink in a child too young to sit)

4. Describe the process from when a child seeks care for any of the above signs and symptoms to where the case is registered and entered into the IB-VPD surveillance system. Comment if this differs during the day, evening, weekends, holidays, etc.

12

B) Hospital Log-Book Reviews: Admissions and laboratory log-books Rapidly review the hospital admissions log-book for the past month to determine the following:

17. In the past 30 days, were at least: a) a total of 100 children <5 years of age admitted to the hospital for any illness? Yes No

b) a total of five (5) children <5 years of age admitted with meningitis? Yes No

Collect information on three (3) hospital admissions among children younger than 5 years with suspected meningitis admitted over 30 days ago. Follow the records of these children through the system, including in the specimen collection log-book, lab log book and the surveillance data base.

C) Specimen Collection and Transport

Refer to the visual aid “Suspect Meningitis: Process for Collecting CSF Fluid”. Determine whether procedures differ from the WHO recommended protocol.

20. Which specimens are collected for IB-VPD surveillance at the hospital? How do procedures differ during nights, weekends or holidays?

Specimens collected Y or N Night/ weekend/holiday

CSF Y or N

Blood Y or N

Pleural fluid Y or N

Other fluid: ________________________ Y or N

Clinical information: completed during visit to admissions and specimen collection areas

Laboratory information: completed in hospital laboratory

Case info in surveillance data

base Patient ID number and name

Gender Age Specimen collected

Time and date of

specimen collection

Lab received all

specimens? Y/N

Time and date received in lab?

AND Received within

in 1 hour?

Clinician notified of

result? Y/N,

If yes, day & time?

Patient in data base? Y or N

If yes, all info

correct?

Male

Female

CSF Blood

Y or N

Y or N

Y or N

When:

Y or N

Correct?: Y or N

Male

Female

CSF Blood

Y or N

Y or N

Y or N

When:

Y or N

Correct?: Y or N

Male

Female

CSF Blood

Y or N

Y or N

Y or N

When:

Y or N

Correct?: Y or N

13

21. Are specimens routinely taken before antibiotic therapy is initiated?

Always Mostly Sometimes Rarely

22. Are supplies of essential materials (i.e. lumbar puncture kits, gowns and gloves, sterile equipment supplies for blood collection, etc.) available?

Always Mostly Sometimes Rarely

Observe stock on hand and comment:

23. How many tubes of CSF are usual taken from suspect meningitis cases? One Two Three >3

24. How much CSF is usually taken in total from children <5 yrs of age? ___________________mL

25. What is the minimum volume of blood to be taken for cultures? Children < 5 years_________ mL Others _________________ mL

26. Do specimen collection procedures follow sterile techniques? Yes No

27. If multiple tubes or specimens are collected, describe what is done with each tube:

28. Is specimen volume measured or recorded? Yes No

If yes, describe what is done if specimen volume is inadequate

29. How are specimens handled or stored while awaiting transport to the laboratory: temperature,

container (protected from light?), average time and maximum time, materials.

14

30. Is there a supply of Trans-Isolate media? Yes No 31. Is there a focal person to oversee specimen transport to the laboratory? Yes No

32. Is the time of collection of the specimen recorded on the ward? Yes No

33. Is the time specimens are received recorded in the laboratory? Yes No

34. Are results of laboratory testing sent immediately to medical staff (i.e. results of gram stain provided to clinical staff within one (1) hour)? Yes No

35. Are the results of blood or CSF culture for bacteria or antibiotic susceptibility testing sent to the ward in a timely manner? Yes No

34. What are the two (2) most important strengths and two (2) most important weakness of IB-VPD

surveillance?

Thank the individuals for their cooperation and ask if there are any questions. Complete the following, to be used in the verbal feedback and written report. 1) The most important identified strength in the clinical and specimen handling components:

2) The most important identified weakness in the clinical and specimen handling components:

Move onto the laboratory.

15

Instructions: After the walk through (potentially in evening following the visit), review the Case Report Form/Case Investigation Form and Other Documents and complete the following.

Is the same unique patient ID recorded on case-report forms and specimen labels? Yes No

Variable description Case report form

Specimen label

Present? Present?

Epidemiology and case identification

Unique patient ID* Y or N Y or N

Name of patient Y or N Y or N

Age Y or N Y or N

Sex Y or N Y or N

Residence district/municipality Y or N Y or N

Residence province/state Y or N Y or N

Clinical signs and symptoms Y or N Y or N

Date of symptom onset Y or N Y or N

Initial diagnosis Y or N Y or N

Final diagnosis Y or N Y or N

Antibiotic use prior to admission Y or N Y or N

Vaccination history Y or N Y or N

Outcome Y or N Y or N

CSF specimen

Lumbar puncture performed Y or N Y or N

Date of CSF specimen collection Y or N Y or N

Time of CSF specimen collection Y or N Y or N

Appearance (clear, cloudy, bloody) Y or N Y or N

CSF chemistry Y or N Y or N

Leukocyte count Y or N Y or N

Results of Gram stain Y or N Y or N

Results of antigen detection Y or N Y or N

Bacterial culture performed Y or N Y or N

Results of bacterial culture Y or N Y or N

Blood specimen

Blood specimen collected for bacterial culture Y or N Y or N

Date of blood specimen collection Y or N Y or N

Time of blood specimen collection Y or N Y or N

Bacterial culture performed Y or N Y or N

Results of bacterial culture Y or N Y or N

Chest x-ray

Chest x-ray performed Y or N Y or N

Date of chest x-ray Y or N Y or N

Time of chest x-ray Y or N Y or N

Test results

Antibiotic susceptibility Y or N Y or N

PCR (specify: Y or N Y or N

16

Part IV. Sentinel site assessment: laboratory specimen processing

Refer to the visual aid “Identifying Bacteria Causing Vaccine-Preventable Meningitis”. Observe how clinical specimens are received, processing of CSF, blood and other specimens, functioning of equipment and available stocks of essential materials (i.e. media, reagents, supplies). Pay attention to quality control procedures, including testing media, maintenance of reference strains, use of proper controls, etc. Obtain copies of laboratory reporting forms to complete the data management portion of this questionnaire.

A) Hospital Log-Book Reviews: Admissions and laboratory log-books First, complete the relevant sections of the table on page 11 for 3 suspected meningitis cases. This information will provide you with a quick impression of the information flow at the sentinel site.

B) Standard Operating Procedures 1. Are copies of standard protocols available for reference? Yes No 2. Are visual aids displayed for the laboratory staff to follow? Yes No

C) Specimen Processing and Information Recording 3. For CSF specimens, which of the following are performed in the laboratory? Indicate the normal

order in which they are performed (1=first, 2=second, etc): Procedures Hospital laboratory

Performed? Y or N

Order (1st, 2nd, 3rd, etc)

CSF examination

Microscopy: White Blood Cell count Y or N

Microscopy: Gram stain Y or N

Chemistry: Protein Y or N

Chemistry: Glucose Y or N

Centrifugation Y or N

Latex agglutination Y or N

Immunochromatographic (Binax) tests Y or N

Culture on Blood Agar Plate Y or N

Culture on Chocolate Agar Plate Y or N

PCR (conventional or real-time) Y or N

4. For tests (such as protein or leukocyte count) that may not performed in the microbiology laboratory, are results included on case-report forms? Yes No

5. Does the laboratory have an adequate stock of essential reagents for routine CSF examination and culture? Yes No

6. Does the laboratory have an adequate stock of essential reagents for routine blood specimen processing? Yes No

7. Does the laboratory perform tests for identification/characterization of S. pneumoniae, H. influenzae and N. meningitidis? Yes No If yes, which tests for which bacteria?

17

8. What type of blood does the laboratory use for solid media? (check all that apply) Sheep Horse Goat Human

D) Reporting to Clinicians 9. For urgent cases, does the lab record the time results are reported to clinicians? Yes No 10. Are Gram stain laboratory results reported to medical service within 1 hour? Yes No

11. Are complete laboratory results reported to the medical service within 48 hours of specimen receipt? Yes No E) Specimen/Isolate Storage and Collaboration with Reference Laboratory

12. Are bacterial isolates stored for further testing? Yes No If yes, describe how isolates are stored.

13. If CSF remains after processing, how are CSF specimens stored?

Not stored refrigerator -20 freezer -70 freezer

14. Does the sentinel site laboratory send samples and isolates for testing at a national or regional reference

laboratory? Yes No

If yes, when were specimens and isolates last sent? __________

What percentage of isolates are sent for quality control? ____

And how frequently are isolates sent for quality control? _______________

F) Quality Assurance 15. Does the laboratory participate in External Quality Assessment? Yes No

If yes, which EQA program? ________________________________

When was the EQA last performed? _________________________ Did the laboratory receive a passing score? Yes No

16. Are batches of blood and chocolate agar plates routinely tested with standard strains of S. pneumoniae, H. influenzae and N. meningitidis? Yes No

17. Are temperatures of refrigerators, freezers and incubators monitored daily? [Observe sheets] Yes No

18. Do any of the protocols and procedures differ during nights and weekends? Yes No If yes, describe:

18

G) Physical Aspects of the Laboratory

19. What is the general condition of the laboratory building and the infrastructure. Score from 1 (lowest) to 5 (highest) LOW HIGH

a) Condition of walls and floors: 1 2 3 4 5 b) Windows and doors 1 2 3 4 5 c) Benches / work space 1 2 3 4 5 d) Lighting 1 2 3 4 5 e) Waste disposal 1 2 3 4 5 f) Equipment 1 2 3 4 5

20. Are work areas clean and well maintained? Yes No

21. Comment on lab's physical appearance and organization: infrastructure, equipment, biosafety/waste disposal, sinks, nonporous bench tops (for disinfection), autoclaves, electricity, auxiliary power supply and division of rooms/areas for different activities.

19

Instructions: Complete the following laboratory checklist. Equipment must be working; reagents/supplies must be available in quantities sufficient for routine use until new supplies arrive.

Available and

Working? Yes or No

Equipment

Microcentrifuge Y or N

Light microscope Y or N

O2 Incubator Y or N

CO2 incubator or candle jar Y or N

Refrigerator Y or N

-20°C Freezer Y or N

-70°C Freezer Y or N

Water bath and/or heat block Y or N

Vortex Y or N

Automated blood culture Y or N

Conventional PCR machine Y or N

Real-time PCR machine Y or N

Clean area for PCR reagent mix Y or N

Clean autoclave for media prep Y or N

Autoclave for infectious waste Y or N

Biosafety Cabinet Y or N

Reagents

Gram stain reagents (QC’d) Y or N

Nucleic acid extraction reagents (if PCR is performed)

Y or N

Latex agglutination kits Y or N

Immunochromatographic kits (e.g. Binax) Y or N

Catalase Y or N

CTA sugars Y or N

Supply of blood agar plates Y or N

Supply of chocolate agar plates Y or N

Tryptic soy agar plate Y or N

Antibiotic disks/E-test strips Y or N

Deoxycholate solution Y or N

Growth factors for Haemophilus Y or N

Oxidase strips Y or N

Skim milk and glycerol Y or N

Optochin disks Y or N

Bacterial reference strains Y or N

Available and

Working? Yes or No

Supplies

Gloves Y or N

Lab coats Y or N

Goggles Y or N

Microscope slides Y or N

Immersion oil Y or N

Ethanol Y or N

Bleach/disinfectant Y or N

Blood culture bottles Y or N

Loops/flame Y or N

Pipettors and tips Y or N

1.5 ml centrifuge tubes Y or N

Cryovials for storage Y or N

Thermometers Y or N

Silica packets and swabs to ship isolates Y or N

Conventional blood culture supplies Y or N

Additional resources

System for tracking inventory and supplies

Y or N

List of manufacturers for supplies Y or N

Repair contracts for equipment Y or N

Daily monitoring record on incubator Y or N

Backup electricity supply Y or N

20

22. What are the two (2) most important strengths and two (2) most important weaknesses of IB-VPD surveillance?

Thank the individuals for their cooperation and ask if there are any questions. Complete the following, to be used in the verbal feedback and written report. 1) The most important identified strength of the laboratory aspects of surveillance:

2) The most important identified weakness of the laboratory aspects:

Move onto the data management area.

21

Part V. Sentinel site assessment: data management, analysis and reporting

Ask to review the file of Case Report Forms and the Surveillance log book.

A) Surveillance Data Management 1. Who is responsible for updating and maintaining the Case Report Forms / Case Investigation Forms

/ Case Log Book? Describe the process of how these are completed.

2. Are the 3 meningitis cases identified in the hospital admissions log book on page 11 recorded in the

IB-VPD surveillance database? (if no data base on site, then in surveillance log book?)Yes No If no, why not?

3. How often is the hospital admissions log book reviewed to identify suspected meningitis cases to

be enrolled into IB-VPD surveillance? It is not reviewed Weekly Monthly Quarterly Other (describe):_________________________________________________________________

4. Are other log books or registers reviewed to locate suspected meningitis cases? Yes No

If yes, describe:

5. Pick a month that ended 30 days before this visit. For the suspected cases admitted to the hospital for that month, is the data complete, with clinical, lab and outcome data recorded? Yes No

6. Is there a problem with the linking of clinical and laboratory data? Yes No 7. Describe how the clinical and laboratory data are linked:

8. Who does the hospital send the surveillance data to (check all that apply)?

MoH WHO CO WHO RO Other Describe: 9. How and how often are surveillance data sent to health authorities or to WHO for surveillance?

- Telephone Monthly Quarterly 6 Months Yearly Other

- Fax Monthly Quarterly 6 Months Yearly Other

- Mail Monthly Quarterly 6 Months Yearly Other

- Computer with Internet Monthly Quarterly 6 Months Yearly Other

Describe:

22

10. Does anyone verify the data prior to sending it? Yes No

If yes, who?

11. Is the data used at the hospital? Yes No

If yes, how (e.g. to guide antibiotic administration, etc)?

B) Infrastructure and Data Security 12. Does the sentinel site have a computer for IB-VPD surveillance? Yes No

If yes, are any surveillance data entered into the computer? Yes No If there is a computer, is the surveillance data backed up? Yes No

13. Are there procedures defining who accesses or modifies data (i.e. password protection)? Yes No C) Surveillance Data

14. Using surveillance reports or laboratory data, complete the table for each of the past 12 months working with the person responsible for data management at the surveillance site:

Meningitis (Tier 1) surveillance data and indicators for cases among children <5 years of age

Data Month

1 2 3 4 5 6 7 8 9 10 11 12

Number of suspected meningitis cases admitted to hospital*

Number of suspected meningitis cases enrolled in surveillance

Number of enrolled cases with CSF collected

% of enrolled cases with a lumbar puncture/CSF

Number of enrolled cases with a lumbar puncture with probable bacterial meningitis**

Number enrolled cases with a lumbar puncture with laboratory confirmation of:

N. meningitidis H. influenzae S. pneumoniae

*May not be tracked electronically by surveillance system. If too time consuming to complete, leave blank. **Probable bacterial meningitis: A suspected meningitis case with CSF examination showing at least one of the following: Turbid appearance, or leukocytosis (> 100 cells/mm3) or leukocytosis (10-100 cells/ mm3) AND either an elevated protein (>100 mg/dl) or decreased glucose (< 40 mg/dl) Note: if protein and glucose results are not available, diagnose using the first two conditions

23

15. If the hospital conducts Tier 2 (meningitis-pneumonia-sepsis) surveillance, compare numbers of cases from the surveillance system with numbers of children age <5 years admitted to the sentinel hospital with the following clinical diagnosis during the same period.

Monthly hospitalizations among children <5 years of age

Clinical diagnosis Month

1 2 3 4 5 6 7 8 9 10 11 12

Pneumonia

Sepsis/septicemia

Other

18. What are the two (2) most important strengths and two (2) most important weakness of IB-VPD surveillance?

Thank the individuals for their cooperation and ask if there are any questions.

Complete the following, to be used in the verbal feedback and written report. 1) The most important identified strength for data management issues:

2) The most important identified weakness for data management issues:

The formal evaluation is now completed. Take some time to gather your thoughts for the verbal

debriefing with hospital authorities.

24

Part VI. Preliminary verbal feedback for hospital authorities Instructions: Explain briefly how the assessment was conducted and which departments were observed. Summarize the main findings related to the surveillance for invasive bacterial diseases (below). The summary should take no more than 20minutes. This should follow with a discussion of the findings and recommended next steps that should continue for as long as needed. Report the information on the most important strength and weakness in each area evaluated that were completed on this questionnaire during the assessment.

Inform the hospital authorities that a written report will be provided by the evaluation team to WHO, and that WHO will forward to the MoH. If it is needed, ask the hospital authorities for permission to return to the laboratory or other areas to either hold in depth feedback discussions or training, as needed. If substantial training will be required, this should be discussed with the MoH and WHO representatives before any such training is scheduled. Thank the hospital authorities again for participating.

Notes

Template for Final Report Name of Sentinel Hospital

Evaluation of Invasive Bacterial Vaccine Preventable Diseases (IB-VPD) Surveillance City, Country, Date of Evaluation

Key Persons Met:

List: Name, title

Evaluation Team Members:

List: Name, title

The team met with the Ministry of Health and sentinel hospital key professional staff members involved with IB-VPD surveillance and conducted a rapid assessment of surveillance activities. This was the first (second) assessment of the sentinel site surveillance activities. The team is grateful to the Ministry of Health and sentinel site coordinators for the information provided.

Overall recommendation (check one):

Sentinel surveillance and laboratory functioning well: follow-up in 6-12 months

Sentinel surveillance or laboratory need improvement: support needed

Site does not meet minimum criteria for sentinel surveillance: review site selection

Summary report of standardized findings:

Include this completed table in the final written report to provide a standardized score for this

visit that will be useful to evaluate the hospital's performance over time.

Current visit

Current Score

Previous visit

Sufficient number of the target population enrolled into surveillance.

1) Sentinel surveillance site enrols all children <5 years of age with suspected meningitis into surveillance.

Y N 0 or 1 Y N

2) Sentinel site enrols at least 100 children with suspected meningitis annually (Tier 1).

Y N 0 or 1 Y N

Quality clinical specimens taken and transferred to laboratory in a timely basis.

3) At least 90% of all enrolled children with suspected meningitis have a lumbar puncture performed.

Y N 0 or 1 Y N

4) Clinical specimens are collected following WHO protocols (before antibiotics, sterile technique, appropriate volumes etc).

Y N 0 or 1 Y N

5) CSF specimens arrive at laboratory within 1 hour of collection or are inoculated into Trans-Isolate media.

Y N 0 or 1 Y N

Quality laboratory processing of specimens with correct identification of bacterial pathogens.

6) The laboratory maintains reference strains and performs quality control on media.

Y N 0 or 1 Y N

7) The laboratory uses appropriate solid media (i.e. sheep, horse, or goat blood) and never uses human blood.

Y N 0 or 1 Y N

8) Either latex agglutination or immunochromatigraphic test (Binax) is performed on CSF specimens.

Y N 0 or 1 Y N

26

9) Clinicians are notified of Gram stain results of CSF specimens within 1 hour of specimen receipt.

Y N 0 or 1 Y N

10) Complete laboratory results are provided to medical staff within 48 hours of specimen receipt.

Y N 0 or 1 Y N

11) Isolates and appropriate specimens are stored according to WHO protocols.

Y N 0 or 1 Y N

12) Isolates and appropriate specimens are transported to the National Reference Laboratory or Regional Reference Laboratory for serotyping and quality control.

Y N 0 or 1 Y N

13) Sentinel laboratory is identifying bacterial organisms in CSF. Y N 0 or 1 Y N

14) Laboratory participated in EQA programme in past 12 months.

Y N 0 or 1 Y N

15) Laboratory received a passing score in the EQA programme during the past 12 months.

Y N 0 or 1 Y N

High quality surveillance data is available, and shared with national authorities.

16) The sentinel hospital maintains an accurate electronic/paper data system to track patients and laboratory specimens.

17) Data for cases enrolled into surveillance is consistent through the process from hospital admission to final patient outcome, including linking of clinical and all laboratory data.

Y N 0 or 1 Y N

18) Appropriate data management processes are followed including secure data storage, back-up, cleaning, and analysis.

Y N 0 or 1 Y N

19) Surveillance data that has been appropriately 'cleaned' is regularly analysed / assessed.

Y N 0 or 1 Y N

20) Data is forwarded to relevant authorities including Ministry of Health according to the agreed timeline, or at least quarterly.

Y N 0 or 1 Y N

21) WHO received the sentinel site data according to the agreed timeline, or at least quarterly.

Y N 0 or 1 Y N

22) The Ministry of Health and the sentinel site received feedback on the IB-VPD surveillance within the past 6 months.

Y N 0 or 1 Y N

23) Ministry of Health conducted supervision visit within past 6 months.

Y N 0 or 1 Y N

24) Essential clinical and laboratory supplies and equipment are available, functioning and monitored regularly.

Y N 0 or 1 Y N

25) Sentinel hospital conducts surveillance activities 7 days a week.

Y N 0 or 1 Y N

Standard score for visit (0 to 25)

Template for Final Report

Detailed report:

Structure the final written report following the below format that is organized to follow the structure of this evaluation questionnaire.

Surveillance Component Strengths and Weaknesses Recommendations

Part I: Ministry of Health A) Background information B) Sentinel hospital selection C) Surveillance data availability and use D) MoH oversight and vision for surveillance

Strengths:

Weaknesses:

Write 'None' or list via bullet points

Part II: Hospital management team A) Background information B) Surveillance Coordination C) Standard Operating Procedures D) Training, supervision, and use of data

Strengths:

Weaknesses:

Write 'None' or list via bullet points

Part II. Clinical aspects and specimen transport to the hospital laboratory A) Case detection, registration, and reporting B) Hospital log-book review C) Specimen collection and transport

Strengths:

Weaknesses:

Write 'None' or list via bullet points

Part IV. Laboratory specimen processing A) Log book review B) Standard operating procedures C) Specimen processing and information recording D) Reporting to clinicians E) Specimen/Isolate Storage and Collaboration with Reference Laboratory F) Quality assurance G) Physical aspects of the laboratory

Strengths:

Weaknesses:

Write 'None' or list via bullet points

Part V. Data Management, Analysis, and Reporting A) Surveillance data management B) Infrastructure and data security C) Surveillance data

Strengths:

Weaknesses:

Write 'None' or list via bullet points

28

29

This page left blank intentionally to allow contact table on the next page to be torn off and used on site.

Names and Contact Information

Name Title Department Telephone e-mail