the nccor measures registry and catalogue of surveillance systems

THE NCCOR MEASURES REGISTRY AND
CATALOGUE OF SURVEILLANCE SYSTEMS
Robin A. McKinnon, PhD, MPA
National Cancer Institute, National Institutes of Health
For the Registry and Catalogue Working Groups
Institute of Medicine Committee on Evaluating Progress in Obesity Prevention Efforts
Washington DC
June 29, 2012
1
Agenda
•
•
•
•
NCCOR Overview
Measures Registry
Catalogue of Surveillance Systems
Summary and Next Steps
Agenda
•
•
•
•
NCCOR Overview
Measures Registry
Catalogue of Surveillance Systems
Summary and Next Steps
The National Collaborative on Childhood Obesity
Research (NCCOR) brings together four of the
nation’s leading research funders to address the
problem of childhood obesity in America.
NCCOR Funders
Centers for Disease Control and
Prevention (CDC)
National Institutes of Health
(NIH)
National Center for Chronic Disease Prevention
and Health Promotion
Division of Nutrition Research Coordination
National Cancer Institute
National Heart, Lung, and Blood Institute
Eunice Kennedy Shriver National Institute of
Child Health and Human Development
National Institute of Diabetes and Digestive
and Kidney Diseases
Office of Behavioral and Social Sciences
Research
Division of Community Health
Division of Nutrition, Physical Activity and
Obesity
Division of Population Health
National Center for Health Statistics
Robert Wood Johnson Foundation
(RWJF)
U.S. Department of Agriculture
Food, Nutrition and Consumer Services
Center for Nutrition Policy and Promotion
Food and Nutrition Service
Research, Education and Economics
Agricultural Research Service
Economic Research Service
National Institute of Food and Agriculture
NCCOR Goals
Agenda
•
•
•
•
NCCOR Overview
Measures Registry
Catalogue of Surveillance Systems
Possible Next Steps
Measures Registry Background
• A key priority for NCCOR is promoting the use of common
measures and methods
• Standard measures are needed to describe and evaluate
interventions to prevent childhood obesity, particularly those
projects that address policies and environments
• A measures registry should facilitate access to existing
measures, identify gaps, and encourage development of new
measures
February 2009
Phase I: Concept
Development
NCCOR Workgroup
• Drafted Statement of
Work
• NCCOR
Coordinating Center
awarded contracts
• Quarterly NCCOR
meetings
• Regular conference
calls
Registry Development
Phase II: Definition and
Data Abstraction
Phase III: Web Site
Development and Testing
Expert Panel
• Provided feedback on
plans for overall process
• Reviewed standardized
abstraction form for data
collection
Website Design and
Development
• Web-tool: IMS
• Other website pages:
NCCOR Coordinating
Center
Data Abstraction
• Mathematica contractors
abstracted data
Subject-Matter Expert
Group Measures Review
• Extensive review of all
data for quality control
Usability Testing
• Formative research
• Product features and
functions
Measures Collation
• Compiled copies of actual
measures (PDF versions
and web links)
Launch
April 2011
Phase IV:
Dissemination
NCCOR
Communications
• Workgroup:
communications plan
• Development of
supporting products
and brochures
• Webinars
• Conference
presentations
• Other presentations
Measures Registry Overview
• Searchable online registry with 800+ measures
• Measures are categorized into four domains:
 Individual dietary behavior
 Individual physical activity behavior
 Food environment
 Physical activity environment
• Details about each measure include:
 Domain(s) measured
 Validity and reliability
 Protocols on use
 Settings, geographic areas, and populations
Measures Registry Features
• Search, browse, or
compare measures
•
Access specific details for
each measure
•
Link to related resources
•
Find measures in
development
•
Submit new measures for
inclusion in the registry
www.nccor.org/measures
Feedback
The NCCOR Measures Registry and Catalogue of Surveillance
Systems are excellent resources. I find it especially useful to be
able to access detailed information on hundreds of measures
and data sources all in one place. The websites are extremely
easy to navigate and provide everything I would want to know
about the measure or surveillance system. I highly recommend
these resources, especially to anyone preparing grants or
developing studies with a focus on obesity.
Keryn E. Pasch, PhD, MPH
Dep’t of Kinesiology and Health Education
University of Texas
Registry Usage
Number of hits
since launch:
~250,000
Average monthly
usage April 2011
through May 2012:
344 unique visitors
Agenda
•
•
•
•
NCCOR Overview
Measures Registry
Catalogue of Surveillance Systems
Summary and Next Steps
Catalogue Background
• Heightened interest in social, environmental and policy
determinants of childhood obesity
• Relatively little research has incorporated data linkages across
the ecological spectrum
• Awareness and use of data resources limited
Catalogue
Obesity-Related Data Resources
Researchers
February 2009
Phase I: Concept
Development
NCCOR Workgroup
• Developed product
concept document
• Quarterly NCCOR
meetings
• Regular conference
calls
Catalogue Development
Launch
February 2011
Phase II: Definition
Phase III: Development /
Testing / Review
Phase IV:
Dissemination
Subject-Matter Expert
Group
• Representatives from NIH,
RWJF (Bridging the Gap),
CDC
• Determined inclusion
criteria
• Profile pilot process
• Reviewed template
information
Website Design &
Development
• Web-tool
• Other website pages:
NCCOR Coordinating
Center
NCCOR
Communications
• NCCOR Coordinating
Center & NCI staff:
communications
plan
• Press release
• Supporting products,
e.g., brochure,
videos, infographics
• Webinars
• Conference
presentations
• Posters
• AJPM commentary
Usability Testing
• Product features &
functions
Profile Standards
• Template language, form
Profile Development/
Review
• Draft profile development
• NCCOR subject matter
expert group
• NCI staff
• Surveillance system
contacts review
• Science writer review
Catalogue Overview
• First-ever online directory of obesity-related data
resources
• One-stop access to data related to environmental
and policy factors, as well as health behaviors,
outcomes and determinants of obesity
• Includes almost 90 systems
• Provides information on national, state and local
data resources
• Encourages data-linkage across multiple levels
Inclusion Criteria
• Relevant to childhood obesity research
• Data collected within the last 10 years
• Publically available, raw data
• Gathered in the United States
Catalogue Features
• Identify and locate
surveillance systems of
interest
• Search by topic, e.g.,
physical education, disability
• Compare attributes across
systems
• Access profile information for
systems of interest
• Access other resources of
interest, e.g., summary
statistics, legislative
databases
www.nccor.org/css
Feedback
Too bad the Catalogue wasn’t designed when I was in
graduate school – I would’ve graduated a year
earlier! This tool is exactly what I needed when I was
designing my dissertation. I spent a countless amount of
time and energy sifting through data sources, in a very
haphazard manner, because I had no organized way of
identifying the data that I needed for my particular
study. Finding the right data source is a process that can
take weeks, if not months; with the Catalogue, it can be
completed in a day or two.
Daniel Taber, PhD
Postdoctoral Research Associate
University of Illinois at Chicago
Feedback Cont’d
Average monthly usage Feb 2011 through May 2012: 1083 unique visitors
Number of hits since launch: Over 700,000
Catalogue Usage
1800
1600
1200
*
1000
800
600
400
200
2012
June
April
March
February
January
Dec
Nov
Oct
Sept
August
July
May
April
March
February
June
2011
May
* As of June 27, 2012
0
January
Unique Visitors
1400
Feedback Cont’d
Average monthly usage Feb 2011 through May 2012: 1083 unique visitors
Number of hits since launch: Over 700,000
AJPM article
Catalogue Usage
1800
in print
online
1600
Launch
Webinar
1200
*
1000
800
600
400
200
2012
June
April
March
February
January
Dec
Nov
Oct
Sept
August
July
May
April
March
February
June
2011
May
* As of June 27, 2012
0
January
Unique Visitors
1400
Users from the Following Countries Have
Accessed the Catalogue
United States
Italy
South Africa
Columbia
Guam
Iceland
Japan
Puerto Rico
Slovak Republic
New Zealand
Czech
Republic
Papua New
Guinea
Canada
Malaysia
Thailand
Bangladesh
Fiji
Mauritius
France
Mariana Islands
Turkey
Qatar
El Salvador
Great Britain
Netherlands
Finland
Uganda
Vietnam
Namibia
Spain
Kuwait
Austria
Macau
Egypt
Germany
Brazil
Russian
Federation
Saudi Arabia
Estonia
Republic of
Serbia
Iran
Israel
Costa Rica
Singapore
Switzerland
Belgium
China
Poland
Hong Kong
Australia
Latvia
Myanmar
Oman
Denmark
Bermuda
Ghana
Taiwan
Lebanon
Belize
Ireland
Hungary
Nigeria
Albania
Saint Kitts &
Nevis Anguilla
Mexico
Ethiopia
Peru
Indonesia
Kenya
Sri Lanka
India
Chile
Portugal
Lithuania
Palestinian
Territories
Malta
Philippines
Greece
Norway
Argentina
Georgia
Guatemala
South Korea
United Arab
Emirates
Botswana
Romania
Virgin Island
Andorra
Ukraine
Jordan
Agenda
•
•
•
•
NCCOR Overview
Measures Registry
Catalogue of Surveillance Systems
Summary and Next Steps
Measures Registry
Immediate Next Steps
• Ongoing updates
 NIH staff and contractors review
new articles
 Nearly 100 measures in process to
be added
• Evaluation project
(August 2012 – March 2013)
 Web analytics
 Testing needs for practitioners
Measures Registry: Next Steps
For Consideration
Key question: How do researchers select the “best” measure?
• Measures Registry facilitates access to measures and allows for
comparison based on key features
 Not developed to identify the “best” measure for each domain
• Challenges in the harmonization/standardization process




Are there existing efforts to standardize measures of childhood obesity?
Measures standardization could help advance field
Positive lessons learned from tobacco control
Clear differences among domains in readiness/potential for harmonization
and/or standardization
Measures Registry: Next Steps
For Consideration
Key question: How do researchers select the “best” measure?
• Different approaches exist to define or move toward consensus:
 NIH PhenX Toolkit: “top-down” approach, experts selected the “gold standard”
measure to recommend for identified key constructs
 NCI Grid-Enabled Measures (GEM): “bottom-up” approach, research community
“ranks” measures online for identified key constructs and provides comments
• Possibilities for guidance and consensus for selection of
measures
 Domain-specific “How-to Manual” to provide general guidance about selecting
measures (e.g., for individual diet assessment: NCI Diet Primer)
 Expert consensus to develop decision-tree system(s) to guide selection of
measure
Catalogue of Surveillance Systems
Immediate Next Steps
• Ongoing updates
 NIH staff, contractors, system
contacts
 10 systems added since launch
 10+ new profiles in development
• Evaluation project
(August 2012 – March 2013)
 Web analytics
 Testing needs for practitioners
Catalogue of Surveillance Systems
For Consideration
Macro/Policy
• Assess adequacy of available
surveillance systems at all levels of
social ecological model (prioritized)
• Adequate links between systems
also needed
Example: Food Stream
Food Available
Foods As-Purchased
Foods As-Eaten
(includes food supply)
(includes
marketplace,
manufacturer, home)
(includes home,
school, other food
outlets)
School & Community
Household
Individual
Catalogue of Surveillance Systems
For Consideration
• General lack of policy surveillance
(vs. policy tracking)
Exceptions
 NIH’s CLASS: school nutrition & physical activity
 CDC’s School Health Policy and Programs Study
 Bridging the Gap State Snack & Soda Tax System
Non-Obesity Policy Surveillance Examples
 National Institute on Alcohol Abuse and
Alcoholism (NIAAA) Alcohol Policy Information
System
 CDC State Tobacco Activities and Tracking
Evaluation System
Source: IOM: Measuring Progress in Obesity Prevention: Workshop Report, 2011
Acknowledgements
Measures Registry
Catalogue of Surveillance Systems
Data Abstraction
Content Experts
NCCOR Working Group
David Berrigan
Mathematica
Mary Arnold
Susan Krebs-Smith
Eileen Hanlon
Patricia Crawford
Amy Auchincloss
Robin McKinnon
Suzanne Le Menestrel
Ann Forsyth
Laura Brennan
Latetia Moore
Patty Freedson
Melissa Cater
Jill Reedy
Lorrene Ritchie
Kelly Clifton
Sameer Siddiqi
Brain Saelens
Stephanie Jilcott
Celeste Torio
Gail Woodward-Lopez
Sharon Kirkpatrick
Jill Reedy
Bill Kohl
Sandy Slater
NCCOR Working Group
Susan Welsh
Site Design
Audrey Lipps
Website Development
Jean Cyr
Bran Handley
Senior Editor
Anne Brown Rodgers
Expert Panel
Alice Ammerman
Lydia Marek
Carol Boushey
Steve Moore
Frank Chaloupka
Kimberly Morland
L. Beth Dixon
Christopher Reinold
Susan Handy
Lisa Shadron
Stephen Hooker
Claire Wang
Leslie Lytle
Jim Sallis
Kristi Sorti
Sameer Siddiqi
System Content Experts
Jamie Chriqui
Deb Galuska
Lisa Powell
Site Design
Audrey Lipps
System Profile Developer
Kathy Huamani
Senior Editor
Anne Brown Rodgers
Website Developers
Jean Cyr
Bran Handley
Questions?
robin.mckinnon@nih.gov
Screen Shots
Locate Surveillance Systems of Interest
Compare Systems
View System Profiles
Access Other Surveillance Resources