Basis Upon Which Winner Will Be Selected

The judging panel will make recommendations to the award approving official based upon the following 8 criteria. Each criterion will be scored with the maximum of 5 points.

1) Quality of the research agenda (0-5 points): How well is the research design or conceptual framework developed? Is it unique and clinically meaningful? Does the research agenda describe a logical, feasible plan and timeframe for addressing addiction knowledge gaps?

2) Proposed ResearchKit modules (0-5 points): How many existing features of the ResearchKit does the app use? How are the modules applicable for conducting future addiction research? Does the Contestant consider creating new modules?

3) Add-apters (0-5 points): Does the app utilize novel add-apters? How are the proposed add-apters applicable to the future research study?

4) Compliance with applicable legal policies (0-5 points): Although the competition requires that the submission must not contain any research data about real individuals, and the Contestants must not use real data in the development of the app, the submission will be evaluated on whether the app design and research agenda would be compliant with all applicable federal, state, local, and institutional laws, regulations, and policies. These include, but are not limited to, Substance Abuse Confidentiality Regulations at 42 CFR Part 2, HIPAA protections, HHS Protection of Human Subjects regulations at 45 CFR Part 46, and FDA regulations. Would the app ensure compliance with consent requirements for future potential addiction studies? Would the app clearly explain study participation to the user? Would data management be safe and secure?

5) Study participant’s engagement (0-5 points): How well would the app attract and retain human subject engagement? Does it assure the high level of human subject participation?

6) Durability of study participation (0-5 points): How reasonable is the plan for retaining human subjects and data collection over the duration of the future, proposed research study?

7) Clarity of the app context (0-5 points): Will the app provide a transparent, engaging user experience for both addiction researchers and human subjects? Would the future human subjects of research be able to easily track their overall progress during the research study? Would future human subjects of research know what information is being collected, why, and what will happen with their data?

8) Data quality for researchers (0-5 points): Is it easy for addiction researchers to monitor and manage the overall progression of the research study? Is the data management plan appropriate? Are data clearly presented to the researcher?

The evaluation process will begin by anonymizing and removing those that are not responsive to this Challenge or not in compliance with all rules of participation eligibility. Submissions that are responsive and in compliance will next undergo a review by federal employees with expertise in the relevant areas of science and executive scientific advisors. A panel of judges consisting of federal employees will then score responsive and compliant submissions entries in accordance with the judging criteria outlined above. Final recommendations will be determined by a vote of the judges based on score. Scores from each criterion will be weighted equally, but failure to meet a minimum standard for any one criterion might disqualify an application. The score for each submission will be the sum of the scores from each of the 5 voting judges, for a maximum of 200 points.

For official information, please see the Federal Register Notice.