Funding Opportunity
Standard Grant Announcement: BPPI

Department of Health and Human Services
Substance Abuse and Mental Health Services Administration

Best Practices Planning and Implementation Grants - BPPI 05 PA

Modified Announcement - March 8, 2004

V. APPLICATION REVIEW INFORMATION

1. EVALUATION CRITERIA

Your application will be reviewed and scored according to the quality of your response to the requirements listed below for developing the Project Narrative (Sections A-E for Phase I applications and A-D for Phase II applications). These sections describe what you intend to do with your project.

In developing the Project Narrative section of your application, use these instructions along with any additional instructions found in the NOFA. These are to be used instead of the “Program Narrative” instructions found in the PHS 5161-1.

The Project Narrative may be no longer than 30 pages.

You must use the sections/headings listed below in developing your Project Narrative. Be sure to place the required information in the correct section, or it will not be considered. Your application will be scored according to how well you address the requirements for each section of the Project Narrative.

Reviewers will be looking for evidence of cultural competence in each section of the Project Narrative. Points will be assigned based on how well you address the cultural competence aspects of the evaluation criteria. SAMHSA’s guidelines for cultural competence can be found on the SAMHSA web site at www.samhsa.gov. Click on “Grant Opportunities.”

The Supporting Documentation you provide in Sections F-I and Appendices 1-5 will be considered by reviewers in assessing your response, along with the material in the Project Narrative.

The number of points after each heading is the maximum number of points a review committee may assign to that section of your Project Narrative. Bullet statements in each section do not have points assigned to them. They are provided to invite the attention of applicants and reviewers to important areas within the criterion.

1.1 Phase I Criteria

Section A: Statement of Need (10 points)

Describe the environment (organization, community, city, or State) where the project will be implemented.

Describe the target population (see Glossary) as well as the geographic area to be served, and justify the selection of both. Be sure to check the NOFA for any specific requirements regarding the target population and/or catchment area. Include numbers to be served and demographic information. Discuss the target population’s language, beliefs, norms and values, as well as socioeconomic factors that must be considered in delivering programs to this population.

Describe the problem the project will address. Documentation of the problem may come from local data or trend analyses, State data (e.g., from State Needs Assessments), and/or national data (e.g., from SAMHSA’s National Household Survey on Drug Abuse and Health or from National Center for Health Statistics/Centers for Disease Control reports). For data sources that are not well known, provide sufficient information on how the data were collected so reviewers can assess the reliability and validity of the data.
Non-tribal applicants must show that identified needs are consistent with the priorities of the State or county that has primary responsibility for the service delivery system. Include, in Appendix 5, a copy of the State or County Strategic Plan, a State or county needs assessment, or a letter from the State or county indicating that the proposed project addresses a State- or county-identified priority. Tribal applicants must provide similar documentation relating to tribal priorities.

Describe the best practice selected and how it will impact the problem.
Check the NOFA for any additional requirements.

Section B: Proposed Evidence-Based Practice (30 points)

Clearly state the purpose, goals and objectives of your proposed project. Describe how achievement of the goals will address both the overall program purpose as specified in the NOFA and the needs you have identified in Section A. Provide a logic model (see Glossary) that links need, key components of the proposed project, and goals/objectives/outcomes of the proposed project.
Identify the evidenced based practice that you propose to implement. Describe the evidence-base for the proposed practice and show that it incorporates the best objective information available regarding effectiveness and acceptability. Follow the instructions provided in #1, #2 or #3 below, as appropriate. Depending on the evidence you provide, you may follow more than one set of instructions:
1. If you are proposing to implement a practice included in NREPP (see Appendix C), one of the CMHS tool-kits on evidence-based practices (see Appendix D), the list of Effective Substance Abuse Treatment Practices (see Appendix E), or the NOFA (if applicable), simply identify the practice and state the source from which it was selected. You do not need to provide further evidence of effectiveness.

2. If you are providing evidence that includes scientific studies published in the peer-reviewed literature or other studies that have not been published, describe the extent to which:
- the practice has been evaluated and the quality of the evaluation studies (e.g., whether they are descriptive, quasi-experimental studies, or experimental studies)
- the practice has demonstrated positive outcomes and for what populations the positive outcomes have been demonstrated
- the practice has been documented (e.g., through development of guidelines, tool kits, treatment protocols, and/or manuals) and replicated
- fidelity measures have been developed (e.g., no measures developed, key components identified, or fidelity measures developed)

3. If you are providing evidence based on a formal consensus process involving recognized experts in the field, describe:

the experts involved in developing consensus on the proposed service/practice (e.g., members of an expert panel formally convened by SAMHSA, NIH, the Institute of Medicine or other nationally recognized organization). The consensus must have been developed by a group of experts whose work is recognized and respected by others in the field. Local recognition of an individual as a respected or influential person at the community level is not considered a “recognized expert” for this purpose.
the nature of the consensus that has been reached and the process used to reach consensus
the extent to which the consensus has been documented (e.g., in a consensus panel report, meeting minutes, or an accepted standard practice in the field)
any empirical evidence (whether formally published or not) supporting the effectiveness of the proposed services/practice
the rationale for concluding that further empirical evidence does not exist to support the effectiveness of the proposed services/practice

Justify the use of the proposed practice for the target population. Describe the types of modifications/adaptations that may be necessary to meet the needs of the target population, and describe how you will make a final determination about the adaptations/modifications to be made to meet the needs of the population.

Identify any additional adaptations or modifications that may be necessary to successfully implement the proposed practice in the target community. Describe how you will make a final determination about the adaptations/modifications to be made.

Describe how the proposed project will address issues of age, race, ethnicity, culture, language, sexual orientation, disability, literacy, and gender in the target population, while retaining fidelity to the chosen practice.

Check the NOFA for any additional requirements.

Section C: Proposed Implementation Approach (25 points)

Describe how the proposed grant project will be implemented. Address any and all expectations/required activities specified in the NOFA.
Provide a realistic time line for the project (chart or graph) showing key activities, milestones, and responsible staff. [Note: The timeline should be part of the Project Narrative. It should not be placed in an appendix.]
Describe the strategies or models that will be used to build consensus, including a description of how key stakeholders (see Glossary) will be educated about the best practice. Describe potential barriers to achieving consensus among stakeholders. What resources and plans will you use to overcome these barriers?
Describe the process that will be used to develop a strategic plan to implement the best practice. Address such issues as needs assessment, identification of specific milestones that must be achieved in order to implement the best practice, and plans for assigning responsibility for achieving milestones among participating organizations/stakeholders. Identify potential funding source(s) that will help implement the best practice. Describe how the funder(s) will join in the consensus building and strategic planning.
Describe the key stakeholders (including representatives of the target population), how they were selected for participation in the project, and how they represent the community.
Describe the involvement of key stakeholders in the proposed project, including roles and responsibilities of each stakeholder. Clearly demonstrate each stakeholder’s commitment to the consensus building and strategic planning processes. Attach letters of support and other documents showing stakeholder commitment in Appendix 1: Letters of Support. Identify any cash or in-kind contributions that will be made to the project by the applicant or other partnering organizations.
Describe how the project components will be embedded within the existing service delivery system, including other SAMHSA-funded projects, if applicable.
Check the NOFA for any additional requirements.

Section D: Management Plan and Staffing (20 points)

Discuss the capability and experience of the applicant organization and other participating organizations with similar projects and populations, including experience in providing culturally appropriate/competent services.

Provide a list of staff members who will conduct the project, showing the role of each and their level of effort and qualifications. Include the Project Director and other key personnel, including evaluators and database management personnel.

Provide evidence that the service staff proposed to conduct the evidence-based practice have the level of abilities and experience necessary to implement the practice with fidelity to the model, once they have received any necessary training.

Identify the project staff or contractor(s) who will develop the implementation manual, and demonstrate that they have the requisite skills and experience.

Describe the racial/ethnic characteristics of key staff and indicate if any are members of the target population/community. If the target population is multi-linguistic, indicate if the staffing pattern includes bilingual or bicultural individuals.

If you plan to have an advisory body, describe its composition, roles, and frequency of meetings.

Describe the resources available for the proposed project (e.g., facilities, equipment), and provide evidence that services will be provided in a location that is adequate, accessible, compliant with the Americans with Disabilities Act (ADA), and amenable to the target population.
Check the NOFA for any additional requirements.

Section E: Evaluation Design and Analysis (15 points)

Describe the design for evaluating the consensus building and strategic planning processes. Include a detailed discussion of how all variables (e.g., community representation and stakeholder support) will be defined and measured. Explain how the evaluation plan will ensure that the decision to adopt is an accurate reflection of the stakeholders’ intent.
Document your ability to collect and report on the required performance measures as specified in the NOFA, including data required by SAMHSA to meet GPRA requirements. Specify and justify any additional measures you plan to use for your grant project.
Describe the process for providing regular feedback from evaluation activities to the Project Director and participants.
· Describe plans for data collection, management, analysis, interpretation and reporting. Describe the existing approach to the collection of relevant data, along with any necessary modifications.
Discuss the reliability and validity of evaluation methods and instruments(s) in terms of the gender/age/ culture of the target population.
Check the NOFA for any additional requirements.

1.2 Phase II Criteria

Section A: Need, Justification of Best Practice, and Readiness (30 points)

If you previously received a Phase I BBPI award and are applying for a Phase II award to continue the project, include the following information:

Describe briefly the target population (see Glossary), setting, need and best practice approved for the Phase I award.
Describe and justify any changes to the target population and setting. Discuss the factors that led to a decision change in the target population and setting.
Describe any changes in the need for the best practice in the target community. The statement of need should include a clearly established baseline for the project. Documentation of need may come from a variety of qualitative and quantitative sources. The quantitative data could come from local data or trend analyses, State data (e.g., from State Needs Assessments), and/or national data (e.g., from SAMHSA’s National Household Survey on Drug Abuse and Health or from National Center for Health Statistics/Centers for Disease Control reports). For data sources that are not well known, provide sufficient information on how the data were collected so reviewers can assess the reliability and validity of the data.
Provide an updated projection of the number of individuals to be served as well as demographic information. Discuss the target population’s language, beliefs, norms and values, as well as socioeconomic factors that must be considered in delivering programs to this population.
Describe and justify any additional modifications or adaptations to the best practice as compared to the practice approved for your Phase I project.
Provide evidence that the community of stakeholders (see Glossary) achieved a “decision to adopt” the practice. Attach a copy of the Phase I process evaluation or other evidence including contracts, memoranda of agreement, administrative memos, or other documents signed by key stakeholders that show their firm commitment to support the practice. Attach these supporting documents in Appendix 6: Evidence of Intent to Adopt.
Provide and describe the financing plan. Include anticipated costs and sources of revenue that will maintain the practice. Attach the financing plan, signed by the funding source(s), stating their intent to fund in Appendix 6: Evidence of Intent to Adopt.
Check the NOFA for any additional requirements.

If you are applying for a Phase II award but did not previously receive a Phase I award, include the following information:

Clearly state the purpose, goals and objectives of your proposed project. Describe how achievement of the goals will address the overall program purpose as specified in the NOFA and produce meaningful and relevant results. Provide a logic model (see Glossary) that links need, the services or practice to be implemented, and outcomes.
Describe the target population as well as the geographic area to be served, and justify the selection of both. Include the numbers to be served and demographic information. Discuss the target population’s language, beliefs, norms and values, as well as socioeconomic factors that must be considered in delivering programs to this population.
Describe the nature of the problem and extent of the need for the target population based on data. The statement of need should include a clearly established baseline for the project. Documentation of need may come from a variety of qualitative and quantitative sources. The quantitative data could come from local data or trend analyses, State data (e.g., from State Needs Assessments), and/or national data (e.g., from SAMHSA’s National Household Survey on Drug Abuse and Health or from National Center for Health Statistics/Centers for Disease Control reports). For data sources that are not well known, provide sufficient information on how the data were collected so reviewers can assess the reliability and validity of the data.

Non-tribal applicants must show that identified needs are consistent with priorities of the State or county. Include, in Appendix 5, a copy of the State or County Strategic Plan, a State or county needs assessment, or a letter from the State or county indicating that the proposed project addresses a State- or county-identified priority. Tribal applicants must provide similar documentation relating to tribal priorities.

Identify the evidenced based service/practice that you propose to implement. Describe the evidence-base for the proposed service/practice and show that it incorporates the best objective information available regarding effectiveness and acceptability. Follow the instructions provided in #1, #2 or #3 below, as appropriate:

1. If you are proposing to implement a service/practice included in NREPP (see Appendix C), one of the CMHS tool-kits on evidence-based practices (see Appendix D), the list of Effective Substance Abuse Treatment Practices (see Appendix E), or the NOFA (if applicable), simply identify the practice and state the source from which it was selected. You do not need to provide further evidence of effectiveness.

2. If you are providing evidence that includes scientific studies published in the peer-reviewed literature or other studies that have not been published, describe the extent to which:
- the service/practice has been evaluated and the quality of the evaluation studies (e.g., whether they are descriptive, quasi-experimental studies, or experimental studies)
- the service/practice has demonstrated positive outcomes and for what populations the positive outcomes have been demonstrated
- the service/practice has been documented (e.g., through development of guidelines, tool kits, treatment protocols, and/or manuals) and replicated
- fidelity measures have been developed (e.g., no measures developed, key components identified, or fidelity measures developed

3. If you are providing evidence based on a formal consensus process involving recognized experts in the field, describe:

the experts involved in developing consensus on the proposed service/practice (e.g., members of an expert panel formally convened by SAMHSA, NIH, the Institute of Medicine or other nationally recognized organization). The consensus must have been developed by a group of experts whose work is recognized and respected by others in the field. Local recognition of an individual as a respected or influential person at the community level is not considered a “recognized expert” for this purpose.
the nature of the consensus that has been reached and the process used to reach consensus
the extent to which the consensus has been documented (e.g., in a consensus panel report, meeting minutes, or an accepted standard practice in the field)
any empirical evidence (whether formally published or not) supporting the effectiveness of the proposed services/practice
the rationale for concluding that further empirical evidence does not exist to support the effectiveness of the proposed services/practice

Justify the use of the proposed service/practice for the target population. Describe and justify any adaptations necessary to meet the needs of the target population, as well as evidence that such adaptations will be effective for the target population.

Identify and justify any additional adaptations or modifications to the proposed service/practice.

Describe the community of stakeholders in the project, and provide evidence that they have achieved a “decision to adopt” the practice. Such evidence may include contracts, memoranda of agreement, administrative memos, or other documents signed by key stakeholders that show their firm commitment to support the practice. Attach these supporting documents in Appendix 6: Evidence of Intent to Adopt.
Provide and describe the financing plan. Include anticipated costs and sources of revenue that will maintain the practice. Attach the financing plan, signed by the funding source(s), stating their intent to fund in Appendix 6: Evidence of Intent to Adopt.
Check the NOFA for any additional requirements.

Section B: Proposed Approach (25 points)

Provide a strategic plan, including key action steps, that addresses each of the following elements, as appropriate: pilot testing the best practice, evaluating the pilot test, modifying the best practice based on the pilot test, developing training materials, hiring/training staff, and securing funding to sustain services beyond the project period. Address any and all expectations/required activities specified in the NOFA.
Describe the involvement of key stakeholders in the proposed project, including roles and responsibilities of each stakeholder. Demonstrate each stakeholder’s commitment to the proposed project. Attach letters of support and similar documents showing stakeholder commitment in Appendix 1: Letters of Support. Identify any cash or in-kind contributions that will be made to the project.

Describe how the proposed project will address issues of age, race/ethnicity, culture, language, sexual orientation, disability, literacy, and gender in the target population.

Describe potential barriers to the successful conduct of the proposed project and how you will overcome them.
Describe oversight or feedback mechanisms to ensure that the implemented practice is consistent with the best practice model.
Check the NOFA for any additional requirements.

Section C: Management Plan and Staffing (25 points)

Provide a realistic time line for the project (chart or graph) showing key activities, milestones, and responsible staff. [Note: The time line should be part of the Project Narrative. It should not be placed in an appendix.]

Discuss the capability and experience of the applicant organization and other participating organizations with similar projects and populations, including experience in providing culturally appropriate/competent services.

Provide a list of staff members who will conduct the project, showing the role of each and their level of effort and qualifications. Include the Project Director and other key personnel, including evaluators and database managers.

Describe the racial/ethnic characteristics of key staff and indicate if any are members of the target population/community. If the target population is multi-linguistic, indicate if the staffing pattern includes bilingual and bicultural individuals.

Describe the resources available for the proposed project (e.g., facilities, equipment), and provide evidence that services will be provided in a location that is adequate, accessible, Americans with Disabilities Act (ADA) compliant, and is amenable to the target population.

Check the NOFA for any additional requirements.

Section D: Evaluation Design and Analysis (20 points)

Document your ability to collect and report on the required performance measures as specified in the NOFA, including data required by SAMHSA to meet GPRA requirements. Specify and justify any additional measures you plan to use for your grant project.
Provide a logic model (see Glossary) for the evaluation of the pilot test of the best practice as well as other implementation activities (e.g., training, securing financing).
Provide a plan for evaluating the pilot test of the best practice and other implementation activities that includes both process and client outcome measures. Describe the recruitment plan and sample size for your project. Describe any literature or pilot testing done to verify the validity and reliability of the instruments to be used. Also discuss the appropriateness of the evaluation methods and instrument(s) in terms of the gender/age/culture of the target population. Attach instrumentation in Appendix 2: Data Collection Instruments.
Describe how the adaptations of the best practice will be documented. Demonstrate its fidelity to the best practice model. If no fidelity scale exists for the practice, describe how you will develop one.
Describe the process for providing regular feedback from evaluation activities to the Project Director and participants.
Describe the database management system that will be developed.
Check the NOFA for any additional requirements.

NOTE: Although the budget for the proposed project is not a review criterion, the Review Group will be asked to comment on the appropriateness of the budget after the merits of the application have been considered.

2. REVIEW AND SELECTION PROCESS

SAMHSA applications are peer-reviewed according to the review criteria listed above. For those programs where the individual award is over $100,000, applications must also be reviewed by the appropriate National Advisory Council.

Decisions to fund a grant are based on:

the strengths and weaknesses of the application as identified by peer reviewers and, when appropriate, approved by the appropriate National Advisory Council;

availability of funds;

equitable distribution of awards in terms of geography (including urban, rural and remote settings) and balance among target populations and program size; and

after applying the aforementioned criteria, the following method for breaking ties: When funds are not available to fund all applications with identical scores, SAMHSA will make award decisions based on the application(s) that received the greatest number of points by peer reviewers on the evaluation criterion in Section V-1 with the highest number of possible points (for Phase I, Proposed Evidence-Based Practice—30 points; for Phase II, Need, Justification of Best Practice, and Readiness—30 points). Should a tie still exist, the evaluation criterion with the next highest possible point value will be used, continuing sequentially to the evaluation criterion with the lowest possible point value, should that be necessary to break all ties. If an evaluation criterion to be used for this purpose has the same number of possible points as another evaluation criterion, the criterion listed first in Section V-1 will be used first.

Funding Opportunity