Appendix 4

Sample REU Program Evaluation Questions

Many evaluation items can be posed in a form where the individual respondent answers with a numerical rating suitable for statistical treatment. According to individual program needs and the nature of the evaluation questions, rating scales might be presented as satisfaction, agreement, quality, or frequency:

Rating Score	Satisfaction Scale	Agreement Scale	Quality Scale	Frequency Scale
1	Very Satisfied	Strongly Agree	Excellent	Always
2	Satisfied	Agree	Very Good	Frequently
3	Neutral	Neutral	Average	Sometimes
4	Slightly Dissatisfied	Disagree	Below Average	Rarely
5	Dissatisfied	Strongly Disagree	Poor	Never

Most of the items listed on the sample questionnaires use a satisfaction scale to measure the participants’ perceptions of the effectiveness or usefulness of a particular program or component. The first section on both the student and host scholar/mentor questionnaires solicits ratings of some general or overall program aspects. The remainder of the student questionnaire follows in arrangement the four dimensions of the MAP framework for program evaluation: 1) Student Learning Environment, 2) Student Learning Assessment, 3) Resources for Academic and Student Support, and 4) Program Administration and Development (Institute for International Education of Students [undated]). Finally, there are some open-ended questions on both questionnaires, soliciting qualitative data to assess program effectiveness.


Sample Student Questionnaire

Organizational Quality and Overall Experience		Strongly Agree	Agree	Neutral	Disagree	Strongly Disagree
1	This international REU program was well organized as a whole.	1	2	3	4	5
2	The international component(s) of the program added value to the underlying scientific experience.	1	2	3	4	5
3	The foreign institutional site was well suited to host this program.	1	2	3	4	5
4	This program helped me to improve my research skills.	1	2	3	4	5
5	This program helped me to increase my knowledge in a specific research field.	1	2	3	4	5
6	This program helped me to increase my general scientific knowledge.	1	2	3	4	5
7	I will continue my pursuit of a career in science or engineering as a result of this International REU experience.	1	2	3	4	5
8	I would recommend this program to others.	1	2	3	4	5
9	I would recommend my foreign REU advisor for future projects.	1	2	3	4	5
I. STUDENT LEARNING ENVIRONMENT
Satisfaction With Pre-Trip and On-Site Orientations
How satisfied were you with:		Very Satisfied				Dis-satisfied
1	The information presented during the pre-trip orientation meeting?	1	2	3	4	5
2	The orientation materials sent by mail to prepare you for the program?	1	2	3	4	5
3	The information in the on-line (Web-based) handbook?	1	2	3	4	5
4	The print version of the program handbook?	1	2	3	4	5
5	The information presented during the on-site orientation(s)?	1	2	3	4	5
Satisfaction With Reentry Orientation
How satisfied were you with:		Very Satisfied				Dis-satisfied
1	The information presented after return to the U.S. for help in reintegration into home academic environment?	1	2	3	4	5
2	The information presented about continuing your academic studies in graduate school?	1	2	3	4	5
Quality and Value of Scientific Program
How satisfied were you with:		Very Satisfied				Dis-satisfied
1	The laboratory tours at the host institute?	1	2	3	4	5
2	The lunchtime chalk talks by program mentors?	1	2	3	4	5
3	The weekly seminar presentations by visiting researchers?	1	2	3	4	5
4	The feedback from program mentors on your interim data presentation(s)?	1	2	3	4	5
5	The end-of-program poster project?	1	2	3	4	5
Quality and Value of Cultural Program
How satisfied were you with:		Very Satisfied				Dis-satisfied
1	The “survival” language training classes?	1	2	3	4	5
2	The historical tour of the host city?	1	2	3	4	5
3	The cultural outings and field trips sponsored by the host institutions?	1	2	3	4	5
4	The home stay program?	1	2	3	4	5
II. STUDENT LEARNING ASSESSMENT*
Value of Scientific Research Experience
How satisfied were you with:		Very Satisfied				Dis-satisfied
1	Your individual research project?	1	2	3	4	5
2	The value of your project to your academic career?	1	2	3	4	5
3	The value of your project to your future graduate school or professional career?	1	2	3	4	5
4	The time available for you to complete your individual project?	1	2	3	4	5
5	Your overall academic background and readiness to participate in the program?	1	2	3	4	5
6	Your specific preparation (including recommended coursework) for your overseas research experience?	1	2	3	4	5
Communication Skills
How satisfied were you with:		Very Satisfied				Dis-satisfied
1	The program in preparing you to deliver oral presentation about your research work?	1	2	3	4	5
2	The program in preparing you to conduct scientific research in an international context?	1	2	3	4	5
3	The program in preparing you to work as part of an international research team?	1	2	3	4	5
III. RESOURCES FOR ACADEMIC AND STUDENT SUPPORT
Quality of Individual Research Guidance and Mentorship
How satisfied were you with:		Very Satisfied				Dis-satisfied
1	The guidance provided by your foreign research advisor during your project?	1	2	3	4	5
2	The availability of your foreign advisor when you needed him/her?	1	2	3	4	5
3	Your acceptance into the lab group as a contributing member?	1	2	3	4	5
4	The assistance/advice provided by your foreign (under)graduate student “colleague” in routine matters?	1	2	3	4	5
5	The assistance provided by the host lab in preparing your poster presentation?	1	2	3	4	5
Quality of Research and/or Laboratory Facilities
How satisfied were you with:		Very Satisfied				Dis-satisfied
1	The research facilities at the host institute?	1	2	3	4	5
2	The availability of research facilities to you for your individual research project?	1	2	3	4	5
3	The availability of computer support?	1	2	3	4	5
4	The availability and convenience of access to e-mail and World Wide Web services at the host institute/laboratory?	1	2	3	4	5
5	The availability of adequate library facilities at the host institute/laboratory?	1	2	3	4	5
Quality of Support Services
How satisfied were you with:		Very Satisfied				Dis-satisfied
1	The timeliness of the delivery of your airline ticket?	1	2	3	4	5
2	The timeliness of the disbursement of your participant allowance?	1	2	3	4	5
3	The location of participant housing?	1	2	3	4	5
4	The cleanliness of housing accommodations?	1	2	3	4	5
5	The safety of housing accommodations?	1	2	3	4	5
6	The dining hall food?	1	2	3	4	5
7	The convenience/location of laundry facilities?	1	2	3	4	5
IV. PROGRAM ADMINISTRATION
Administration and Helpfulness of Program Administrators
How satisfied were you with:		Very Satisfied				Dis-satisfied
1	The accuracy of the advertising about the International REU site program?	1	2	3	4	5
1	The information provided by the program Web site?	1	2	3	4	5
2	The application form on the program Web site?	1	2	3	4	5
3	The promptness of responses to your e-mail inquiries?	1	2	3	4	5
4	The appropriateness of your assigned/selected project to your interests?	1	2	3	4	5
5	The appropriateness of your assigned/selected host scholar to your project and interests?	1	2	3	4	5
6	The helpfulness of on-site program staff during the foreign program?	1	2	3	4	5
It is important to have objective measures for assessing student learning outcomes. Here are some suggestions:
Use a pre-/post-program problem-solving test. Present each student with a problem prior to participation in the REU program, and present the same problem after participation in the program. Real-life case studies and problems built on scenarios of various sorts work particularly well since students have the opportunity to apply their knowledge and skills freely and in various ways to address a given problem at several possible conceptual and intellectual levels. Devise a laboratory exercise where students design, execute and/or analyze an experimental solution to a research problem in their field of interest.
Open-Ended/Freeform Questions
1. How did you first hear about the International REU Program?
2. Did you download the application form(s) from the program Web site? YES NO
3. How has your participation in this REU program affected your future career plans or interests?
4. How has your participation in this REU program helped you academically?
5. Are there any specific courses or other projects you are planning in the near future as a result of your REU project?
6. Approximately how many hours per week did you spend with your foreign advisor?
7. Are there courses or experiences that you wish you had before starting this project?
8. Did you consider other REU programs in addition to this one? YES NO
If YES:
What were the important factors in your decision to participate in this program?
What (approximately) was the typical notification date for the other program(s) to which you applied?
What was the value of the stipend in the other program(s)?
9. Would you recommend this REU program to other students? YES NO
Sample Host Scholar/Mentor Questionnaire
Organizational Quality and Overall Experience
How satisfied were you with:		Very Satisfied				Dis-satisfied
1	The organization of this international REU program as a whole?	1	2	3	4	5
2	The expected level of commitment of your time and resources to the program?	1	2	3	4	5
3	Your ability to accommodate the needs of the program within your schedule?	1	2	3	4	5
4	The fulfillment of your expectations for the program?	1	2	3	4	5
Quality of Undergraduate Student Participants and of Student-Host Match
How satisfied were you with:		Very Satisfied				Dis-satisfied
1	The academic qualifications of the student participant assigned to your laboratory?	1	2	3	4	5
2	The work ethic and productivity of the student assigned to your laboratory?	1	2	3	4	5
3	The quality of the student’s research results?	1	2	3	4	5
4	The integration of the student visitor into your laboratory group?	1	2	3	4	5
5	The time allotted for the student to complete his/her project?	1	2	3	4	5
6	The appropriateness of the student’s research interests to yours?	1	2	3	4	5
Administration and Helpfulness of Program Administrators
How satisfied were you with:		Very Satisfied				Dis-satisfied
1	Your role in the selection of a student researcher to work in your laboratory?	1	2	3	4	5
2	The program’s overall management of on-site logistics?	1	2	3	4	5
3	The promptness of responses to your inquiries about the REU program?	1	2	3	4	5
4	The program’s management of student housing and allowance disbursement without intervention on your part?	1	2	3	4	5
Usefulness of Host Scholar/Mentor Orientation(s)
How satisfied were you with:		Very Satisfied				Dis-satisfied
1	The pre-arrival orientation meeting(s) for host scholars/mentors?	1	2	3	4	5
2	The information provided to you about the REU program calendar and activities?	1	2	3	4	5
3	The explanation of your responsibilities as a host scholar/mentor, and the resource materials provided to you?	1	2	3	4	5
4	The on-site orientation and your role in the program’s opening ceremonies?	1	2	3	4	5
Open-Ended/Freeform Questions
1. How has your participation in this International REU program benefited you and/or your laboratory?
2. Approximately how many hours per week did you spend with your assigned student participant?
3. Are there courses or experiences that you wish your assigned student had before starting his/her project?
4. Would you participate again as a host scholar/mentor? YES NO
5. Would you recommend this program to your colleagues? YES NO

Appendix 5

---

Last updated July, 2002