Goal 8:
Carry
out a formative and summative evaluation of Project activities.
Objective
/ Outcomes
/ Individuals Responsible
/ Time Line
/ Budget /
Weekly Update
Dec. 9, 1999
Draft survey was prepared for Baseline data collection and shared with Team leaders and Faculty development team for feedback and testing for web-based delivery. Feedback was obtained; survey was revised; bugs were worked out for delivery on the web.
Baseline Response Form was distributed to 98 listserve participants on Sunday, Dec. 5; a follow-up reminder was sent to listserve on Thursday, Dec. 9.
Mailing labels
were obtained for members not on the listserve. Email addresses were found
for 14 members and the form was sent to them on email on Dec. 9. Paper
copies were mailed to 29 members without email addresses on Friday, Dec.
10.
Nov. 18, 1999
I attended the faculty training workshop and faculty development team planning sessions in Northampton, MA Nov. 11-13. I collected evaluation data, analyzed, and reported the results for the faculty development workshop. I developed a plan for collection of baseline data for all the goals for the project. The first step in the plan was to identify a web based survey program that could be used for this purpose. The one I have selected is: Perseus Survey Solutions for the Web and can be found at: http://www.perseusdevelopment.com/
I asked the project directors and the faculty development team for reactions to this choice and the plan for data collection. Here is what the Perseus program can do: It allows me to send out a survey as an attachment to email. So, people who can and will open attachments can answer that form and their responses will be sent directly to me in an analyzable data base. It can also be saved to a web site and people who prefer and can, can click on the web address and answer the survey that way and again the results will be sent to me. I requested information from Harold re: putting the survey on ACE-D/HH?s web site as well as information re: how many of ACE-D/HH members do not participate in the listserve and must have a paper copy of the survey sent to them. I will be preparing a draft of the survey itself sometime between today (Th. and Mon.) and will share that for your review and comment.
11/1/99
I began the process
of asking the grant goal leaders to participate in the determination of
need for baseline data for their goals. I asked them to review a survey
that Alan Marvelli has used at Smith to determine needs for training in
technology. I realize that the focus of this questionnaire is on
faculty development and so closely supports the collection of baseline
data for the faculty development goal of the technology grant. However,
as I review the other goals, it seems that baseline data could be useful
for the other goal areas also. I would like to have only one survey sent
out to the ACE-D/HH members that would cover all the goals. Therefore,
I asked that each team leader look at the survey to determine issues
and/or items that could be added to this survey then we could modify it
and send it out over the ACE-D/HH network. I would like to do this be the
end of November if possible. I suggested that the team leaders circulate
this message to their team members and get feedback to me by Monday next
week (Nov. 8).
Specifically,
I asked Karen to determine how we could address the goal of increasing
participation in technology-based training for which we said that we would
determine the extent of participation and then identify barriers for nonparticipants.
How can that be addressed in the survey? For Katharine, we said we'd
have at least 5 syllabi that integrate technology... Is there a need for
any baseline data on this goal? For Barbara, we said we'd have 2-3
case studies...Is there a need for baseline data here? For Alan,
we said we'd see a 25% increase in number of faculty using technology-based
teaching, so we need to know the number that is out there now.
For David, what baseline data would be helpful for the funding goal to
include in grant proposals? For Harold, what kind of recruitment
baseline data would be helpful and is this the right mechanism for getting
that? I will see Karen, Harold, and Alan in MA the end of next week
(Nov. 11-13) and that would be a good time to talk. It would be helpful
to have feedback from the other team leaders at that time for me to bring
into the discussion. DonnaMMertens Project Evaluator
10/07/99
I have made arrangements to attend the team leader planning meeting and workshop the second week of Nov. I have reviewed another possible survey form to collect data base information, one that was used at Smith College. It looks very good. I am thinking we might be able to settle on an instrument During the first day of the planning meetings, then we could pilot test it with the people who attend the first meeting. Alan and I have plans to talk next week to discuss evaluation as an agenda item for the planning meeting and the evaluation of the first workshop. I requested data on goal 1 from the team leaders to document progress in terms of participation and to use as a sampling source when baseline data are collected. I continue to store weekly progress reports for all goals in a searchable data base.
9/30/00
Asked team leaders for feedback on baseline. Am still waiting for response. Agreed to attend meeting second week of Nov. with team leaders and workshop. Will plan to bring copies of possible baseline data collection instruments to meeting for review and discussion. Harold agreed that he could put the survey on the web site if that is the best way to reach respondents. Added data to data base on weekly up-dates for each goal.
9/23/99
I sent the following message to team leaders:
I have been doing some research looking for instruments that others have used to determine the extent to which faculty in teacher ed programs use technology in their teaching. I thought I'd make inquiry as to whether any of you had a favorite survey form that you have used or had used on you in the past. If so, I'd appreciate a copy fax 202 651-5710 (voice 202 651 5202), or email as attachment if you have it in electronic form. I came across an interesting report entit led: Will new teachers be prepared to teach in a digital age? A national survey on information technology in teacher education. The survey was of departments of education and asked the respondents to rate their teacher-training institutions on coursework, faculty capacity and use of technology, facilities, field experience opportunities and graduates' skills. They distributed it via email, HTML, and surface mail. You can view the full report and instruments at the following web site:
www.milkenexchange.org
Or, if you want to see the report in ERIC, the number is: ED428072. It seems to touch on many of the areas of interest present in our own endeavor. I would be curious if you have a chance to look at it and if you have any reactions, feedback you would like to share with me. If there is enthusiasm, I could approach the publisher/author to see if they would let us use some or all of the survey. We could distribute it through the listserve or web site or surface mail...But, I thought I'd let you know what I had found so far. We may want to go with something a bit simpler, but I welcome your thoughts on the matter.
In addition, I set up a system to keep a searchable data base of the weekly up-dates that I can use as evaluation data later in the project. And, I have been in communication with Allan re: logistics for a team leaders meeting in early Nov.
9/15/99
I reviewed the proposal for important evaluation activities. The collection of baseline information on faculty use of technology is the first priority. This activity will be useful for all the other goals in terms of measuring progress during the period of the grant. I have reviewed a number of possible surveys that have been used in other research studies of technology use. I have a few more that I want to look at. I plan to contact the team leaders to see if they have knowledge of other surveys that they thought might be appropriate. I am also wondering if it would be best to try to share these with the team leaders via fax or email or if we will have a team leader meeting where we could discuss this approach and the possible surveys.
9/9/99
This week I spoke with Harold Johnson on the phone to learn about the beginning activities in the project. He sent me the list of goal leaders with contact information. I made email contact with the goal leaders to introduce myself to them and let them know my plan for the next week. My plan is to review the proposal and identify early evaluation activities that need to be undertaken, and to review possible instruments that could be used to establish baseline information re: faculty use of technology in teaching. I will send these possible instruments around to goal leaders to get their input into the appropriateness of the information to be collected.
submitted by Donna M. Mertens
Determine
progress towards each of the seven other project goals, including the determination
of the extent of involvement of D/HH teacher education programs in the
project; document the development and quality of the course syllabi and
case studies; determine the needs of faculty for further equipment and
training; asses the identification of potential funding sources; evaluate
strategies for recruitment into the field; and develop a plan to restructure
D/HH education.
Timely information to aid in decision making throughout the project for all seven goals, with specific reports on numbers participating, reasons for nonparticipation, quality of syllabi and case studies, sensitivity of recruitment strategies for under-served populations, future needs for faculty development, and production of a plan to restructure D/HH teacher education programs in the U.S.
Note: See "Evaluation
Plan" for details
Dr.
D. Mertens.
Throughout
the project, monthly progress reports; milestone reports as needed.
Personnel:
Project Evaluator (PE) -
Donna Mertens
uploaded by Harold Johnson