Defining criteria for measuring the impact of discipline specific information skills training for PhD students

Most library trainers at our institution circulate session feedback forms to ascertain the value that participants feel they have derived from their sessions.  This tends to be done immediately as the session ends so that forms are returned and not forgotten about.  However, Markless and Streatfield contend that these reactionnaires are of limited value because they are administered too soon after an event to be able to ascertain whether any actual learning or growth in confidence has occurred (2006, p.145).  I aim to explore less obtrusive and more quantitative alternatives to traditional focus group and reactionnaire methods of collecting feedback on the effectiveness of library instruction.Similarly, the use of a pre-course self-assessment questionnaire to ascertain the level at which content should be pitched may not be a valid indicator of skill level -  recent research by Patterson (2009) has indicated that self assessment in personal development plans often leads to over-rating of skills and thus may not always produce valid responses from those surveyed.   This will be taken into consideration when developing impact measures for library instruction as pre- and post-course feedback may not be true measures of the effect of instructional intervention.From existing knowledge of the area, in particular what has been written by Markless and Streatfield (2006; 2008) and the published report of the UK LIRG SCONUL Impact Implementation Initiative (Conyers Payne, 2005), it is clear that it will be necessary to develop quite specific and focused success criteria.    But it is hoped that if valid measures of impact and value are developed that they can be practically employed in this instance and then adapted for other instructional programmes.This paper will report on the progress being made in developing criteria to measure the impact of a series of information skills workshops aimed at PhD students and staff researchers.  Piloting of the criteria is due to commence in January 2010 so initial findings will also be reported.