Posted by: Ali | December 17, 2009

Maxim #9 Using impact analysis studies – self reflection

a) list the data collection methods that are used in your workplace

1.  feedback slips for students to give comments

2. – Insync survey – major customers satisfaction survey for staff and students that is run every few years (is same survey instrument used by other Australasian universities, which enables comparisons to be made, but probably reduces the flexibility as to the questions we can ask)

3. Information from students obtained as a part of surveys on specific topics e.g. subject guides – usually as part of a wider project

b) do you think that the data collection methods that are used in your library adequately assess impact? Remember that impact assessment is the evaluation of the effectiveness of outcomes, rather than the efficiency of process in information services

– current methods are better at problem identification and/or establishing levels of satisfaction – so efficiency rather than effectiveness)

(efficiency =  outputs / imputs – so the outputs the library achieves for the inputs put in e.g. human resources, materials)

(effectiveness = outputs/inputs – the greater the outcomes (useful actions by the user) in relation to the same inputs, the more effective the service is

1. Feedback slips – generally the library user is asking a specific question related to some aspect of the service – very often it is about the physical environment or physical resources (water cooler, PC availability, power outlets). So if we look at impact in terms of “effect of service” then we can make some inferences – i.e. lack of power outlets is having an impact on students ability to use library spaces effectively, but because we don’t specifically ask for direct impacts, then we are missing out on that actual data

2. This survey (which is run every few years) asks the respondents to rate each of 29 statements (considered critical to the success of the library) – firstly to measure the importance of each of the statements and secondly to measure their impression of the library’s performance e.g. books and articles requested are delivered promptly; library staff are approachable and helpful. The words “value” and “impact” aren’t used. The satisfaction survey gathers quantitative data (that can be compared with other institutions) and a large number of verbatim comments are also collected. So we get an idea of how satisfied students are with the library’s performance but there isn’t really any attempt to get an idea of value or impact – some of this does come through in verbatim comments, but again this data isn’t gathered specifically

3. Similar comments to those in #2

c) how could the assessment of impact in your library or information service be improved?

– ask those filling out feedback slips to detail what the impact of their interaction with the library service really means to them

– run some pre and post training impact assessment for selected training courses

– pre and post training impact assessment for selected new services

– mine the verbatim comments from the Insync survey to look for specific comments about impact

– incorporate some questions into the Insync survey that actually do focus specifically on impact (this may be difficult as it is a survey organised for several institutions)


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: