Quality assurance ??? responding to a changing information world

I’ve been asked to speak at a Research Information Network meeting tomorrow night on the subject of “Quality assurance – responding to a changing information world”. I’m part of a panel so I’ll only get about 10 minutes – but the following is the sort of thing that I thought I’d say. As ever comments and thoughts are very welcome. If I can crowd source some brilliance it would be very much appreciated.

 

….

 

Quality is a difficult concept to work with. The meaning of the word has changed and it is claimed for various purposes. So I’m going to rephrase the question by asking “how can we know where valuable information resides?” As researchers our job is to find, synthesis and analyse information and so there is a common sense rationale that what we come up with is likely to be strongly influenced by the information that we base it on. If this information is rubbish, we are likely to be unable to generate anything useful from it.

The academic community has generally had a pretty straightforward answer to this. Research information is something that should be produced, quality assured and consumed largely by a professional class of researchers. Quality assurance, through the peer review system and other similar mechanisms, was therefore designed to ensure that professional researchers had their work judged by other professional researchers and that the results of the process were useful to this same group of people. This system has been enormously successful and the intention of this talk is not necessarily to call these basic systems of academic quality assurance into question.

 

However, we live in a culture in which people are increasingly uncomfortable with decisions that are made by a professional class behind closed doors. Because we’re in the Royal College of Physicians I’ll try and draw on some medical examples today. The days when a doctor could tell you “take two of these and you’ll be alright in the morning” are long gone. Doctors are increasingly required to explain their decisions, to educate their patients and to generally make their decisions transparent and open. There are clearly both some positive and negative aspects to these kinds of challenges to authority. On the positive side it can represent a democratic check on what George Bernard Shaw described as a “conspiracy against the laity” on the negative side it can lead people into the mistaken belief that professionals know nothing and they can pick and choose from their advice, perhaps combining it with the advice of those who have little knowledge or expertise.

 

In general, I’d favour an increased democratisation of information, not least because I think that this has the potential to engage people in making discovering more about the world around them and making critical judgements about it (AKA education). As a researcher this means that I’m interested ways to open up information to the broadest possible number of people. It also means that I’m interested in opening up and increasing the transparency of decisions about what information is valuable and in recognising that everyone might consider this value in a different way.

 

If we think of the mass of information that exists around an academic subject one of the biggest problems is dealing with the information overload. We can’t read everything. In fact we can’t even read everything that has been judged as being valuable by conventional processes of academic peer review. Our job as consumers of information is not just about quality assurance but rather about a process of filtering and streamlining. In an information rich environment this process of filtering is at least as important as quality assurance. However filtering is not something that can be done centrally by a professional class, how we filter is much more personal and represents a complex interaction between our interests, our judgement, our ability in resource discovery and so on.

 

If we think about all of the printed and electronic information that might exist on a particular type of cancer we are likely to see how information rich our world is. A search for “pancreatic cancer” produces About 7,880,000 results (0.16 seconds) on Google. It produces 334,000. (0.29 sec) on Google Scholar. This is simply information overload. Whether these resources are good, bad or indifferent is no longer the question. The question is how to filter and to ascertain where most value might be. Furthermore the answer to this question will be different depending on whether you are a research scientist, a doctor, a nurse or a patient (all people who might have legitimate reason to consult research based information about pancreatic cancer).

 

I haven’t been invited here today to discuss pancreatic cancer. I’d guess that I’m here because I’m a fairly active user of social media and I’ve been doing some research looking at how other researchers are using social media in their practice as researchers. I’m therefore going to suggest three social media-based processes that researchers use to identify where value lies in the information overload. In and of themselves none of these challenge the existing academic quality assurance process. However taken together I think that they suggest a possible trajectory in which the quality assurance of research becomes more democratic.

 

Trust in your networks:  Research has always been based around networks. The whole practice of citation binds papers together in ways that make academic networks obvious to even the most casual observer. Citation has also always fulfilled the function of social filtering e.g. if three articles all cite the same seminal work, you had better read it. However the development of social media tools has enhanced this process of filtering and identification of value. The people whose opinion I value provide me with constant updates about what they are reading and looking at
through a range of technologies (social citation, social bookmarking, blogging, microblogging etc). Collectively we are identifying value and enabling the individual to draw on that information when making decisions about what to read or how to read it. Furthermore our networks have the potential to challenge us by alerting us to new areas that other are drawing information from. This is where the development of a social approach to resource discovery has the potential to be far more powerful than a “search” based approach. The development and management of a network therefore becomes a central element of the professional practice of a researcher.

 

Moving from taxonomy to folksonomy: The research world is conventionally divided up in a taxonomical way. Our interests are divided, sub-divided and sub-divided again until we end up with an area of interest that is small and contained enough for us to be able to exercise some mastery over it. However as the social media theorist David Weinberger says, “everything is miscellaneous” and it therefore does not always conform to our structured taxonomies. These taxonomies made sense when we had to maintain a card indexes and keep books alphabetically on shelves, however once we are in a flexible, digital world everything can be labelled with as many labels as it needs to describe it. What is more everyone can have different sets of labels. What is more we can also look at what the most popular set of labels is to help us understanding how other people see a particular area. So we might ask what labels do doctors use to talk about pancreatic cancer and how does this differ from nurses or research scientists and so on.?

This discussion about metadata (labels) is important because it recognises that academic value does not just reside in the single piece of research itself. It resides in the linked communities and disciplines. Our ability to judge value is relative and related to a system of thinking. It is only when we can access what that system of thinking is that we can really understand and benefit from the value of a particular piece of information. However the idea of folksonomy makes these concepts more flexible and enables us to reuse and repurpose information so it has value to many different people.

 

Participatory co-creation: Finally I’d like to suggest that social media enables us to go beyond just judging value or quality assuring. Social media creates conversations around ideas and research information. These conversations are like the conversations that people have at academic conferences, but they go on for as long as participants find them interesting and they enable as many people as are interested to come together regardless of geography. This process of discussion and debate is essentially a process of participatory co-creation through which the research community and others can add value to information and ideas. In other words the process of participation has the potential to enhance quality and speed up constructive feedback. Conventional quality assurance sets a bar over which people must jump, participatory co-creation helps produce work that is able to move over this bar. It is also the case that it provides a forum for the discussion of the nature of the bar and the organisation of challenges to it.

 

In summary

Much of what I’ve talked about is enabled by social media, but is not dependent upon it. If there is a change in the way the research community finds, filters and quality assures information it will not happen because of the invention of a new machine. Rather, what I have described are working practices that I would argue tap into the essence of what research and intellectual enquiry are about. The ability to draw on the wisdom of others, to move between collective and individual interpretations the world and the ability to work together to enhance what others have done, these are the best traditions of research. My argument is that social media facilitates them and makes them more likely. However, at the moment these practices are still confined to a minority. How they develop will depend on the attitudes of the research community and those with a stake in it.

 

Advertisement