Yesterday, I took a little break from editing my literature review (kind of). Part of my previous literature searches have explored research methods involved in looking at how effective training is and how we know training is successful. So as part of my literature review, I am going to discuss this but I decided that I needed to do a little work on this before I began…
To begin the search, I really had to start off quite generalist and do some digging on what research methods are actually out there. Particularly, I needed to find out general information on how training can be researched and measured in terms of its effectiveness. I was able to find quite a few articles on what researchers have done to assess training and how they have gone about designing a study for this purpose. So I have decided to share some of my findings with you today!
First of all, I found out that there are many types of research methods and these can be split into either quantitative or qualitative methods. Now for me, my work will include both, but I wanted to gain an idea of what type of methods were being used in training first of all to help me gain an understanding of how I might go about carrying out this part of my research. During my search, I also found that all research methods have some kind of flaws. No matter what research method you use, there will always be drawbacks to that design… but there will also be advantages to using it too. The one thing that I discovered (with the help of Robert, one of my supervisors) is that I will be using a multi-method approach which takes these weakness into consideration. Now if you are not too sure what that is then I can explain it for you…
Basically, I will be carrying out both quantitative and qualitative research within my study, which is great (numbers and words woo-hoo!). I will be doing smaller studies which can either stand independently or be combined together as a whole to make one larger study (if these are researching the same questions). More importantly, this will allow triangulation of results. That means, if the results of each smaller data collection suggest the same thing about the research questions, then these could be used to increase validation of results and also cross-verify that the results are travelling in the same direction. Now normally, researchers use the multi-method approach to overcome weaknesses in their individual research methods and use others to compensate for these weaknesses and this is technically what I will be doing to justify what methods I use. What is more important is that researchers need to be aware of what these weaknesses are and also what the benefits of different research methods are to help determine which approach and method might be most useful.
So at the moment, some of the research that I have explored includes the following methods:
|Singular case study design||Researchers can gain detailed data in a short length of time which means that it will not take as long to collect.
|Researchers cannot generalise their findings outside of that case study or research setting. This also means that comparison cannot be made between organisations either.
There may be issues with non-compliance. If an organisation refuses to answer questions, are you going to challenge that?
There will also be people who cannot take part, and this could cause a bias in the sample. Who is to say that results from one set of people (who have opted to take part) are not the same or different to those who are not?
Using one source of data can cause measurement error and also method bias if there is no other organisation considered.
|Multiple case study design||You can highlight organisation differences.||You still need to take organisation differences and influences into account when explaining data results.
This also only gives the perspective of the organisations on that certain aspect of research, and no other views.
|Semi-structured interview||Researchers can control the conversation and direct it to help answer the questions presented. This allows the conversation to flow.||These is the likelihood of the participant giving some irrelevant information to what is necessary and often this cannot be controlled.|
|Unstructured interview||This encourages a detailed discussion on a cetin topic so more data can be collected.||Again, irrelevant information might be given and the researcher will be unable to control this for the duration of the conversation.|
|Focus group||It encourages interaction between participants, hopefully forming a verification of stories that they tell.
Themes can be explored collectively and views might be shared amongst participants. They do share some characteristics anyway.
Participants can also question each other on the views expressed, and challenge these opinions if appropriate. Often, this can be done without the need of the researcher.
|You do need to take into account group member demographics (age, gender etc) and external variables that could influence data collection.
These is a chance of group members influencing individuals and also some individuals not taking as much as others.
This method requires a facilitator to maintain control and keep participants on track.
|Questionnaire (quantitative survey method)||It is quite simplistic to administer and collect a lot of data quickly. Some can even be carried out online and the link sent via email or other ways of communication to read more people.
|Using a self-report method means that some people might just lie, or not tell the truth in what they are answering. They may not understand what they are being asked to assess (like the concept) or even what the questions mean.
You may be able to identify a cause-effect relationship but this does not mean that you can explain why the relationship has developed.
|Pre and post intervention assessment||If the assessment uses standardised tools then this can increase validity.
You can see if the training has influenced the participant directly if all other variables are controlled for.
|You need to ensure that any results are a direct consequence of the intervention, and not influences by other variables.
You might need a control group to compare those who took part in training and those who did not.
|Evaluation of intervention throughout|| You can capture processes happening as they occur.
The quality and success of an intervention can be highlighted throughout.
|You need to ensure that the evaluation takes place before, during and after so that it has been measured throughout and can identify changes in evaluation results.|
As you can see, each method has its advantages and disadvantages so it is the duty of the researcher to identify these and design their own methodology to suite the research questions at hand. Methods must be justified and exemplified, highlighting reasons behind the choices made.
I also found out that research methods are determined by the philosophical viewpoint that the researcher adheres to. I have only read one book on this so far, so I can explain a little about it now. But I am sure as I progress in my research I will devote a whole blog post this topic. The topic is called the Philosophy of Science and it aims to explain what qualifies as science and questions the reliability of scientific evidence in determining what research methods might be most appropriate. This in turn determines how ‘scientists’ explore concepts and phenomena.
In a book called ‘Research Methods: the basics’* I read about the four positions that researchers can take in terms of how they are exploring social phenomena. I explain these four below using some direct quotes form the text to help:
- Positivism – ‘scientific investigation is based on acceptance that the world around us is real, and that we can find out about these realities. Knowledge is derived using scientific method and based on sensory experience gained through experiments or comparative analysis. It aims to develop a true description of any chosen aspect of the world regardless of what people think. Knowledge is then built up in scientific fashion. Science builds on what is already known.’ (p.21)
This philosophical position uses measures that are highly organised, measurable and are based on approaches taken form science researching behaviours in the natural world. It can often mean quantitative data being used to explore phenomena.
- Relativism (interpretivism / idealism / constructivism) – ‘the view of the world around us is based on the mind. We can only experience things through our perceptions which then influence our preconceptions, beliefs and values. We are not neutral observers but part of society. The researcher is not observing the phenomena from outside the system, but it is inextricably bound into the human situation that he/she is studying. As well as concentrating on the search for constants in human behaviour which highlights the repetitive, predictable and invariant aspect of society, the researcher does not ignore what is subjective, individual and creative – facts and values cannot be separated. The researcher encounters a world already interpreted and his/her job is to reveal this according to the meanings created by humans rather than discover the universal law. Therefore there can be more than one perspective and interpretation of a phenomenon.’ (p.21-22)
This philosophical position uses measures such as qualitative data, unstructured interviews and participant observations. It aims to explore the many interpretations of the phenomena and is not restricted to quantitative data.
- Postmodernism – ‘challenges key issues such as meanings, knowledge and truth which has opened up new perspectives and ideas about the essence of research. It denounces the meta-narratives (all-embracing theories) of the modern movement as a product of the Enlightenment, and insists on links between knowledge and power. In fact, there is no universal knowledge or truth. Science is just a construct and only one of many types of knowledge that are all subjects of continual renovation and change.’ (p.22)
This philosophical position questions what we understand to be knowledge and suggests that science is made up of knowledge that can constantly change. Therefore, results can change and science can be challenged.
- Critical realism – ‘a reaction to the postmodernist challenge to traditional science which threatens a descent into chaos and powerlessness because of lack of possibility of agreement on truths and reality. It has been labelled Critical Reality based on Critical Reasoning. Critical reasoning recognises that there is natural order in social events and discourse but claims that this order cannot be detected by merely observing a pattern of events. The underlying order must be discovered by interpretation while doing theoretical and practical work in the social sciences. Concepts and theories about social events are developed on the basis of their observable effects and interpreted in a way that they can be acted upon, even if this interpretation gets revised and grows.’ (p.22-23)
Much research is based on this. We hypothesis about phenomena, we the test it and we then interpret this to make sense of what is happening. This is not set and it can be re-tested, which is key aspect of current research in terms of theory development and developing evidence in support of this.
So there you have it! My day of research methods and philosophy was filled with things that I did not know. But now I know. I am now aware that philosophy can influence the decisions of researchers and their methodologies and I also understand I will need to do this on a larger scale for my own research. I also understand that each method has its own advantages and disadvantages, but that researchers often use multiple methods in order to account for the weakness in some and make up of these in advantages of others.
* Walliman, N. (2011). Research Methods the basics. Abingdon, Oxon: Routledge