Critically Reviewing the Literature
(S. Bevins, 2000)

 

Why is critically reviewing the literature important?  It is important to know how to review literature so that you may become an educated research consumer. If one is not an educated research consumer, he/she cannot distinguish research that is well done from poorly conceived and implemented research.  Being able to deconstruct a research article allows the reader to make his/her own judgments about the researcher's decision at any point in the research process.

Whether one is reading and critiquing quantitative or qualitative research, there are basic questions that must be asked.  For your assignment, critique one of the assigned articles by answering each of the following questions:

1.  Was the study clear, unambiguous, and internally consistent?  Think of internal consistency as the degree to which everything fits together in the study.  It wouldn't make sense to have a qualitative research question and then use a quantitative data collection method.
2.  What is (are) the research questions?  Are they clearly and adequately stated?
3.  What is the purpose of the study?  Sometimes, the author explicitly tells you.  Other times, you have to pull it out of the literature review and rationale for the study.
4.  How does the purpose influence the design and the conclusions?  If your purpose is to describe phenomena related to a particular circumstance, the design of your study must allow you to fulfill this purpose.  The conclusions you reach must be related back to what your purpose was.  You wouldn't be able to draw conclusions about the phenomena if you did not choose a design that allowed you to see the phenomena.
5.  Describe the theory that guides the study and the conceptual framework for the project.  Are they clearly presented and relevant to the study?  Think of conceptual framework as the theoretical framework.  This framework is what organizes what you know about a subject and helps you fit your ideas together.  Instead of just having random bits of unconnected information, a framework provides a structure.  Let's say that I believe that children develop skills through some interconnected process.  I believe children's development of skills in one developmental domain impacts on skills in another.  For example, if I am interested in looking at the relationship between children developing a language concept and acquiring a new gross motor skill, how I ask my question, what design I use, etc. depends on this conceptual or theoretical framework. Let's say that another researcher believes that development proceeds in parallel tracts.  In other words, what happens in the language domain does not effect what happens in the motor domain and vice versa.  Because of this different conceptual or theoretical framework, the entire research study may be different.
6.  What are the key constructs identified in the literature review? Remember your books discussion about the difference between concepts and constructs.  A concept is the first level of abstraction and is a symbolic representation of an observable or experienced referent.  the example used in the book is "dog".  You can see this referent and know this is a dog.  however, if the dog comes running up to a child and scares the child, you cannot observe fear.  You can see the behaviors associated with it such as crying, shaking, turning pale, sweating, etc.  These are observed behaviors of the construct "fear".
7.  What are the dependent and independent variables in the quantitative research study? Although not an exact parallel, what are the general areas of interest in the qualitative research study?
8.  What level of theory is suggested in the literature review?  Is it consistent with the selected research strategy? For the purposes of this assignment, think of this as whether or not the researcher has developed a theory in depth 'a priori'.  This is what should happen in quantitative research. If the researcher does not do this in quantitative research, this would be an example of poor research.  In qualitative research, if the researcher lays down some of the groundwork without a full fledged theory, this would be appropriate.  If the researcher says she is conducting a qualitative study but then sets out to prove a theory, this is not consistent with the research strategy.  Remember that quantitative research goes from general to specific in level of theory and qualitative research goes from specific to general.
9.  What is the rationale for the design found in the literature review?  (This means , "Does the author make his case?")  Is it sound?
10.  Does the design of the project fit the level of theory?  Is relevant knowledge presented in the literature review?
11.  For quantitative studies, diagram the design (remember the Xs, Os, and r).  For qualitative studies, describe the design.
12.  Does the design answer the research question?  This means, "Did the author choose a design that allows him to answer the question he posed"?
13.  What are the boundaries of the study?  How are the boundaries selected?  Think of this from the perspective of sampling.  You'll deal with ethical issues in a later question.  From the perspective of a quantitative study, how was the sample chosen? What specific sampling strategy was used? Is it representative?  Does the sample size seem adequate?  For qualitative studies, how was the sample chosen?  Does the author explain how individuals were chosen?  Does the sample size seem adequate?  Remember to think about data saturation rather than simply looking at numbers to determine adequacy.
14.  What efforts did the investigator make to ensure validity and reliability?  Think about both internal and external validity, control, bias, and manipulation as well as reliability for quantitative research and trustworthiness in qualitative research..
15.  What data collection techniques were used? Is the rationale for these techniques specified in the literature review and/or in the methods section?  The point here is that the researcher must have a reason for choosing a particular method of data collection.  This is not random and you should be able to see why the choice was made.
16.  How does the data collection fit with the study purpose and study question?  This gets at the idea of consistency again.
17.  How are the data analyzed?  Does the analysis make sense for the study?  How does the analysis plan fit with the study purpose and the study question?  Again, this must all fit together.  How the author planned to analyze the data and how he actually did it must be consistent with the purpose of the study and what he was trying to find out.
18.  Are the conclusions supported by the study?  In other words, can the author really say what he is saying?  Can he make these conclusions based on what actually happened?
19.  What are the strengths (and weaknesses) of the study?
20.  What level of knowledge is generated?  Think of this as the type of knowledge generated.  Remember the difference between quantitative and qualitative research with respect to knowledge going from specific to general (qualitative) and general to specific (quantitative).
21.  What use does this knowledge have for health and service practice?  In other words, what are the applications and impact on practice?
22.  Are there ethical dilemmas presented in this article? What are they?  Did the author(s) resolve them in a reasonable and ethical manner?
23.  Overall, do you think this was a good research study?  Why or why not?