“A systematic investigation (i.e., the gathering and analysis of information) designed to develop or contribute to generalizable knowledge.”
[The Code of Federal Regulations – Protection of human subjects (45 CFR 46.102(d)) (2009)]
In this blog, I shall express my critical view on the way we conduct research. My critical view revolves around the following arguments:
- The choice of both quantitative and qualitative approaches should be informed by the existing body of relevant literature.
- Both quantitative and qualitative apporaches should be addressed by different research.
- The philosophical underpinning should not be taken in reverse order.
No matter what topic is being studied, the value of the research depends on how well it is designed and done. As focusing on how to design a study, this blog also can be beneficial for other researchers from other disciplines. Research requires following a systematic process known as ‘the scientific method’. This process is illustrated as follows:
Researchers may not require going through all the stages of the scientific method. That is, some research is guided by either some research questions or some hypotheses. The observation stage helps researchers to formulate some research questions. Then, researchers can proceed to conduct a systematic literature review to provide a theoretical support. However, sometimes a scoping literature review is used as the researcher does not have a specific question (see also thematic literature review method). For the case of using hypotheses, researchers may have to review the existing body of relevant literature to reinvestigate a hypothesis that were tested (to replicate the study), or to draw on the hypotheses that were recommended for future research. There are different types of hypotheses (see further details: https://www.studyandexam.com/hypothesis.html).
Research in linguistic and education usually requires both data collection and data analysis. To do so, researchers should design the study by selecting a methodology (its characteristics should be reflected in the research) and some methods (for data collection and data analysis). Part of the scientific method is ‘designing the study’, which I focus on in this blog.
Each study should have a research paradigm, which could be adopted from other studies or designed by the principal investigator (the researcher who is undertaking the research). According to Scotland (2012), research paradigm consists of the following components: ontology, epistemology, methodology, and methods. However, in Crotty’s (1998) conceptual framework ontology was dropped out, as he thinks, it goes together with epistemology and theoretical framework (note that this theoretical framework does not refer to the selection of theories and concepts from a discipline or disciplines) was added as researchers need to identify what approach is useful. These components are identified as ‘philosophical underpinnings’, understanding the philosophical underpinnings is enormously important.
Too much information is availble to date. That is what makes designing a study quite difficult. I shall provide the following summary which represents the different philosophical underpinnings (Patel 2015):
- Epistemological beliefs: positivism; constructivism/interpretivism; pragmatism; and subjectivism
- Theoretical frameworks: positivism and post-positivism; interpretivism; pragmatism; postmodernism, structuralism, and post structuralism; and Marxism, queer theory, and feminism
- Methodologies: experimental research; survey research; ethnography; grounded theory; phenomenology; heuristic inquiry; action research; discourse analysis; feminist standpoint research; and critical discourse analysis
- Research methods: sampling; measurement and scaling; statistical analysis; questionnaire; focus group; interview, observation; case study; life history; narrative; theme identification; data mining expert review; usability testing; physical prototype; semiotics; literary analysis; pastiche; intertextuality; ideological review; and civil actions
While discussing these in your research, you reflect a story of how the research is thought about. Sections for the discussion of the philosophical underpinnings related to the research should be arranged . The philosophical underpinning should be taken the following order:
This is the order that should be taken into consideration when designing the study. Epistemology, which addresses the question of ‘what/how can I know reality/knowledge?’, may determine the type of the theoretical framework, which addresses the question of ‘what approach can be used to get knowledge?’. Then, the researcher delves into the wide variety of methodologies, which addresses the questions of ‘what procedure can we use to acquire knowledge?. Last but not least, some research methods can be used, if they really help to obtain data (methods from data collection) and reveal findings (methods for data analysis). some research require using more research methods (e.g., classroom observations or online observations; interviews; questionnaires, focus group interviews and discussions; think-aloud procedure). The research methods may never determine the type of the methodology. Referring to the philosophical underpinnings in reverse order is just wrong.
The divide of Qualitative-quantitative approach
The use of qualitative and quantitative approaches is one of the points that I would like to stress in my critical view on doing research. While I was listening to some presenters in conferences, some statements made about the way both approaches were selected do not have a methodological basis. I could realise that some researchers select the approaches according to the type of the methodology (e.g., action research).
There are two different views on the difference between the quantitative and qualitative approaches. On the one hand, Crotty (1998, p.15) claims that the difference between qualitative and quantitative research occurs at the level of methods not at the level of epistemology or the theoretical perspective, and “quantification is by no means ruled out within non-positivist research”. On the other hand, Trochim (2006) asserts that the differences occur at the level of epistemological assumptions not the data; all qualitative data can be coded quantitatively, and all quantitative data is based on qualitative judgment.
The two views suggested that quantification may always be possible. This means that the procedures followed in data analyses are by no means limited to the epistemological beliefs or the methods. Quantitative and qualitative approaches are interrelated, Creswell (2014) explains that qualitative, quantitative, and mixed methods approaches are not different categories, a study can be primarily qualitative or the other way around.
Others seem that they love measuring items (quantitative analysis), without stating the reason for doing so or giving a theoretical support. Some research questions cannot be answered just by conducting quantitative or qualitative analysis because that could not be enough to provide an evidence-based input. Therefore, a mixed-method study could be required. For example, research questions such as ‘to what extent does X occur …’, require some quantitative analysis, and cannot be discussed by using the participants’ opinions. If to use findings of the qualitative analysis, there should be another research question for the qualitative side. Besides, the selection of both approaches also should be informed by the literature review. Sometimes the literature shows that it is impossible to measure the items objectively, acknowledging the possibility is needed.
Testing the feasibility of the research paradigm
When designing a study, researchers attempt to conduct pilot studies to test the feasibility of the overall research paradigm. In this respect, Hulley et al. (2007, p.169) asserted that pilot studies are conducted by focusing “primarily on determining the feasibility, time required and costof recruiting adequate numbers of eligible participants … also be designed to demonstrate that planned measurements, data collection instruments and data management systems are feasible and efficient.”
Thabane et al. (2010) point out to the rationale of a pilot study which is classified into four broad areas:
- Process: to assess if the steps taken within the main study are feasible
- Resources: to assess the problems of time and budget while conducting the main study
- Management: to find if there any arising issues concerning data management
- Scientific: to assess, for example, treatment safety, determination of dose levels and response, and treatment effect
The conclusions of the pilot study define the feasibility objectives, rather than reinforcing the significance of the study as in the main study. Thabane et al. (2010) have explained that the criteria for interpreting the pilot study are based on the primary feasibility objectives, which are listed as follows; “stop – main study not feasible; continue, but modify protocol – feasible with modifications; continue without modifications, but monitor closely – feasible with close monitoring; and continue without modifications – feasible as is” (p.5).
Bibliography
Creswell, W. John. 2014. Research design: qualitative, quantitative, and mixed methods approaches. 4th ed. USA: SAGE.
Crotty, Michael. 1998. Foundations of social research: Meaning and perspective in the Research process. London: SAGE.
Hulley, B. Stephen, Steven R. Cummings, Warren S. Browner, Deborah G. Grady, and Thomas B. Newman. 2007. Designing Clinical Research. 3nd ed. USA: Lippincott Williams & Wilkins. file:///C:/Users/99904870/Downloads/Designing+Research.pdf [Accessed 02November 2017].
McLeod, S. A. (2018, August 10). What is a hypothesis? Simply Psychology. https://www.simplypsychology.org/what-is-a-hypotheses.html [accessed 23 January 2019]
Patel, Salma. 2015. The research paradigm-methodology, epistemology and ontology- explained in simple language. [Blog] Healthcare, Technology, Participation, Research and a PhD. http://salmapatel.co.uk/academia/the-research-paradigm-methodology-epistemology-and-ontology-explained-in-simple-language [Accessed 20 October 2017].
Scotland, James. 2012. Exploring the philosophical underpinnings of research: Relating ontology and epistemology to the methodology and methods of the scientific, interpretive, and critical research paradigms. English Language Teaching, 5(9):9–16. Canadian Center of Science and Education. http://dx.doi.org/10.5539/elt.v5n9p9
Sekaran, U. & Boujie, R., (2016). Research methods for business: A skill building approach (7th Edition). John Wiley & Sons.
Thabane, Lahana, Jinhui Ma, Rong Chu, Ji Cheng, Afisi Ismaila, Rios P Lorena, Reid Robson et al. 2010. A tutorial on pilot studies: the what, why and how. BMC Medical Research Methodology, 10 (1): 1-10. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2824145/ [Accessed 14 June 2017].
Trochim, William.M.K. (2006). Research methods knowledge base. (2nd ed.) Available at: http://www.socialresearchmethods.net/kb/qualdeb.php [Accessed 20 October 2017].
Research & Creativity for Education and Linguistics 