STAGE 4: Collecting Data

Experimental design is one of the traditional methods to carry out quantitative educational research. It has been used to establish a cause and effect between independent and dependent variables. This method might involve you attempting to ensure that all other variables remain constant to measure different outcomes caused by the variable. For example, you might want to investigate how a revamped curriculum might impact students' understanding towards other Southeast Asian states.

Because groups may have a different base starting point, you may want to adopt a pretest and post-test approach when conducting experimental design. In the pretest, you assess participants in your identified characteristic before you carry out your experiment and introduce changed variables. Post test involves measuring the same characteristics after change has been implemented. For example, if 2 classes are involved in the research, an understanding towards other Southeast Asian states in both classes is first measured before the different curriculums are implemented.

It is also important to consider internal validity and external validity of your experiment. See more here for an explanation on internal and external validity.

• Experimental designs are well-suited for determining causal relationships.
• Well-designed experiments can be replicated by other researchers, leading to a stronger evidence base.

• Experimental designs often take place in controlled environments that may not accurately reflect real-world settings.
• Ethical limitations might arise as to what can be manipulated in educational research. For example, withholding potentially beneficial educational interventions from a control group can raise ethical concerns.
• The complexity of educational settings involving many interacting variables (e.g., teacher quality, student background, school resources) makes it difficult to isolate the effect of a single variable.
• It is resource intensive, especially on a larger scale.

Case study is an in-depth research method which sets clear boundaries to investigate an individual, group of people, organisation, or event. A case study is appropriate to analyse a single phenomenon together with its context and background. By the intent of your research, you can embark on different types of case studies with varying methods of inquiry and analysis.

To ensure the validity of conclusions, you should carry out triangulation of data, that is, using two or more forms of data collection tools/perspectives to understand the area of study.

Case studies offer context-specific insights that cannot be generalised easily. Instead, they provide valuable understanding of complex situations.

• Allows carefully bounded area of study, which emphasises on depth and intensity
• Allows greater insights and understanding of a particular context

• Might be time-consuming
• Results may be limited in terms of applying it to other situations
• People might change their behaviour when they are aware they are being observed
• Potential researcher bias

Design-based research (DBR) is a methodology that involves researchers developing a potential solution, based on theory or previous research, to an identified problem and testing it out. Once it is first piloted in a real-world setting, the solution is evaluated and tweaked based on feedback and then implemented to investigate the results. The cycle repeats. For example, you might identify a particular learning challenge faced by students and then design a potential “solution” to the problem in the form of instructional tools.

DBR involves the contributions of participants in the process – from conception to evaluation and reflection. Rather than just a specific intervention to change the learning environment, a “design solution” is implemented as a larger scale change over a longer period that does not necessarily have to keep all the other variables constant like in an experiment.

DBR should be based on established learning theories, which guide the design of instructional tools and are refined throughout the research process. Research should directly address specific challenges, leading to tangible improvements in learning.

DBR should also produce guiding principles for future instructional tool development and implementation to make it applicable across different educational settings. It is important that you carry out long-term, iterative experiments in classrooms to accurately assess the impact of instructional tools on student learning, providing a more comprehensive evaluation.

• Combines and refines learning theories with practical application
• Produces findings directly applicable to educational settings
• Allows for continuous refinement of tools and strategies and long-term studies capture detailed impacts on learning
• Generates actionable guidelines for future educational tools
• Collaborative approach enhances relevance of research

• Iterative and long-term nature requires significant time and resources, substantial time, funding, and collaboration is needed
• Due to context-specific findings, results may be difficult to generalise across different settings

Ethnographic research aims to describe what is happening in a particular setting and context, including participants’ perspectives on the events that are happening. Ethnographic research takes a holistic view of a social group by examining all events and phenomena occurring simultaneously.

Direct observation and interviews are common data collection methods in ethnographic research, which is often conducted in classroom settings within educational research.

This method might be particularly relevant for researchers who are interested to immerse themselves in the learning environment. Because it portrays social groups in their natural situations, the research relies on heavy involvement and immersion into the culture, bringing personal experience as an important aspect of the research. It differs from merely observing and moves into a sphere where the researcher should actively be part of the community.

Ethnography is particularly suited for exploratory research when you are not familiar with the education setting. For example, you might want to compare different universities, but you are not familiar with one of them. In this case, ethnographic research might be useful to learn more about it. In education research, ethnographic research projects tend to focus on institutional culture, teaching methods, or behaviours.

• Holistic understanding of context and research area
• Ability to get first-hand knowledge without worrying about reliability of secondary data
• Provides rich data, especially useful for complex situations

• Time consuming
• Building trust with participants to obtain honest responses is difficult
• Possible observer bias
• Not representative and findings may not extend to other similar situations

Mixed-method research design is a procedure that mixes both quantitative and qualitative methods. For example, a study looking into the impact of learning management systems could use surveys to measure quantitative data on students' use of these systems. At the same time, interviews or focus groups can provide qualitative data on students' experiences and opinions.

Mixed method design is useful when merely using one type of data is not sufficient to address the research question. You can leverage on the strengths of both quantitative and qualitative data, complementing each other in addressing your research question.

When employing a mixed-method design, consider the following factors: the emphasis on quantitative versus qualitative data collection, the sequence in which the data are collected, whether you will combine the data in a single analysis or analyze them separately, and the timing for integrating the data.

• Allows for method flexibility
• Gains valuable insights by drawing on both qualitative and quantitative data

• Time-consuming
• Requires extensive data collection and analysis
• Requires familiarisation with multiple methods of data collection

Action research aims to address a real-life problem by formulating a concrete solution as part of the research process.

For example, if researching how SMU can encourage students to take more interdisciplinary courses, you might use questionnaires and interviews with students, and gather their reflections from conversations with professors or peers. You could also consult personnel from other universities. This approach aims to develop practical solutions for exploration.

There are two types of action research design: practical action research and participatory action research. Practical action research addresses immediate, micro-level problems within schools or classrooms, whereas participatory action research aims to contribute to broader societal changes.

It is important to get participants' commitment throughout the process since continued engagement is crucial in action research design.

Clearly outlining the problem definition is crucial. Without a thorough prior understanding and investigation, you might rely on incorrect assumptions or discover that the problem differs from your initial perception, particularly when collaborating with external stakeholders.

Where possible, research existing theories to enhance your understanding and formulation of the research problem. It may be particularly useful to explore broader concepts, such as social behaviour and communication.

• Highly relevant to educational practitioners given its rootedness in practical action
• Leads to tangible benefits in educational settings, enhancing teaching and learning processes

• Introduces subjectivity and bias through practitioners' involvement as researchers
• Requires a significant investment of time and resources
• Ethical issues can arise, especially related to consent, confidentiality, and the impact of interventions on students

Grounded theory examines specific phenomena to develop new theories based on real-world data. Unlike traditional research, which starts with a hypothesis, grounded theory is inductive, meaning theories emerge from the data itself. You begin by collecting empirical data and then develop a theory grounded in these results. The process involves repeatedly collecting and analysing data until no new insights emerge.

Grounded theory is ideal when existing theories do not adequately address your research interests. Compared to other qualitative designs like ethnography or narrative research, grounded theory studies are more structured and scientific. They typically include a clear problem statement, methods, discussion, and results, presented in a third-person, objective tone. The conclusion of a grounded theory project should be a theory that abstracts the examined process.

• No need for a hypothesis, making it suitable for underdeveloped areas of research
• Flexibility due to the absence of preconceived hypotheses
• Tightly interwoven data collection and analysis, which may enhance validity and reduce confirmation bias

• Lower theoretical sensitivity if the researcher lacks sufficient knowledge about the topic
• Potentially time-consuming, especially if conflicting data requires further investigation
• Risk of researcher bias as observations are translated into text
• Validity may be impacted since grounded theory is not repeatable and opinions or beliefs can change

Discourse analysis is used to study and analyse language use, focusing on how and what is said. It examines texts, meanings, and inferences to understand the relationship between language and social constructs, revealing hidden motivations in both others and itself.

In education research, discourse analysis involves analysing written, spoken, or visual texts to uncover underlying assumptions, ideologies, and power dynamics. This can include studying classroom interactions, educational policies, and curriculum materials to explore how different 'voices' in the text reflect relationships between the creator and the audience.

• Encourages a critical examination of language
• Captures the complexity and nuance of interactions, allowing for a more comprehensive understanding

• Interprets subjectively due to the nature of discourse analysis
• Requires significant time and expertise, making analysis complex and demanding
• Limits generalisability, as findings are often context-specific
• Risks over-interpreting data, potentially attributing unintended patterns or meanings