Chapter 8
Quantitative Methods
Introduction
• Designing quantitative methods for a researchproposal • Survey and experimental designs • Careful measurement, parsimonious variables, theory-guided
Creswell, Research Design 5e SAGE Publishing, 2018 3
Defining Surveys and Experiments
• Survey design – Quantitative description of trends, attitudes, oropinions of a population – Testing association – Studying a sample of that population
Creswell, Research Design 5e SAGE Publishing, 2018 4
Defining Surveys and Experiments
• Experimental design – Systematic manipulation of one or more variables to evaluate an outcome – Holds other variables constant to isolate effects – Generalize to a broader population
Creswell, Research Design 5e SAGE Publishing, 2018 5
Components of a Survey MethodPlan
• The survey design • The population and sample • Instrumentation • Variables in the study • Data analysis and interpretation
Creswell, Research Design 5e SAGE Publishing, 2018 6
Components of a Survey MethodPlan
Creswell, Research Design 5e SAGE Publishing, 2018 7
Components of a Survey MethodPlan
Creswell, Research Design 5e SAGE Publishing, 2018 8
Components of a Survey MethodPlan
The survey design: • Provide a purpose for using survey research • Indicate why the survey method is preferred • Indicate the type of survey design – Cross-sectional (data collected at one point in time) – Longitudinal (data collected over time)
• Specify the form of data collection – telephone, mail, Internet personal/group interviews) and rationale
Creswell, Research Design 5e SAGE Publishing, 2018 9
Components of a Survey MethodPlan
The population and sample: • Identify the population including size and sampling frames • Specify the sampling design – Single-stage –Multi-stage (clustering)
• Type of sampling – Probability – Nonprobability
Creswell, Research Design 5e SAGE Publishing, 2018 10
Components of a Survey MethodPlan
The population and sample: • Indicate if the study involves stratification – ensuringspecific population characteristics (e.g. gender) are represented • Indicate number in the sample and procedure todetermine • Use a power analysis if you plan to detect significantassociations
Creswell, Research Design 5e SAGE Publishing, 2018 11
Components of a Survey MethodPlan
The population and sample: • Power analysis involves
– Estimating the size of correlation – An alpha value (type I error rate) – A beta value (type II error rate) – Conducting the power analysis (e.g., G*Power)
• Conduct power analysis during planning
Creswell, Research Design 5e SAGE Publishing, 2018 12
Components of a Survey MethodPlan
Instrumentation: • Name the survey instrument used to collect data • Indicate how instrument was developed • Describe the established validity scores from past use – Content validity – Predictive or concurrent validity – Construct validity
• Describe reliability of scores from past use – Internal consistency – Test-retest
Creswell, Research Design 5e SAGE Publishing, 2018 13
Components of a Survey MethodPlan
Instrumentation: • When modifying or combining instruments, the original validity and reliability may not hold • Include sample items from the instrument • Indicate major content sections in the instrument – Cover letter – Items – demographics, attitude items, behavioritems, factual items – Closing instructions – Type of scale for responses
Creswell, Research Design 5e SAGE Publishing, 2018 14
Components of a Survey MethodPlan
Instrumentation: • Discuss pilot testing or field-testing
– Rationale for plans – Content validity and reliability – Improve question
• Steps for administering for a mailed survey
Creswell, Research Design 5e SAGE Publishing, 2018 15
Components of a Survey MethodPlan
Creswell, Research Design 5e SAGE Publishing, 2018 16
Components of a Survey MethodPlan
Data analysis: • Computer programs used for analysis • Data analysis steps – Step 1. Number who did and did not respond – Step 2. Method to determine response bias – Step 3. Plan to provide descriptive analyses – Step 4. Calculate total scale scores – Step 5. Statistics and program for inferentialstatistical analyses – Step 6. Present results in figures or tables and interpret
Creswell, Research Design 5e SAGE Publishing, 2018 17
Components of a Survey MethodPlan
Interpreting results and writing a discussion section: • Report how the results answered the research question or hypothesis
• Practical evidence in terms of effect size and confidence interval
• Discuss implications – Consistent with, refute, extent previous studies
Creswell, Research Design 5e SAGE Publishing, 2018 18
Components of a Survey MethodPlan
Creswell, Research Design 5e SAGE Publishing, 2018 19
Components of an ExperimentalStudy Method Plan
• Experimental method plan – Participants – Variables – Instrumentation and materials – Experimental procedures – Threats to validity – Data analysis – Interpreting results and writing a discussion
Creswell, Research Design 5e SAGE Publishing, 2018 20
Components of an ExperimentalStudy Method Plan
Participants: • Describe procedures for recruiting participants • Describe the selection of participants as either – Random – Nonrandom (convenience)
• True experiment – individuals randomly assigned to groups • Quasi-experiment – partial or no control over random assignment
Creswell, Research Design 5e SAGE Publishing, 2018 21
Components of an ExperimentalStudy Method Plan
Participants: • May measure second predictor variables • Conduct and report power analysis • End with formal experimental design statement – “The experiment consisted of a one-way two-groups design comparing burnout symptoms between full-time and part-time nurses”
Creswell, Research Design 5e SAGE Publishing, 2018 22
Components of an ExperimentalStudy Method Plan
Creswell, Research Design 5e SAGE Publishing, 2018 23
Components of an ExperimentalStudy Method Plan
Variables: • Specify the variables and describe in detail
– Identify the independent variables – Include a manipulation check measure – Identify dependent variable – Identify other variables measured • Participant demographics • Measure variables that contribute noise • Potential confounding variables
Creswell, Research Design 5e SAGE Publishing, 2018 24
Components of an ExperimentalStudy Method Plan
Instrumentation and materials: • Describe the instrument(s) participants complete in theexperiment – Development, items, and scales – Reliability and validity reports of past uses
• Thoroughly discuss materials used for the treatment • Cover story to explain procedures if deception is used
Creswell, Research Design 5e SAGE Publishing, 2018 25
Components of an ExperimentalStudy Method Plan
Experimental procedures:• Identify the type of experiment
– Pre-experimental, true experiment, quasi-experiment,single subject design
• Identify the type of comparisons – within-group or between-subject – Provide a visual model to illustrate the research design used • X = treatment • O = observation • R = random assignment
See Examples 8.2–8.5 Creswell, Research Design 5e SAGE Publishing, 2018 26
Components of an ExperimentalStudy Method Plan
Creswell, Research Design 5e SAGE Publishing, 2018 27
Components of an ExperimentalStudy Method Plan
Creswell, Research Design 5e SAGE Publishing, 2018 28
Components of an ExperimentalStudy Method Plan
Creswell, Research Design 5e SAGE Publishing, 2018 29
Components of an ExperimentalStudy Method Plan
Creswell, Research Design 5e SAGE Publishing, 2018 30
Components of an ExperimentalStudy Method Plan
Threats to validity: • Internal validity – procedures, treatments, or experiences of the participants that threaten inferences in experiments • External validity – drawing incorrect inferences from sample data to other persons, settings, situations
Creswell, Research Design 5e SAGE Publishing, 2018 31
Components of an ExperimentalStudy Method Plan
Threats to validity: • Statistical conclusion validity – inadequate statistical power or violation of statistical assumptions • Construct validity – inadequate definitions and measures of variables
Creswell, Research Design 5e SAGE Publishing, 2018 32
Components of an ExperimentalStudy Method Plan
• Threats to internal validity – History – Compensatory/resentful – Maturation demoralization – Regression – Compensatory rivalry – Selection – Testing –Mortality – Instrumentation – Diffusion of treatment
Creswell, Research Design 5e SAGE Publishing, 2018 33
Components of an ExperimentalStudy Method Plan
• Threats to external validity – Interaction of selection and treatment – Interaction of setting and treatment – Interaction of history and treatment
Creswell, Research Design 5e SAGE Publishing, 2018 34
Components of an ExperimentalStudy Method Plan
The procedure: • Administer measures of the dependent variable or a variable closely correlated • Assign participants to matched pairs • Randomly assign one member of each pair to the control and experimental group • Expose experimental group to the treatment • Administer measures of dependent variables • Compare performance of the experimental and control groups
Creswell, Research Design 5e SAGE Publishing, 2018 35
Components of an ExperimentalStudy Method Plan
Data analysis: • Report descriptive statistics (e.g., means, standarddeviations, ranges) • Indicate inferential statistical tests (e.g., t test, ANOVA,ANCOVA, or MANOVA) • Report confidence intervals and effect sizes in addition to statistical tests • Use line graphs for single subject designs
Creswell, Research Design 5e SAGE Publishing, 2018 36
Components of an ExperimentalStudy Method Plan
Interpreting results and writing a discussion section: • Interpret findings in light of hypotheses and research questions • Whether supported or refuted • Why results significant or not, literature • Indicate implications • Suggest future research
Creswell, Research Design 5e SAGE Publishing, 2018 37
Summary
• The methodological approach to a survey or experiment • Surveys – purpose, population and sample, instruments, relationship, research questions, items, analysis • Experiments – identify participants, variables,instruments, type of experiment, validly, analysis
Creswell, Research Design 5e SAGE Publishing, 2018 38
