AL-88 · Selected Studies — Sport Management

Survey Study Designer

A well-designed survey study is harder than it looks. This guided workflow walks through the five stages of survey research design — from sharper research questions through sampling strategy, instrument design, pilot planning, and analysis plan. Grounded in the methodology of a 2005 study on performance-enhancing supplement use among student-athletes at a Division I university (n=226, SPSS analysis), which demonstrated the messiness of real research design.

📊 Data: Survey methodology: Fowler (2014), Survey Research Methods (5th ed.); Dillman et al. (2014), Internet, Phone, Mail, and Mixed-Mode Surveys; Likert (1932) scale origins

Stage 1 — Research Question

Draft Research Question Write your initial draft — we'll sharpen it next.

Quality Checklist

Is this question empirically answerable (not philosophical or purely normative)?
Is it specific enough to be testable — a single, bounded claim?
Is it feasible within your available sample and resources?
Is it original — not already definitively answered in the literature?

RQ Sharpener — Example Transformations

❌ Vague / untestable

"Do athletes use supplements?"

✓ Specific / empirically testable

"What is the self-reported prevalence of dietary supplement use among NCAA Division I student-athletes at [University X], and what are the primary perceived reasons for use?"

❌ Too broad

"How does coaching affect athletes?"

✓ Bounded and measurable

"Is there a statistically significant relationship between perceived coach pressure (5-point Likert scale) and self-reported frequency of dietary supplement use among Division I athletes?"

❌ Normative (not empirical)

"Should student-athletes use performance-enhancing supplements?"

✓ Descriptive / comparative

"Do Division I athletes who receive coach-provided nutrition guidance report lower rates of unsanctioned supplement use than those who do not?"

Hypotheses Builder

Hypothesis 1 (Directional)

Hypothesis 2 (Directional)

Hypothesis 3 (Null / Exploratory)

Study Design Type

Stage 2 — Population & Sampling

Target Population The full group your research question is about.

Accessible Population The subset you can actually reach.

Sampling Strategy

Simple Random Sampling

Equal probability for all; requires complete roster.

Stratified Random

Sample within subgroups; ensures representation.

Cluster Sampling

Sample natural groups (teams, schools); less precise.

Convenience / Purposive

Whoever is available; fast but limits generalizability.

Systematic Sampling

Every nth person from a list; simple and practical.

Sample Size Estimator

Population Size (N) Total accessible population

Margin of Error (%) ±% acceptable error

Confidence Level

Required n

—

Formula Used

            n₀ = Z² × p × (1–p) / e²

            n = n₀ / (1 + (n₀–1)/N)

            Z=z-score, p=0.5 (max variance), e=margin of error

Nonresponse bias note: Nonresponse is not random. Who doesn't respond tells you something. Plan for it: use incentives, send reminders, and compare early vs. late responders to test for response bias. A 40% response rate with nonresponse bias can be worse than a 70% response rate without it.

Stage 3 — Instrument Design

Number of Constructs / Survey Sections

Likert Scale Builder

5-Point Agree/Disagree

            1 — Strongly Disagree

            2 — Disagree

            3 — Neither agree nor disagree

            4 — Agree

            5 — Strongly Agree

Use for: attitude/opinion items. Neutral midpoint is genuinely useful here.

5-Point Frequency

            1 — Never

            2 — Rarely

            3 — Sometimes

            4 — Often

            5 — Always

Use for: behavioral frequency items. Avoid "always" anchors for behaviors that vary.

7-Point Intensity Scale

            1 — Extremely Low

            4 — Moderate

            7 — Extremely High

Use for: measuring degree of construct (e.g., perceived pressure). More variance than 5-point.

Semantic Differential

            Harmful — 1 2 3 4 5 — Beneficial

            Weak — 1 2 3 4 5 — Strong

Use for: bipolar constructs (good/bad, strong/weak). Osgood's original scale.

Item Writing Checklist (per question)

Single topic per item — no double-barreled questions ("Do you use protein powder and creatine?": bad)
No leading language ("Don't you agree that supplements are dangerous?" is leading)
No double negatives ("I do not disagree that..." — confusing)
Clear, concrete, unambiguous — every respondent interprets the question the same way
Consistent with response scale — don't ask frequency with an agree/disagree scale
Appropriate reading level for your target population

Sample Question Bank

Supplement Use (Frequency)

Behavioral Frequency · 5-Point Scale"How often do you use dietary supplements (protein powders, creatine, vitamins, etc.)? 1=Never, 2=Rarely, 3=Sometimes, 4=Often, 5=Always"

Type Checklist · Binary"Which of the following supplements have you used in the past 12 months? (Check all that apply): □ Protein powder □ Creatine □ Multivitamins □ Energy drinks □ Pre-workout formulas □ None"

Perceived Pressure (Likert)

Coach Pressure · 5-Point Intensity"To what extent do you feel pressure from coaches to improve athletic performance through nutrition or supplementation? 1=No pressure at all, 5=Extreme pressure"

Peer Influence · 5-Point Agree/Disagree"Most of my teammates use dietary supplements." (SA/A/N/D/SD)

Knowledge / Awareness

True/False Knowledge Item"Creatine is classified as a banned substance by the NCAA. □ True □ False □ Not sure" (Correct answer: False — creatine is not banned by NCAA)

Knowledge Scale Item"How familiar are you with the NCAA's banned substance list? 1=Not at all familiar, 5=Extremely familiar"

Instrument Draft Generator

Generate sample question text for each section defined above. Edit the output before use — this is a starting scaffold, not a final instrument.

Stage 4 — Pilot Plan

Cognitive Interviewing: Test 5–7 people from your target population. Ask them to think aloud while completing the survey. What are they actually interpreting? Pay attention to: how long they pause on specific questions, when they ask for clarification, and whether their verbal reasoning matches the intent of the item.

Pilot Checklist

Timing: How long does it actually take to complete? (Aim for ≤15 minutes for survey studies. Long surveys have dramatically higher dropout rates.)
Instructions: Are they clear without verbal explanation? Can a respondent complete the survey cold, without you present?
Skip logic: Does conditional routing work correctly? (e.g., "If Q3 = Never, skip to Q7")
Response distributions: Is there any question where everyone gives the same answer? (Ceiling/floor effects — the question has no discriminatory power)
Sensitive questions: Do any questions cause visible discomfort or refusal? Plan to reframe, make optional, or remove.
Terminology: Do respondents understand all terms? (e.g., Do athletes know what "NCAA-sanctioned" means?)

Common Pilot Findings & Fixes

Finding	Fix
All responses clustered at one end of scale	Add more extreme anchors; rephrase to increase variance; consider 7-point scale
Respondents frequently skip question X	Rephrase for clarity; make it optional; or remove if not central to RQ
Respondents interpret question differently than intended	Rewrite with explicit operationalization; add brief definition in parentheses
Survey takes 25 minutes instead of 10	Cut lowest-priority items; combine constructs; use shorter stems
Double-barreled question identified	Split into two separate items; decide which is most important to your RQ

Stage 5 — Analysis Plan

Always report descriptive statistics before inferential tests. Readers need to know means, SDs, and n before they can interpret a p-value.

Descriptive Statistics to Report

Frequencies and percentages for all categorical/nominal variables (sport, gender, year in school)
Mean (M) and standard deviation (SD) for each Likert scale item
M and SD for composite scale scores (sum or mean of items within a construct)
Range (min–max) and n per item (flag missing data)
Cronbach's alpha (α) for multi-item scales — reliability check (α ≥ 0.70 acceptable; ≥ 0.80 preferred)

Inferential Test Selector

Match your test to the design type (Stage 1) and measurement level of your variables.

Scenario	Test	SPSS Path
2 groups, continuous/interval outcome	Independent samples t-test	Analyze → Compare Means → Independent-Samples T Test
3+ groups, continuous outcome	One-Way ANOVA	Analyze → Compare Means → One-Way ANOVA
2 categorical variables (association)	Chi-square test of independence	Analyze → Descriptive Statistics → Crosstabs (Chi-square)
2 continuous variables, normal distribution	Pearson correlation (r)	Analyze → Correlate → Bivariate (Pearson)
2 ordinal/non-normal variables	Spearman rho (ρ)	Analyze → Correlate → Bivariate (Spearman)
Predict continuous outcome from predictors	Multiple regression	Analyze → Regression → Linear

APA Reporting Templates

t-test

An independent samples t-test was conducted to compare [outcome] for [Group A] (M = ___, SD = ___) and [Group B] (M = ___, SD = ___). There was a [significant / no significant] difference; t(df) = ___, p = .___, d = ___.

Chi-square

A chi-square test of independence revealed a [significant / no significant] association between [Variable 1] and [Variable 2], χ²(df, N = ___) = ___, p = .___, Cramér's V = ___.

Pearson Correlation

There was a [positive / negative / no] correlation between [Variable A] and [Variable B], r(n–2) = ___, p = .___. [Variable A] accounted for ___% of the variance in [Variable B].

Missing Data Plan

Report n per item in your results table. Consider listwise vs. pairwise deletion strategy. Mean imputation is acceptable for <5% missing on any given item. If missing data exceeds 10%, report it explicitly and discuss potential nonresponse bias. Never simply drop cases without acknowledging the implications for your sample.