Sitonce
Country: US
Show exams for United States Hong Kong
Sign in

Research Methods, Statistics and Program Evaluation (FTCE School Psychologist PK-12)

Updated 7 min read
Key takeaway

FTCE School Psychologist PK-12 (036) is an 80-item multiple-choice examination scheduled for 150 minutes.

  • It covers research and data use, development, academic and mental-health intervention, safety, consultation, ethics and education law.
  • For research methods, statistics and program evaluation, use relevant evidence and professional procedures.
  • Check FDOE for current scoring and certification guidance.
On this page6 sections
  1. Measurement, reliability and validity
  2. Statistical ideas for school decisions
  3. Choose research design to fit the question
  4. Apply findings without overgeneralizing
  5. Common research and statistics traps
  6. Practice with a research decision

Measurement, reliability and validity

Section 36 begins with measurement, test construction, reliability and validity. Reliability concerns the consistency of scores or classifications under specified conditions. Validity concerns the evidence supporting an interpretation and use of results. Neither is a blanket property that makes a measure appropriate for every student or decision. A school psychologist asks what construct the tool measures, who was represented in the comparison sample, how it was administered, and whether the proposed conclusion follows from the evidence. The current FDOE Section 36 outline names these concepts as examinable knowledge.

Suppose a teacher asks whether a student’s low mathematics performance reflects a stable skill difficulty. A single classroom quiz may be inconsistent because it covers a narrow unit, while a standardized score may have measurement error and may not explain the student’s work process. Review curriculum samples, instructional opportunity, progress data and assessment conditions. A score can contribute evidence, but its confidence interval, norms, language demands and intended construct affect how much weight it can carry.

Do not use reliability and validity as synonyms. A measure could produce highly consistent scores while measuring a construct that is not the question at hand. A test might reliably capture rapid naming but offer limited evidence about a student’s broader academic access. Conversely, a useful interpretation may require several evidence sources because no single tool measures the full referral question. In exam scenarios, identify the proposed use first, then judge whether the available evidence supports it.

Statistical ideas for school decisions

Statistics help describe patterns, uncertainty and relationships. Mean, median and mode summarize distributions differently. Range and standard deviation describe spread, while percentiles compare a score with a reference group rather than percent correct. Correlation describes association, not causation. A statistically significant finding does not by itself show that an intervention matters educationally, that its effect is large, or that it applies to a particular student. Interpret the statistic in the context of the measure, sample and decision.

Consider a school report showing that attendance and course grades are associated. That pattern may help identify questions for further study, but it does not prove absences caused every low grade. Health, transportation, course access, housing instability, school climate and other variables may be related. A responsible interpretation describes the association, considers alternative explanations and avoids attributing blame. The next step may be to review individual context or evaluate a support, not to apply a causal remedy based on correlation alone.

Scores also have uncertainty. A small difference between two scores may not represent meaningful change, especially when the test’s standard error is large. A percentile is a relative rank within a norm group and can shift with the group or test edition. Raw scores, standard scores and percentiles are not interchangeable. On an exam item, look at the scale, comparison group and question being asked before selecting an interpretation.

Choose research design to fit the question

Quantitative research can estimate patterns or compare outcomes, but the design determines what can be concluded. Random assignment can strengthen causal inference when implemented appropriately, while quasi-experimental designs must address group differences and other confounds. Qualitative methods such as interviews and observations can explain experiences and context, though they answer different questions from an effect estimate. Single-case designs examine repeated measures for an individual or small number of cases under planned conditions. Program evaluation asks whether a defined service is implemented and produces useful outcomes in its local setting.

A school team wants to know whether a new check-in routine improves assignment completion. Before looking at outcomes, define the routine, the target behavior, the observation period, the comparison or baseline, and how missing data will be handled. If several supports start at the same time, it may be impossible to attribute change to one feature. If completion rises during the semester, maturation, teacher prompts, schedule changes or seasonal factors could explain some change. Document design limits alongside the result.

Research is not a contest in which one design is always best. The question, feasibility, ethics and context matter. A randomized study may be inappropriate or impractical for a local service decision; descriptive data may still help a team improve access. A qualitative interview can reveal why a process is difficult even when it cannot estimate prevalence. Select the method that fits the decision and describe what it can and cannot establish.

Apply findings without overgeneralizing

When reviewing a psychoeducational study, examine the population, setting, intervention, comparison, outcomes, duration and implementation. Ask whether the students resemble those served locally, whether the outcome matches the concern, and whether the intervention was delivered as intended. A group average can mask varied responses. A program that showed an effect in one setting is not guaranteed to work for every student or school. Research evidence informs professional judgment; it does not replace assessment, family input, or attention to individual needs.

Example: A published study finds improved attendance after schools used family outreach and a daily reminder system. Before recommending the same approach, examine how attendance was defined, the population, implementation intensity, comparison conditions and possible unintended effects. A student who misses school due to medical treatment may need a different support from a student who is avoiding a class because of bullying. The research can generate options, while the local assessment clarifies fit.

For program evaluation, specify the intended outcomes, implementation indicators and decision rule before collecting results. A program may be delivered faithfully but not improve the target outcome, suggesting the theory or fit needs review. Alternatively, outcomes may improve while implementation is weak, which makes replication uncertain. Evaluate both process and outcome. The exam’s best answer usually links a clear question, defensible evidence, cautious interpretation and a next decision.

Common research and statistics traps

Do not treat a statistically significant result as proof of large practical benefit. Do not infer cause from correlation. Do not compare scores from different scales without understanding their meaning. Do not assume that a standardized measure is culturally neutral simply because it has norms. Do not treat a study’s sample average as an individual prediction. Each of these errors exceeds what the evidence supports.

A common case asks whether a school intervention worked. If the team measured attendance before and after, note whether attendance definitions stayed stable, whether the group changed, whether the intervention was implemented, and what else changed. A simple before-and-after result may be useful for local monitoring but has limitations for causal attribution. The appropriate answer does not dismiss the evidence; it explains what further information would increase confidence.

Use a concise reasoning sequence: define the construct and decision, inspect the measure and design, identify uncertainty and context, interpret at the correct level, then recommend an appropriate next step. This avoids jargon for its own sake. Terms such as reliability, validity, effect size and correlation should clarify the reasoning. If a number does not change the recommendation or answer the question, explain why it matters before using it.

Practice with a research decision

A district compares two schools’ average behavior-referral counts and finds one school has more referrals. A sound first response checks enrollment, reporting practices, time periods, student opportunity, school climate, definitions and available supports. Raw counts cannot show which school has a higher rate unless denominators are considered, and rates still may not reflect comparable conditions. Ask whether the same behavior is documented in the same way and who is most affected.

Next, combine quantitative review with appropriate interviews or observations. A referral trend may identify a pattern; student and staff perspectives can help explain where or when it occurs. Protect confidentiality and avoid naming individual students in a public report. If the goal is to test a change, define the new procedure, train staff, monitor fidelity and examine outcomes over time. Any conclusion should match the design and data quality.

For study, practice distinguishing what the information directly shows from what it suggests. ‘School A recorded more referrals’ is an observation. ‘School A has more student misconduct’ is a broader interpretation that may not be supported. ‘A policy change caused the difference’ is causal and requires stronger evidence. This separation is central to responsible school psychology and to the first competency in the FDOE outline.

Common questions

Does correlation prove that an intervention caused change?

No. Correlation shows association. Causal conclusions require a design that addresses alternative explanations and other relevant limitations.

What is the difference between reliability and validity?

Reliability concerns consistency under specified conditions. Validity concerns evidence supporting a score interpretation and its intended use.

What does program evaluation add to outcome measurement?

Evaluation asks whether a defined service was implemented and produced useful outcomes in its setting. Review both implementation and results, and describe design limits before attributing change to the program.