Skip to main content
← Back to Explore
PSYCHOLOGY LIBRARY / Research methods

A questionnaire can be reliable and still miss the point

Reliability, validity, content coverage and population fit are different questions about a scale.

Editorial sample: This reading has not yet undergone Mind Assembly's editorial review. Verify all sources, interpretation and permissions before public release.

Consistency does not establish meaning

A questionnaire may produce scores that are internally consistent without adequately representing the construct researchers intend to measure. Reliability concerns consistency and sources of measurement error; validity concerns the evidence supporting how scores are interpreted and used.

Three practical checks

  1. Content: Do the items cover the intended construct, and are they comprehensible to the target respondents?
  2. Structure: Do the observed item patterns fit the claimed dimension or dimensions?
  3. Context: Has the measure been examined in the population, language, and setting you plan to study?

COSMIN guidance places strong emphasis on content validity when evaluating patient-reported measures. This provides a useful general lesson, although particular standards depend on the purpose of a scale.

Why an established scale may still be unsuitable

Imagine a measure developed among adults in one language being used with adolescents in a different context. Even a high coefficient reported in the original paper cannot automatically show that the items are suitable, understandable, or culturally equivalent in the new group.

A study-planning habit

Before downloading any questionnaire, write down the construct definition, target population, scoring interpretation, development study, and what validity evidence exists for that use. A familiar measure is not automatically the right measure.

Sources & further reading

← Return to reading catalogue