Research Methodology: Measurement Scale and Level, Reliability & Validity

Research Methodology Measurement Scale & Level

Level of measurement

A deadly magnitude 7.3 earthquake had struck Nepal in April 2015. GST is charged and levied on any supply of goods and services made in Malaysia at 6%.

  1. Nominal
  2. Ordinal
  3. Interval
  4. Ratio

Scales of measurement

  1. Nominal scale
  • Categorical variable
  • Represented by number
  • E.g., gender, employment status, religion, etc.
  1. Ordinal scale
  • Rank variable (highest to lowest…)
  • Distances among scales are different
  1. Interval scale
  • The difference between the two values is meaningful
  • Distances among scales are identical
  1. Ratio scale
  • True zero
  • Differences between values can be compared meaningfully

Scaling

Needs to be:

  • Unidimensional
  • Linear (equal intervals)
  • Reliable
  • Valid
  • Research Methodology Measurement Scale

Research Methodology Measurement Scale

Simple Rating Scale
  • (1) to rate list of items or (2) to rate how much respondents
  • like smtg.
  • 1-3? 1-5? 1-7? 1-10? 1-100?
  • For greater differentiation
  • The higher the number, the more you agree/like
Thurstone Scale
  • Psychologist Robert Thurstone
  • Equal-appearing intervals
  • 11-category scale
  • Neutral statement
Likert Scale (summated rating approach)
  • Psychologist Rensis Likert
  • Strongly agree, agree neutral, disagree and
  • strongly disagree
Semantic Differential Scale

How would you rate your mother on these scales?

Validity

  • “the degree to which a test measures what it is supposed to measure”
  • Fundamental consideration

Reliability

  • “the degree to which a test consistently measures whatever it is measuring”
  • Fundamental consideration.
Types of validity
  • Face validity
  • Content validity
  • Criterion-related validity (1. concurrent validity & 2. predictive validity)
  • Construct validity
  • Discriminate validity
  • Consequential validity
  1. Face validity
  • Subjective assessment / a subjective judgment
  • For instance, you might look at a measure of math ability, read through the questions, and decide that YES, it seems like this is a good measure of math ability (i.e., the label “math ability” seems appropriate for this measure)
  • Weak validity BUT IT DOES NOT MEAN IT IS WRONG

2. Content validity

  • To which a test measures an intended content.
  • TWO types: item validity – whether the test items are relevant to the measurement of the intended content area & sampling validity
  • –how well the test samples the total content area being tested.
  • E.g., being a good science teacher
  • A test score cannot accurately reflect a student’s achievement if it does not measure what the student was taught and is supposed to have learned.
  • Determined by expert judgment (content validation)

3. Criterion-related validity

  1. Concurrent validity

“the degree to which scores on one test are related to scores on a similar, preexisting test administered in the same time frame or to some other valid measure available at the same time.”

  1. Predictive validity
  • “the degree to which a test can predict how well an individual will do in a future situation.”
  • For example, the Graduate Record Examination (GRE), International English Language Test System (IELTS), etc.

4. Construct validity

  • “What is this test really measuring?”
  • Constructs underline the variables that researchers measure. You cannot see a construct; you can only observe its effect.
  • convergent validity – Correlation test (Pearson)
  • discriminate validity -Factor analysisConvergent Validity

Reliability

Dependability or trustworthiness

  • “the degree to which a test CONSISTENTLY measures whatever it is measuring”
  • Is expressed numerically – reliability coefficient (correlation) *1.00
  • Internal consistency reliability
  • Cronbach’s alpha 70% and above
  • Test-retest, Kuder-Richardson 20 (KR 20), split-half reliability