Correlations and likert scales: What is the bias?

A person on ResearchGate asked the following question:

How can I correlate ordinal variables (attitude Likert scale) with continuous ratio data (years of experience)?
Currently, I am working on my dissertation which explores learning organisation characteristics at HEIs. One of the predictor demographic variables is the indication of the years of experience. Respondents were asked to fill in the gap the number of years. Should I categorise the responses instead? as for example:
1. from 1 to 4 years
2. from 4 to 10
and so on?
or is there a better choice/analysis I could apply?

My answer may also be of interest to others, so I post it here as well.

Normal practice is to treat likert scales as continuous variable even though they are not. As long as there are >=5 options, the bias from discreteness is not large.

I simulated the situation for you. I generated two variables with continuous random data from two normal distributions with a correlation of .50, N=1000. Then I created likert scales of varying levels from the second variable. Then I correlated all these variables with each other.

Correlations of continuous variable 1 with:

continuous2 0.5
likert10 0.482
likert7 0.472
likert5 0.469
likert4 0.432
likert3 0.442
likert2 0.395

So you see, introducing discreteness biases correlations towards zero, but not by much as long as likert is >=5 level. You can correct for the bias by multiplying by the correction factor if desired:

Correction factor:

continuous2 1
likert10 1.037
likert7 1.059
likert5 1.066
likert4 1.157
likert3 1.131
likert2 1.266

Psychologically, if your data does not make sense as an interval scale, i.e. if the difference between options 1-2 is not the same as between options 3-4, then you should use Spearman’s correlation instead of Pearson’s. However, it will rarely make much of a difference.

Here’s the R code.

#load library
library(MASS)
#simulate dataset of 2 variables with correlation of .50, N=1000
simul.data = mvrnorm(1000, mu = c(0,0), Sigma = matrix(c(1,0.50,0.50,1), ncol = 2), empirical = TRUE)
simul.data = as.data.frame(simul.data);colnames(simul.data) = c(“continuous1″,”continuous2″)
#divide into bins of equal length
simul.data[“likert10″] = as.numeric(cut(unlist(simul.data[2]),breaks=10))
simul.data[“likert7″] = as.numeric(cut(unlist(simul.data[2]),breaks=7))
simul.data[“likert5″] = as.numeric(cut(unlist(simul.data[2]),breaks=5))
simul.data[“likert4″] = as.numeric(cut(unlist(simul.data[2]),breaks=4))
simul.data[“likert3″] = as.numeric(cut(unlist(simul.data[2]),breaks=3))
simul.data[“likert2″] = as.numeric(cut(unlist(simul.data[2]),breaks=2))
#correlations
round(cor(simul.data),3)

Reviews

Showing 1 Reviews

  • Placeholder
    Sebastian Sauer
    Confidence in paper
    Quality of writing
    Originality of work
    0

    In his article "Correlations and likert scales: What is the bias?", the author E. Kirkegaard argues that as long as more than 5 answer options are used in a Likert scale, bias from discreteness is not be expected substantial.

    More precisely, the argument seems to imply (or, at least, it makes me easy to conclude) that Likert scale items can under some circumstances be seen as practically similar to quantitative (and continuous) variables.

    This idea stems from the fact that the author argues that correlations of Likert scale category scores with a quantitative variable are very high. Simulation code is given to back this claim.

    In sum, the authors follows a sensible and quite novel approach, novel at least as far as literature regarding measurement is involved. The simulation approach chosen (implemented in R) is straight forward. It provides a fresh look at the question of whether/when/to what degree Likert scale items may be regarded as quantitative. The author demonstrate robust knowledge of empirical data analysis as far as one can tell from this work.

    The author is rightly pointing out that Likert scale items are often taken as quantitative, but that practice is misguided.

    However, in my opinion, the authors misses one crucial point. That is, by the approach chosen, equidistance of the Likert intervals is silently assumed. Technically, the "cut()" function cuts a vector of equal size.

    That means it must necessarily follow that the discretized variables correlate well with the original variable. It is by design necessary. However, it cannot be deduced that in general Likert items are as good as quantitative variables.

    For example: Take a vector (x), consisting of the numbers from 1 to 100. Cut the vector in 10 equal parts; let's call the resulting vector x2. Now correlate x and x2.

    Of course, x and x2 will correlate strongly. 

    Now try the contrary. Cut the interval in unequal sizes. The resulting correlation will be lower as the first correlation, as the distances between the categories are not equidistant any more.

    R code:


    x <- 1:100
    x2 <- as.numeric(cut(x, 10))
    cor(x, x2)

    breaks_ <- c(0,4,6,8,10,12,14,30,70,100)
    x3 <- as.numeric(cut(x, breaks = breaks_))
    cor(x, x3)


    Contrary to what may be implied by the article, Likert scales are by no means sufficient to allow for a quantitative conclusion. The number of categories in a Likert scale is (in general) no guaranty that we can infer quantitative relations, ie., equidistance between adjacent categories which resulted in the high correlation here.

    Rather, quantitative relations need to be empirically investigated. A well known historical example is temperature measurement. They cannot be automatically assumed. History tells us that sophisticated measurement apparatus may be needed before quantitative measurement can be discovered (or disproved).

    It has been lamented that psychologists (and other social scientists) appear not to bother about testing quantitative assertions (see this and other papers of Joel Michell). This is a pity, because measurement is at the very fundament of most empirical sciences including psychology.

    There exists a sophisticated theory (theory of conjoint measurement) for testing quantitative relations given that ordinal relations hold. This theory can be employed for such claims.

    A good introductory text to quantitative measurement can be found in this text book.

    I added some related comments to a recent blog post here.

    This review has 1 comments. Click to view.

License

This article and its reviews are distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and redistribution in any medium, provided that the original author and source are credited.