Preference Assessments – Test Retest Reliability
In follow up to our last post Preference Assessments – What Are They Measuring we’re going to take a quick look at one source of contention when you land on the side of seeing preference as primarily a nature phenomenon as compared to a nurture one. That point of contention is test retest reliability.
Test retest reliability for any assessment is important since you want to have a decent level of confidence that the assessment is reliable over periods of time and results are not affected by random chance. This also means that what is being measured by the assessment should be stable over those same periods of time.
If you assume that preference is innate then you would hope to have an assessment measuring these innate preferences to have perfect test retest reliability. Since no preference assessment has perfect test retest reliability, claims that the assessment is measuring innate phenomena simply open up the door for endless criticism of the assessment. Criticism that in our view is justified. For an example of such criticism you can review this article that pops up quite early in a Google search of the MBTI test retest studies.
The unfortunate thing that happens however is that articles such as the one above tend to attack the assessment rather than the assumption of innateness. This pattern of interaction adds very little value in making these widely used assessments more focused, and potentially valuable.
If you move away from the nature vs. nurture positions and try to balance the perspectives of psychology and social construction you can engage in much more useful conversations about what value these assessments can add.
As noted in our last post we see preferences emerging from the left hand loop of the process above and assessments are measuring recognizable patterns of interaction that have emerged over time. These patterns have a certain level of stability so a decent level of test retest reliability in an assessment is important. These patterns are also changeable as everyone can attest to who has ever made some type of behavior change. Given the actual experience and history of preference assessments regarding test retest reliability, it seems they are picking up this experience of change. The stories about these changes (or lack of them) have real value, value that these assessments can add to.
If you assume a greater role of social construction in the emergence of preferences many of the statistical arguments can be left behind and greater focus can be put on how these assessments can add value to our present interactions.
Author – Tom