Cut to the Core of Me

THE CAVALIER DAILY -- Jan 24 -- Some tests pretend to be legitimate and might make you believe their results are valid and meaningful. The most common being Myers-Briggs which breaks your personality up into four categories. Despite these inadequacies, you can find many other semi-legitimate online personality tests. The Enneagram categorizes people into nine unique groups. The "Big Five" uses five dimensions instead of four. Each test offers a different way to categorize yourself and to see how you measure up to other people. The amount of reliable information these tests provide is highly debatable, but many people still enjoy them.

The full article was originally published at The Cavalier Daily, but is no longer available.

Mark Brooks: Myers Briggs actually has 16 character types based upon four main categories, and the Enneagram has 9 personality types based upon four categories. They are both, essentially, subsets of the Big Five test.  Your comments please...

Comments

[James Houran]

Dear List,

Despite their potential power and value, all assessments have limitations. Both online dating sites and their customers need to have realistic expectations about what types of information assessments can and cannot deliver. The strengths and limitations of a given assessment are based on its technical and theoretical underpinnings.

Below are some important points to remember in this respect:

1). Assessment feedback is derived from mathematical extrapolations of behavioral data. As such, feedback reports describe statistical predictions of what attitudes and behaviors a given test taker will likely exhibit. Mathematical models are consistently more valid than subjective observations, but even the finest assessments are never 100% percent accurate 100% of the time.

2). The validity of a report is limited by the reliability of the test taker’s responses. Test-takers may answer assessments unreliably for a myriad of reasons: lack of motivation or interest due to less than ideal testing conditions or test taker’s mood, fatigue from answering a long set of questions, an attempt to answer questions in a socially-desirable way or difficulty understanding particular questions for linguistic reasons (e.g., when English is not the test taker’s first language).

3). All test scores are statistical estimates. Thus, each score is accompanied by its margin of error [also called a confidence interval or standard of error (SE)]. However, properly constructed employee assessments provide information on the statistical reliability of a particular test taker’s test scores, as well as measure the degree to which a test taker seems to be answering the assessment truthfully.

4). Finally, the quality of an assessment (and hence its feedback) is associated with its methodological and statistical principles:

Self-referential vs. normative instruments: Some assessments provide feedback based simply on how a test taker perceives him or herself. In other words, these instruments describe individuals only in a self-referential way, i.e., against themselves. Examples of self-referential instruments are the Myers-Briggs Type Indicator (the inspiration for PerfectMatch.com's test) and the DISC assessment (offered by Thomas Technologies). By contrast, normative instruments are inter-individual because they describe test takers against a reference group. This approach is significantly more valid than the self-referential approach.

Classical test theory vs. modern test theory: Most assessments on the market today are constructed and validated using classical test theory, which essentially treats all assessment questions as equally weighted “points.” A great example is the assessment offered by eHarmony.com. Such assessments consequently provide a total score that is the sum of those points. This approach has been outdated since 1960. Today, test and measurements experts rely on modern test theory (Item Response Theory and Rasch scaling), which yields unbiased, scaled scores for test takers. Modern test theory is the same gold standard statistics used in such well-known assessments like the GRE, MCAT and LSAT. This approach can identify and remove response biases related to age, gender, cultural background and employment level of the test taker. Besides greater technical precision and the protection of meeting legal requirements, modern test theory also yields richer information that traditional approaches miss.

For detailed scientific information on the realities behind compatibility testing, see:

Houran, J., Lange, R., Rentfrow, P. J., & Bruckner, K. H. (2004). Do online matchmaking tests work? An assessment of preliminary evidence for a publicized ‘predictive model of marital success.’ North American Journal of Psychology, 6, 507-526.

For a lay-person's guide to the subject, see:
http://www.onlinedatingmagazine.com/features/compatibilitytesting.html

Thanks,

James Houran, Ph.D.
Online Dating Magazine