Online Personals Watch Exclusive Interview #8 - True.com's Chief Psychologist

OPW INTERVIEW -- July 13, 2005 -- Eric Straus of Cupid.com thinks matching based on personality profiling is B.S.  Online dating upstart, True.com, would beg to differ.  Mark Brooks interviewed True.com's Chief Psychologist, Dr James Houran, to get a second opinion on personality profiling for online dating.   

Tell me about your background Jim?
I’m first and foremost a researcher, but I worked as a mental health counselor with both adults and children in a hospital setting for 6 years. That experience motivated me  to complete my masters in clinical psychology. My focus was how attitudes influenced behaviors, as well as the interplay among imagination, cognition, and personality. After my M,A., I was hired as faculty at the SIU School of Medicine’s Department of Psychiatry  There, my principal clinical and research interests involved advanced psychological testing, gender differences, mental health and wellness, as well as relationship quality.  I worked with couples as well as families in both platonic and romantic contexts.  I completed my Ph.D. became known for my work on the validity of psychological testing.  Unfortunately, many test methodologies used today are outdated.  As a result, much of what we think we know in the social sciences is probably skewed or downright wrong. 

Why did you join True?
True.com gave me opportunity to do what I was already doing – a combination of research, clinical work, and public education – but on a scale that psychologists can only dream of.  Usually in the academic community we worry about funding and finding large samples of research participants.  Now I can conduct state-of-the-art online testing where funding is not a problem and which can make a difference in people’s lives on a topic that touches us all.  So, I could not imagine a better job. Also, I’m able to help lead and pioneer the next era of online testing technology.  Very exciting!

How can you be sure True.com’s tests are true and correct?
Luckily we don’t have to rely on opinion.  There are professional standards for testing, which are outlined in a manual authored by the American Educational Research Association, the American Psychological Association, along with the National Council on Measurement and Education.  One key element in test creation is in providing the test rationale. It’s also important to create and validate tests in the context they are to be used, that is, online tests can’t be created and tested offline as a pencil and paper measure and then transferred online.  True.com uses Item Response Theory based measures.  This is the same approach used by the GRE, MCAT and LSAT.  We’ve known since 1960 that it’s superior to Classical Test Theory methods.  All good tests have a test manual to show their basis, construction and validation… and preferably the research should be published in a peer-reviewed journal or at least independently audited by an expert in modern psychometrics.  To my knowledge True.com is the only company that has done this.  And, that angers me, because I hear people on TV saying, “Trust me, I have a test based in science, trust me.”  Ask to see that science and you don’t get a response back. 

Eric Straus, the CEO of Cupid, says personality profiling for online dating context is B.S.  What do you think?
I would agree to some extent. The public should be skeptical!  Public exposure to professional testing is limited.  The public is more familiar with fun little quizzes in Cosmo or on entertainment websites.  These tests are fun diversions but they are not the real thing.  But, can we identify the variables that are associated with long-term compatibility?  Yes, we can.  Can we measure these variables in a reliable and valid way?  Yes, we can.  Can we use and apply these variables?  Yes, we can.  Published research has already shown this.  However, what we’ve learned from advanced statistics is that things we thought we knew often turn out to be skewed or wrong.  For example, eHarmony’s test says ‘birds of a feather flock together.’  The more similar a couple is the happier they will be in the relationship. That’s incorrect. “Similarity” is a relationship principle that academics have long known from research to be oversimplified.

I can’t make sense of eHarmony. They claim their test is based on an impressive study of 5,000 married couples.  I can’t find that study anywhere.  Last February, eHarmony published a general description of their services in a psychology magazine. Several ‘Letters to the Editor’ came in from professionals criticizing the company for unsubstantiated claims of scientific testing. eHarmony has yet to show the public anything of substance.  The only attempt I’ve ever seen is a paper presented a year ago at a psychology conference. This paper reported research on two sets of married couples. One sample of marrieds met on eHarmony, while the other sample met in the real world.  The conclusion was the eHarmony couples had higher levels of relationship satisfaction.  As we reviewed their research it was apparent that the study and its conclusions were wrong.  A rebuttal was published in the North American Journal of Psychology. This rebuttal showed that eHarmony’s own data contradicted the idea that similarity is the best predictor of relationship satisfaction.  The media is also now starting to question the legitimacy of eHarmony’s testing. Test manuals or any proof of testing research are also absent from Perfectmatch, Tickle, and Yahoo Personals.  Personality profiling tests can indeed be harmful if they are not based in legitimate research.  Many people take these test results to heart and they influence life-changing decisions.  If a company says they use tests based in science they have a legal and moral responsibility to consumers need to back that claim up. 

How can a company give out enough information on these tests to win confidence, without giving away their secrets?
Independent auditors who are established experts in modern test construction and validation can review the tests and certify that they meet professional testing standards. Data can be reported to the public without divulging the specifics of matching algorithms. Take drug companies, for example, you don’t see the specific recipe for the chemical makeup of drugs or detailed explanations of how those drugs work exactly. Yet, drug companies routinely sponsor and publish research on their efficacy of their products.

So what can you tell me about True.com’s matching algorithm?
True matches people based on research concerning similar as well as complementary  relationship variables. We take into account factors well beyond mere personality. Some of those variables include lifestyle preferences, world-views, sense of humor, social life, money management, and readiness to commit. Companies are free to use their preferred methods to pair individuals. Perfect Match uses a test derived from the popular Myers Briggs Type Indicator.   We use what’s known as the ‘big five model.’

eHarmony and Perfectmatch make all their users take their profiling tests.  Why doesn’t True.com?
Not all users want to take a compatibility test for many reasons. Users may be skeptical or just not want to spend the time.  If people are antagonistic about being forced to take a test they won’t give accurate responses.  Compatibility testing implies long-term relationships, and True.com realizes that not everyone is looking for that exclusively. Our research indicates that the bulk of the online daters do not want exclusively short-term or long-term relationships. We’re offer tools to help them find the relationship that is right for them. Our vision is to reduce the divorce rate by pairing people up for good relationships and marriages, as well as by preventing bad marriages in the first place by not prematurely pushing people down a path they’re not comfortable with. 

How is True.com planning to help move the industry forward over the next 6 months?
You’ll see more of a push towards educating people on the power and value of good science-based products.  True.com is committed to the safety and security of all our members and to give them the right tools so they can find the right person.  We’re also working towards answering the second half of the problem.  Once we pair people up… then what?  We’re developing even more tools like supplemental tests and relationship guides to help people maintain and nurture relationships. Basically you’ll see more of a hand-holding approach.  We’re uniquely positioned to help people with other aspects of their life.  We haven’t painted ourselves into a corner to be simply an online dating site. Our tagline is “Live, Love, Learn”’ for a reason.  Love is one element of a person’s life.  We’ll also branch out to help people with self-growth and platonic relationships.  So, you’re going to see us increase in scope.

Comments

[Fernando Ardenghi]

Interesting interview!!

Mr. Brooks, please, try to contact also:
-Dr. Neil Clark Warren (psychologist), from eHarmony.
-Dr. Mark Thompson and Dr. Glenn Hutchinson (psychologists) from WeAttract.
-Dr. Pepper Schwartz (sociologist) from PerfectMatch.
-Dr. Glenn Wilson from Cybersuitors (English for UK).
to interview them.

Kindest Regards,

Fernando Ardenghi.
Buenos Aires.
Argentina.
[email protected]

[David Evans]

The next generation of matchmaking is not scientific, it's about people matching people. This is not to say that scientific matchmaking does not play an important part of the introduction process, you need the critical mass first. People are looking for something new, a twist on the familiar zip code height, weight search routine. I predict this will drive membership much more effectively than more thorough tests, except in a few cases.

I find it quite interesting that James is openly admitting that personality profiling for online dating context is B.S. to some extent.

Even if I disagree personally with his stance on testing, I applaud his getting out in front of the issue and at least taking a stand. Then again, Weattract has no reason to make waves, they've already got the big prize (Yahoo). I can't find the Eharmony study either but then again it took me 6 months to find the True test.

Members do not care much about personality testing. This is a indisputable fact. Match had a low single-digit take rate on their WeAttract tests. Will we ever publicly know Yahoo!'s take rate is? Can we rely on honest reportage about numbers, success conversion rates from large dating sites? Of course not, too much transparency is not good for the industry.

Good point about drug companies. Successful online dating companies are good marketers and gain critical mass through media exposure. Nobody knows what Lipitior actually does, they just watch the commercial. Nobody knows what True or Eharmony test are like, they take it because they heard the commercial.

True needs to do a deal with Dr. Ruth and a movie tie-in or there will be no customers to match. Economics of running a dating site come into play here.

Finally, the dichotomy between the stated mandate to reduce the divorce rate and True.com advertising is troubling. Are the cleavage shots scientifically justified?

[Glenn]

Awesome interview and great comments too. I agree with Fernando's list of people who'd be great to hear from.

I also agree with relaxedguy's comments about 'cleavage shots.' I keep a file with true.com ads because it's just so unbelievable to me that they have slogans like "We're busting at the seams with women" with a shot of just women's breasts and then "We're pulling strings for you," with a closeup of a string bikini. I asked my brother if these ads made sense to him and he said, "It's just marketing, you have to do what gets guys to your site." To me it's tasteless, but I watch with amazed curiosity anyway just because I can't believe someone would try it and I'm curious to see if it works.

I think the answer to "how can you tell that tests are true and correct" doesn't have anything to do with independent auditors or scientific journals. The answer should be based on the ratio of couples matched who find love to those who were incorrectly matched. If a test is published with all the publishing background in the world it still won't hold a candle to one that accurately predicts who will fall in love with whom.

I personally have met many of the matches that the true.com test has made for me and many matches that eharmony made for me and neither has found Ms Right for me yet despite the lack or abundance of literature and scientific review. Imagine if Dr Houran's answer to the question "How can you be sure True.com’s tests are true and correct?" was "The couples we match fall in love with each other twice as often as the couples eHarmony matches." THAT would be a selling point and would make me send a few more winks on their system again tonight... even though twice eHarmony's rate would be a 1 in 500 chance instead of the 1 in 1000 chance Fernando calculated in a previous thread on eHarmony.

- Glenn Gasner

[Fernando Ardenghi]

The next generation of serious dating and matchmaking will be more scientific than ever.

2005: The Flight to Quality process started.

By 2008: Quality Norms ISO 9001:2000 & Legislation will be expected. The market will divide into two well-identified groups.

*13-25 years old persons (teenagers) not interested in serious dating; they will use the service for fun. They will need exclusive CONTENTS for them. They will also play "people matching people game"; although there is a hierarchy between human relationships where feelings are involved; and could be very dangerous and harmful like a "big liquidizer".

*26-and more years old persons interested in serious dating. They will need quality CONTACTS (compatible real persons), Special Services (professionalism): the next generation of dating and matchmaking will be more scientific than ever, most probably the 16PF5 test in different languages (or similar test) will be a "must have" in compatibility matching. This group will not allow their friends / mums / parents / neighbours / relatives or "interested third parties" to be involved in a private matter as building a personal relationship with future in mind; a process that only concerns two persons.
These clients will have a good reason to pay for the service: avoid being hurt in their feelings by other persons.

Actually, many people try to "improve" their profile, to "retouch" photos and to "exaggerate" when filling likes and dislikes formularies in order to "sell better" their profile. People trend to overstate their qualities, they do not say the exact truth. They can and they will lie and also if the membership is FREE they can appear several times with several profiles.
Lucky, a personality test like 16PF5 works as an independent evaluation of a person; an external point of view.

I noticed that many users / subscribers (to sites that use proprietary tests or models) complaint about an actual big problem in "scientific dating and matchmaking": lack of precision / low precision / low successful matching rates. As far as I could analyze, it seems that proprietary tests or models have great precision in measuring different psychological variables but the matching algorithm has low precision when comparing one psycho-pattern to others.
It could be a great improvement if any dating site that uses "scientific dating and matchmaking"; i.e. applying psycho-test to measure personality items; specifies its ENSEMBLE (the whole set of different valid possibilities):
Example:
- Only Myers-Briggs Type Indicator (MBTI) test: 16 different possibilities.
- Improved MBTI with photos, likes-dislikes forms will have more possibilities.
- Big-5 dimensions of personality with 10 degrees per independent variable at the results of the test: 100,000 different possibilities (personality types).
- Complete 16PF5 test with 10 degrees per primary variable, 10,000,000,000,000,000 of different possibilities.

And a figure per each comparison between persons, like: client#01 to client#02 == 74.79865772%
and not a poor bar-graphic or a series of 5 empty / half / full-filled hearts-icon.

Kindest Regards,

Fernando Ardenghi.
Buenos Aires.
Argentina.
[email protected]

[Michael Chung]

Nice thoughts, Fernando. I like where this is going, though I see our progress to date as embryonic at best. But we must begin somewhere, right?

The biggest problem is that our understanding of relationship compatibility is extremely rudimentary. The major hole I see in all of this scientific modeling is Attraction: what causes it, how much of human choice is motivated by this one factor alone, etc.

Once you know why and how two people are attracted to each other, only then is it even worthwhile to explore the nature of compatibility. It is like worrying about designing a flashlight when you haven't invented batteries yet!

[Fernando Ardenghi]

"Once you know why and how two people are attracted to each other, only then is it even worthwhile to explore the nature of compatibility. "

Dear Mr. Michael Chung:

At the United States there are many persons researching on
THEORIES OF ROMANTIC RELATIONSHIPS DEVELOPMENT
Unfortunately, NOT ALL of them agree between themselves!!!

You could see (as an example):

Change Assortative Mating and Marital Quality in Newlyweds: A Couple-Centered Approach, February 2005 at "Journal of Personality and Social Psychology"
Page #19 of PDF whitepaper
Page #322 of magazine
"................................
Couple Similarity and Marital Quality
Our study presents one of the most comprehensive examinations of the similarity–satisfaction link. The observed similarity–satisfaction correlations suggest that similarity on personality related domains was strongly associated with satisfaction,
......................."

"Personality, Family History, and Competence in Early Adult Romantic Relationships"
that says at page #574
"..........
For example, recognizing that behavior in relationships is partially rooted in relatively stable personality traits can help explain why relationship interventions are sometimes unsuccessful. It may be more difficult for some people to change how they act in relationships because these behavior patterns are manifestations of more general and pervasive behavioral tendencies.
..............
....the common sense notion that the personality characteristics of a romantic partner are an important factor to contemplate when considering the viability of a long-term romantic union .
......................"

"Hurrydate: Mate preferences in action" Could be download from
Interesting to see: If its conclusion only shows "people's INFATUATION and FANTASY"! i.e. The conclusion of this paper can not be applied to any serious dating method.
Why U.S. divorce rate is over 47%? Something wrong is happening?

"Two Personalities, One Relationship: Both Partners' Personality Traits Shape the Quality of Their Relationship"

Also a THESIS that could be of your interest.

ROMANTIC REGRESSIONS
An Analysis of Behavior in Online Dating Systems
"Findings: Users opted for sameness more often than chance would predict in all the characteristics examined in this section. This concurs with the overwhelming evidence gathered by relationship researchers (see surveys in Brehm et al. 2002, Fisher 1992) that actual similarity and perceived similarity in demographics, attitudes, values, and
attractiveness correlate with attraction (and, later, relationship satisfaction). However, users demonstrate this homophily to differing degrees for different characteristics." can be read at page 47.

Fernando Ardenghi.
Buenos Aires.
Argentina.
[email protected]

[MARGARET WATSON]

TRUE IS SENDING WHAT I BELIEVE TO BE FRAUDULENT EMAILS.IF MORE INFORMATION IS OF INTEREST MY TELEPHONE IS 703 723 7188

[Joe]

I really just don't think you can come up with a way to automatically fit two people an a match every time. Because personality types differ so greatly not only from person to person but also, any one given person may have different preferences during different stages of life.