Declare the past, diagnose the present, foretell the future; practice these acts. As to diseases, make a habit of two things – to help or at least do no harm.

– Hippocrates, Epidemics Book I

Parents are at their most vulnerable when their child is struggling. They want information, and they trust experts to tell them what is wrong and to give advice about how they can make it better. But they need to be very careful about finding the information they desperately seek and the advice they fervently want: there is an art and a science to testing, and results are not always what they seem. My first experience with testing was as a parent, not as a professional, and it is an experience I will always remember.

When my daughter, Jessica, was about twenty months old, she had her first IQ test. She sat on my lap as the test administrator brought out the first items: blocks! Jessica enthusiastically grabbed the blocks and started to build, but the tester stopped her and told her to make her blocks look like the tower the tester had made. Dubiously, Jessica made the tower as requested, then began to make her own construction again. The test administrator, however, had checked off her item and no longer needed blocks. She grabbed the blocks from in front of Jessica, put them away, and pulled out the next task materials: a cup and a small ball. Jessica was upset at having the blocks taken, but was intrigued by the ball and cup. The administrator covered the ball with the cup, and asked Jessica to find it, which she did. Then she pulled out a couple more cups and began to hide the ball again. Jessica liked this game, and after finding the ball again she was ready to play by hiding the ball from the administrator. However, the administrator had checked off her item, and put the ball and cup away. Needless to say, after a few of these encounters, Jessica was having none of it. In fact, as the administrator put the new materials in front of her, Jessica simply picked them up and threw them at the administrator. (Even twenty-six years later, Jessica is not known for her ability to suffer fools gladly.) Pretty much every item from then on was marked with a zero. Of course, the test results indicated that Jessica had a very low IQ, as she was “unable” to complete these items.

Obviously, this test administrator had not mastered the art of testing. She was testing something, but it was not what she had meant to measure (Jessica’s intelligence). She may have been testing Jessica’s patience or compliance or ability to follow directions, but she was not testing whether or not Jessica could figure out the puzzles she was placing in front of her. Luckily, I was in the room and watched the whole proceeding, so I knew not to trust the results. If I had not been there, though, as a young mother with a disabled child, I would have been devastated by these results. Here was an “expert” who was telling me my child was very mentally retarded.

When Jessica was six years old, she was again tested, this time by the public school system’s educational psychologist in the US. I don’t know if this person had mastered the art of testing, because this time I wasn’t in the room. But I was the person to whom the psychologist explained the results. Jessica had taken the WISC, an IQ test that has several subtests and subscales. On some of them, she had scored above the 95th percentile, in the gifted range. On others, she had scored below the 5th percentile, in the severely impaired range. When you averaged the high scores together with the low scores, the result was 100 – a perfectly average score. The school psychologist (Dr. Somebody) explained to me that the school couldn’t possibly make special accommodations for Jessica because she had a perfectly average IQ. When I pointed out that she was not average in any area – she was either gifted or impaired – he told me that it didn’t matter, because it all averaged together. Again, I was lucky in that by that time I was running an evaluation center at Arizona State University and was clear on the fact that this man didn’t know what he was talking about. He didn’t understand the science of testing. So I moved Jessica to a private school that would address her special needs.

THE ART

In order to test kids, it is important to like kids, to be interested in kids, and to understand how to elicit the best performance possible from kids. The skills every teacher needs – patience, a sense of humor, a bit of firmness, a touch of playfulness – all are important in fostering an environment where a child feels comfortable. If a child does not feel comfortable, the child will not be performing at his or her best. And if the child is not performing at his or her best, then what are the tests measuring?

The tests used in a psychoeducational assessment are complex enough that there is really only one way to score well on them – to score well, a child must have the necessary abilities or skills. But there are always many reasons that a child might score badly on these tests. The child could have a cold, or be overly tired, or be shy or scared or overexcited. He or she could have had a fight with a sibling or parent that morning, a cranky car ride, or a nightmare. I once tested a teenager who (as I found out after the testing) was hung over. Emotional problems such as depression, anxiety, or obsessive-compulsive behaviors might affect test results. (Has everyone seen the episode of Monk in which he takes his police exam?) In other words, there are many ways in which psychoeducational testing can underestimate a child’s true abilities or skills. Part of the art of testing is to recognize (and note down) the things that might be interfering with a child’s best performance and to minimize those things if possible. If the child is old enough, the test administrator should always attempt to assess emotional problems that may be interfering with test results at the same time as he or she measures problem-solving and achievement.

THE SCIENCE

There is a lot to know about testing, and most of it does not concern the actual administration or scoring of the tests themselves. In fact, these tests are beautifully designed, so most of them come with very simple-to-follow manuals, scoring guides, and even computer writers so all the administrator must do is enter the scores in the computer and push “return.” Anyone reading this article, really, could administer and score these tests with a very small amount of training, just as we could all probably look up medications in a Physician’s Desk Reference and find out about how to treat arthritis. So why should only a licensed psychologist supervise and interpret these tests and only a medical doctor prescribe drugs?

Obviously, a good test administrator should know where the tests come from (test construction) and what the statistics mean. That’s just basic. Otherwise, the results cannot be explained and the test itself cannot be evaluated. For example, it seems that the psychologist who tested Jessica when she was six did not understand how the test was constructed to reflect different ability areas that could not be averaged if they were highly divergent.

A good test administrator also has to know what factors are being measured by each subtest, and this is important because the administrator then has to rely on his or her art to eliminate all other factors. For example, there are many subtests that purport to measure processing speed. For older kids, these are fairly straightforward. Younger kids, however, do not always understand the concept of time or what they are doing in a “race” to see how much can be completed in a given period. They can do the practice exercises (4 or 5 of them) very well and very quickly, but if you turn the page to something that contains a hundred problems and say, “Let’s see how many you can complete in a minute,” the child looks at the page and says, “That’s too many. I can’t do that.” He doesn’t understand the concept of racing the clock. (This would be similar to entering a library and telling the child that we were going to measure how many books he can read in a minute, only to have him look at all the books in the library with dismay and say he just can’t read all those books!) This being the case, it is useful to play with the stopwatch with younger kids – to see how many times a child can run across the room, or to allow the child to find out how many times the tester can pat her head, or how many times we can both hop up and down in fifteen seconds. With a few of these exercises, the child is much more eager to begin a longer process and see how many problems can be completed in a minute. If we simply allow the child to moan that there are too many problems, we may be measuring maturity or time-concept, but we are not measuring processing speed, which is the purpose of that subtest.

In another example, there is a subtest that I often use that purports to measure working memory. The test is presented on a tape, and has longer and longer strings of items that must be manipulated and remembered. The tape leaves a space of time for the response, but only allows for the amount of time typically taken by the average child. If a child is working slowly, he or she might bump into the next item on the tape before having completed the response. The subtest’s purpose is to measure working memory – not response speed. In order to correctly measure working memory, the administrator should pause the tape if the child is taking longer than expected. Otherwise, the child becomes anxious or worried, tries to hurry, and/or simply doesn’t complete the task in the time allotted. If this happens, the test may be measuring something but it will not be measuring working memory. There are actually many test administrators who fail to pause the tape, thereby completely mismeasuring the true working memory of the child. Their reason? The company that produces the test made the tape and therefore they feel they are following the dictates of the company. But the truth is that the company that makes the test explains in its own training sessions that the timing on the tape is arbitrary and that the administrator should (of course) pause the tape if the time is not appropriate for a child. Again, it is important to understand the factors being measured, rather than to simply follow the instruction booklet (or tape) blindly.

Of course, for comparison purposes, there are some very strict guidelines that must be observed in any standardized testing situation. The trick is to know what can be added in or changed, remembering always the basic purpose of each subtest. Jessica’s first test administrator, for example, would not have been bending the rules had she brought out the cup while Jessica continued to play with the blocks and, with a little enthusiasm and playfulness, gotten Jessica interested in the new task before putting away the last. Similarly, playing with a stopwatch before testing does not interfere with the standardization of the processing speed tasks. It simply encourages the child to attend to them.

Once the administrator has appropriately approached each specific factor being measured, however, it is also important that he or she understand the factors and how they relate to each other and how they relate to cognitive development and learning. The field of cognitive development is complex and interesting, and doesn’t just consist of a bunch of numbers in a chart. There are many theories and much research concerning the development of thought and thinking abilities. How does working memory develop, and how is it related to learning new materials or to emotion? How is auditory attention related to general intelligence? Is the relationship the same at all ages? Would we expect the same range of phonemic awareness for eighteen-year olds as we would expect for six-year olds? These may sound like esoteric issues, but in fact they can make a big difference when interpreting the numbers or making a diagnosis!

And, finally, if the test administrator is going to give advice to parents and schools, he or she has to know a little about education. What are the pedagogical controversies surrounding dyslexia, for example? How can a teacher’s job be made easier, not harder, by the recommendations? What is normal behavior in a classroom, and what can we do about situations or needs that are not normal? So, if working with teachers, the psychologist needs to have had significant training in or experience with schools. Similarly, if advising parents, the administrator needs to have had significant experience in the field working with children and/or be a parent.

NECESSARY BACKGROUND

If we look at what is needed to learn the art and science of test administration and interpretation, we can see that the necessary qualifications are fairly easy to define. A test administrator has to have:

1. A strong background working with children. The test administrator has to have mastered the art of playing and teaching with kids. This means that the administrator must, at the very least, have a strong and long-lasting interest in kids that has shaped his or her education and career. Work in schools or children-centered facilities is a good indicator of longstanding interest.

2. A strong understanding of cognitive and emotional development and how children think, learn and behave differently at different ages, as well as behavioral and emotional development across different ages. (What could be diagnosed as psychotic in an adult, such as a fear that vampires are in the tennis courts, might be a normal fear for an eight-year old!) This understanding is best achieved through a Ph.D. in a child-centered area of psychology.

3. A strong understanding of tests and measurement, what each test is supposed to measure, and how to best elicit a measure of each factor. The best way to achieve this understanding is, again, with a Ph.D. in an area of psychology related to child cognitive development.

4. If the administrator is then going to give advice to schools and teachers, he or she must have some understanding of what is possible in a classroom. This is best achieved either through direct experience in a classroom (as a teacher) or through having worked directly with teachers in a school setting.

5. If the administrator is going to give advice to parents, he or she should have, again, a strong background working with families and/or have experience as a parent. As every parent can tell you, what is in the textbooks about parenting is not always what happens in the home – nor should it be. Young and enthusiastic graduates, even in child psychology, usually have many ideas about what good parents should do and how this will create perfect children. When they become parents, they often learn that their children haven’t read the textbooks.

Of course, psychologists don’t always give all of the tests in an assessment themselves. Quite often they have hired trained administrators, graduate students or psychometrists who can administer portions of the tests for them. In this case, it is very important that three conditions are met: First, the test administrator must at least have the first quality listed here – a strong background with and interest in children. Someone with an advanced degree in child psychology or education fits this criterion for example. Someone with a background in business administration does not. Second, the test administrator should be specifically trained as to the particular tests involved, the meaning or purpose of each test, and the types of things that might interfere with a correct assessment. Third, while it is not necessary that the psychologist administer all the tests, the psychologist should spend at least some of the testing time in direct administration of tests in order to form a personal understanding of the child and to be able to use professional judgment in the interpretation of results.

QUALIFICATIONS

Educational qualifications are confusing to begin with, but become even more confusing in our multi-cultural setting. Parents may have to do some exploration to find out the background of an individual practitioner. Don’t take the word “Dr.” and the fact that the person is performing the tests to mean that the practitioner has the background described above. A medical doctor, for example, does not study educational testing or cognitive development as part of his or her training, any more than a psychologist studies heart surgery.

But it is more complicated than simply finding a psychologist. For example, in the United States, there is only one qualification for “Licensed Clinical Psychologist,” but many paths to that qualification. To get the qualification, a person must have a Ph.D. in psychology, then do a two-year internship, then take a national exam that covers all areas of psychology, then take local state boards. Sounds like a lot, right? But there are many types of psychology. A person could get a Ph.D. in Industrial/Organizational Psychology (focusing, for instance, on how to maximize employee satisfaction or productivity), do a two-year internship at Merrill-Lynch, then take the exams and be a Licensed Clinical Psychologist – all without ever having studied children, cognitive development, education, or educational testing!

One of many specializations in psychology is that of Clinical Psychology. A person who gets a Ph.D. in Clinical Psychology in the U.S. normally chooses between the adult track and one or many child tracks in their clinical program. Again, someone who follows the adult track may learn about adult psychopathology, but is unlikely to spend much time learning about childhood issues related to education or development or to have completed much, if any, supervised work with children. (And, of course, anyone who has a choice of child or adult tracks and chooses the adult track probably doesn’t have a primary interest in children in the first place.)

Luckily, there is a simple way to sort through this quagmire of qualifications, at least for US-trained psychologists. The best way to find out about the graduate-school interest and focus of a graduate in Clinical Psychology, or of a US-licensed Clinical Psychologist, is to ask about that all-important two-year internship! The internship is typically arranged or approved by the Ph.D. advisor, and will reflect the area of interest, type of classes and supervised practica followed in graduate school. Any Clinical Psychologist should be able to provide proof of their internship, which for these purposes should be in a school, community mental health center, or other setting related to kids and testing.

Of course, most professionals follow their area of training and interest and develop their career around these things. People who do their internships at Merrill-Lynch usually go on to a career in a similar setting. But, unfortunately, some do not as they may find it more lucrative to work outside their area of expertise or interest. As parents, it is important to ask carefully about internship, educational background and interests before you pay money to get meaningless test results or inappropriate recommendations.

A FINAL NOTE

Finally, and this recommendation is true no matter who is providing your testing: trust what you know about your child. No matter how qualified the practitioner, or how carefully the testing was done, the results are based on tests that are not perfect, in a situation that may not be optimal. The results are good guides, and often very helpful in finding reasons for things you may be seeing at home or at school. But if they tell you something that does not make sense to you, they may be wrong. A good practitioner will be cautious in his or her interpretation of the results, and you should be cautious as well. If I had not been, I would have believed that Jessica was extremely mentally impaired when she was two and “average” when she was six. I assure you, now that she is an adult it is clear that she is not mentally retarded and is anything but average.

Along the same lines, take diagnoses with the caution they deserve, and remember that a diagnosis should be helpful, not harmful. One of my favorite quotes is the line by Goody Nurse in The Crucible by Arthur Miller, a story about the Salem witch trials. When she sees the girls writhing on their beds, seemingly stricken by the devil, she says, “I have eleven children, and I am twenty-six times a grandma, and I have seen them all through their silly seasons, and when it comes on them they will run the Devil bowlegged keeping up with their mischief.” (Of course, the town does not listen to Goody Nurse, and the resultant trials kill many innocent citizens!) Goody Nurse’s point is that children do have transient periods of odd or dysfunctional behaviors. It is important to address them, one by one, for what they are, not for what they might be or could turn into or could signify. It is important to proceed with caution and love, forcefully and intensively addressing the problems that have been clearly identified, but without attempting to second-guess what these particular problems may mean about the future of your child. The diagnosis does not define your child: it should be meant to clarify and help, not to limit or dismiss. Don’t turn a “silly season” into a witch trial, and don’t allow others to do so either, no matter how professional or expert they may seem.

– Dr. Jadis Blurton