Starting in fall 2026, New York City's SHSAT became a computer-adaptive test. Many parents who are otherwise unfamiliar with the concept probably wonder what that means.

It's understandable. For parents this raises a lot of questions, and various internet forums, Facebook or WhatsApp groups might be replete with rumors. Some say the first questions count more; others that getting easier questions means your kid is failing.

In addition to all this cacophony, some prep companies sell "adaptive test strategies" whose validity, as far as I can tell, nobody outside the DOE could possibly verify.

In this article, I'll explain what it means for a test to be adaptive, and I'll do it in a practical (and visual) way, so you can not just understand it but also make better decisions about how you approach test prep.

I'll do it in the same way I usually do these things: by going back to the source. I don't mean the DOE source (as we know, they don't publish their exact model behind the test), but the science that almost every adaptive test is built on. It's called Item Response Theory, or IRT, and it has been around since the middle of the last century. It's much less mysterious than its name, so don't worry if you've never encountered it (or if the name sounds too complex). By the end of this article you'll even have run a small adaptive test yourself and seen why it behaves the way it does.

The old test and the new one

The old SHSAT worked like most tests you and I probably took in school. Everyone got the same questions, and the score depended on how many of them you got right (plus some statistical adjustments so that different forms could be compared, perhaps).

An adaptive test is radically different. It resembles an eye exam more than the old-fashioned fixed test. The optometrist doesn't walk you through every lens they have. She starts by giving you one, then asks "better or worse?", and uses your answer to decide which lens to give you next. A few rounds later she establishes your prescription pretty accurately. No time is wasted on lenses that do not work for you.

An adaptive test does the same with questions. If you get one right, the next one tends to be harder. Miss one and the next one tends to be easier.

Here's an example of this difference:

Imagine two students, A and B. In a fixed test (left image), they both get the same questions, which have a single distribution of easy and hard questions (for example, they might start very easy and gradually increase to become very hard). Contrary to that, in an adaptive test (right image), the distribution of easy and hard questions is unique for each student, and depends on how they answer each question. In the example, student A answered the first five questions correctly, and this in turn increased the difficulty of each next question. After an incorrect answer, the next question is easier, and so on. Student B had the opposite situation.

Basically, the test constantly "updates" the difficulty of the questions served to the student based on their performance so far. The update doesn't continue indefinitely, however. Each test has a difficulty ceiling. Once a student reaches it, the difficulty is constant.

Something like this:

Student A answered all 20 questions correctly. In this example the hardest questions in the test have a difficulty of +3, so after question 8 there is nowhere higher to go. The same happens at the floor for student B, who gave incorrect answers to most questions.

(This is also one reason why scores at the very top, and the very bottom, are less precise than scores in the middle. The test has fewer questions at the extremes to measure with.)

Where does the question difficulty come from?

All of this assumes that the test already knows how hard each question is. So where does that knowledge come from?

It doesn't come from the people who write the questions. An expert test writer can guess that a question is hard (and seasoned writers guess reasonably well), but guesses like that are wrong often enough that tests can't really rely on them.

Instead, the difficulty is computed from data. Before a question counts toward anyone's score, it's first given to a large number of students, and their answers reveal how hard it actually is. Test makers call this calibration, or field testing. The SHSAT (like many other tests) does this by mixing a few trial questions into the real test: students answer them like any other question, but they don't count toward the score. That's one of the reasons why some questions may actually repeat two years in a row (once as a field test and once as an actual score-bearing question).

This is important to understand about adaptive testing: in order to work it needs a large body of answers, usually from thousands of students per question. Without that data there is nothing to compute the difficulty from. It's also why no prep company (us included) can reproduce the SHSAT's adaptive engine exactly. We don't have the DOE's data, and the DOE doesn't publish it. We can only (try to) estimate it.

From those field answers, IRT computes not just a question's difficulty but also another important feature: it's called discrimination, and it helps IRT distinguish between stronger and weaker students.

Let's discuss both difficulty and discrimination a bit more so you can understand their difference and importance.

Difficulty: how hard is the question?

The question difficulty is placed on a scale. In the charts in this article, that scale runs from about −3 (very easy) to +3 (very hard), with 0 in the middle. Real tests use their own numbers, but the idea is the same.

Under the hood of the test, student skill and knowledge sit on the same scale as the questions (conversion to a more familiar score is done later). So, a student at +1 level is someone who will usually get questions below +1 difficulty right and questions above it wrong. Questions right at their level are the toss-ups (note, however, that on four-choice questions, lucky guesses push the odds a bit above 50%).

The central idea of IRT is simple: the chance that a student gets a question right depends on the gap between the student's level and the question's difficulty. If the student is well above the question, the chance is high. If the question is well above the student, the chance is low.

Here's an interactive widget so you can see how this works in reality. On the vertical (y) axis is the chance that the student will answer the question correctly; on the horizontal (x) axis is the student's level. The dots are the three students. Move the slider to make the question harder or easier. What do you notice about the students' chances that they'll answer the question correctly?

If you play with it a bit, you'll notice two things.

First, the curve slides left and right but keeps its overall shape (there is a steep part). Making a question harder lowers everyone's chances, but it doesn't change how the question treats strong students compared with weak ones.

Second, the curve never drops students chances to zero (the curve starts higher than zero on the y axis). On a four-choice question, even a student with no idea can guess right about a quarter of the time.

Discrimination: how well does the question separate students?

As you saw, difficulty decides where a question tells students apart. But questions also differ in how well they tell them apart. On a good question, almost every student above its level gets it right and almost every student below gets it wrong. On a poor one, strong and weak students get it right at similar rates.

That is what psychometricians call discrimination (an unfortunate word, I know; it just means "telling apart"). In the chart, it shows up as the steepness of the curve (its slope).

Here's another widget, where you can move the slider and watch the gap between the weaker and the stronger student:

At high discrimination the curve is steep, and the weaker and stronger student end up far apart (the question discriminates them well). At zero it goes flat, and both have the same chance, so the question measures nothing at all. A question like that is usually a sign that something is wrong with it. It may be ambiguous, have two defensible answers, or test something other than what it's supposed to.

Similarly to difficulty, the discrimination of a question cannot be determined by test makers in advance: it is also computed from the scores of the students who have taken the test before.

This has a practical consequence for test-takers. Many adaptive tests take discrimination into account when they compute the score, so a well-discriminating question moves the estimate more than a less-discriminating one. This is why some questions really do count more than others. But since you can't tell which ones they are, the practical advice is this: do every question as well as you can; don't assume the question is easy (or too hard for you).

How the questions are calibrated

Once a question has been tried out in a field test, how do test makers get its difficulty and discrimination from the answers?

They estimate it using statistical methods. Here’s how it works, in a nutshell:

Imagine we line up all the students who answered it, from weaker to stronger. (We know roughly where each student belongs from how they did on the rest of the test.) A single student's answer is only right or wrong, so we put students of similar level into groups and check what share of each group got the question right. Maybe 20% of the weakest group got it right, 50% of the middle group, and 85% of the strongest.

Each of those groups becomes a point on a chart (like Fig. 5 below). Then we draw an S-shaped curve that best fits the points. Why an S-shaped? Well, because that shape allows to to determine those two important features of the question: its difficulty and discrimination. The place on the chart where the curve rises tells us the difficulty of the question, and how steeply it rises tells us its discrimination. If the weaker students mostly miss it and the stronger ones mostly get it, with a sharp switch in between, it's a well-discriminating question. If right and wrong answers are scattered all along the line, then it's not.

This picture helps test makers weed out bad questions. A question that almost nobody gets right, or one almost everybody does, tells the test very little about the students taking it, so it's usually reworked or dropped. The questions that remain stay within the range of the students the test is meant for. Speaking practically, this means that questions that are too easy or too hard for nearly everyone are removed from the test.

Coming up with good test questions reliably takes a lot of students, for the same reason an election poll of 30 people is far less trustworthy than a poll of 3,000. With only a few students, luck matters too much: a couple of lucky guesses or careless slips can make a question look much easier or harder than it is.

To see this in action, play around with the widget below. Imagine we made up a question and set its real curve in advance (the dashed line, which assumes that we know its difficulty and discrimination). Then we gave it to a group of imaginary students, grouped the students by level, and checked what share of each group got it right (those are the dots, and bigger dots mean bigger groups).

Once we do that, we draw a curve that best fits the dots (the solid line). Run the simulation with "30 students" a few times, then with "3,000 students," and watch how the dots, and the solid line with them, settle closer to the real curve (the dashed line) as the groups grow:

You’ll notice that with only 30 students answering the question, the solid curve jumps around with every click. With 3,000 student, it lands almost on top of the real one. The steepness (an indicator of discrimination) is the hardest part to get right, so test makers need a lot of students and a lot of answers to do that.

But this tells us something important: the difficulty of the questions on the test has been established as precisely as possible to challenge the exact skills students at the age of your child possess. Nobody is making those questions too hard to trick your child. This is a precise science, not a whim of the test-makers.

Adaptive testing in real time

Once every question has its difficulty and discrimination values (called "parameters") set up, the adaptive testing can begin.

The aim of the test is to develop (as accurately as possible) a model of the student's skill and capability. It starts with a guess about the student (usually it assumes that the student is of average skill), and after each answer behaves like a detective, asking a question: which level best explains the pattern of answers I've seen so far? That level is the foundation for their final score. It uses statistical estimation to settle at what is the most representative score for a student with a particular pattern of right (and wrong) answers on the test.

Here’s how that happens in real time.

Check the Fig. 6 below. The y axis of this chart is the student strength that the test tries to estimate, and the x axis is a set of 20 questions they answer (this is hypothetical, tests usually have more than 20; for example, SHSAT has 100).

Try it yourself in the widget below. Pick a true level for an imaginary student (click “Simulate a student” and then click “Next question”) and let the test find settle on their strength. Or switch to "Answer yourself" and click “Right” or “Wrong” to simulate answers on the question and see how the test reacts.

What do you notice?

As the test progresses, each student answer moves the estimate; right answers move it up, wrong answers move it down. On the real test, not all answers move the estimate by the same amount, however: surprising answers move it the most. The surprise can go both ways; a strong student missing an easy question tells the test much more than a strong student getting it right.

The estimate also comes with a margin of error, the shaded band that you see around the estimate.

At the start the band is wide, because the test knows almost nothing about the student strength. With every answer it narrows. It's a bit like an unfocused camera lens: at first its blurry but then the object comes into focus (the test assumes your child's actual ability stays the same during the test and is only trying to measure it).

If you ran it a few times, you'll probably notice three things.

First, the first few answers swing the estimate a lot, while later answers barely move it. Second, the questions follow the student up and down and hover near their level.

Third, the estimate rarely lands exactly on the true level. It gets close, within a margin of error, but it's not perfect. This means that it's quite possible that two students of equal capability get slightly different scores. It’s imperfect, but this is how all of these tests work.

The score is not a count of right answers

Many parents believe that if their child gets the same number of answers correct as another student, their final score will be the same. But that's not how this works. On an adaptive test, two students can answer the same number of questions correctly and end up with different scores.

It sounds unfair until you look at which questions they got right. Here are Students A and B from Fig. 1 again. Both got 6 of their 10 questions right. Student A started strong, so the test kept serving harder questions. Student B missed the first four, so the test served easier ones, and his correct answers came on those.

Six right answers on hard questions are stronger evidence of skill than six right answers on easy ones. That's also what makes adaptive tests fair. Students see different questions, but every question has been calibrated, so every student lands on the same scale.

The consequence of this is that mere percentage doesn't mean much. So be careful with any calculator that turns a percentage on an adaptive practice test into a precise SHSAT score: all of those are just rough estimates. Nobody can really convert a percentage into a score with certainty.

What all this means on test day

If you're still reading this (long-ish) article, congrats: you are a committed parent. As a reward, I'll outline a few practical consequences that follow from everything described above. They should ease both yours and your child’s anxiety.

• Harder questions are usually a good sign. They mean the answers so far have pushed the estimate up.
• Feeling like you're getting a lot wrong is normal. An adaptive test is designed to find your child's edge, and the edge is exactly where questions feel hard. Many students walk out of an adaptive test thinking they did worse than they actually did.
• Don't try to outsmart the algorithm. Missing an easy question on purpose, or rushing to "get to the hard ones," only feeds the estimate bad evidence. The best strategy is boringly simple: do each question as well as you can.
• Be wary of confident claims about details nobody outside the DOE knows, such as whether the first questions count more, or exactly how the test decides what comes next. Adaptive tests differ in these details, and the SHSAT's haven't been published.

How to prepare: the test measures, practice teaches

Finally, here are a few tips that will affect how your child prepares for the test.

The SHSAT is built to measure your child. It assumes nothing about them changes during the test, and its job is to find their level as precisely as possible.

But practice has the opposite job: it's supposed to change your child. Different jobs call for different tools, and a practice platform that only imitates the real test is using the wrong one.

Researchers who build tutoring systems use a different family of models for practice (a classic one is called knowledge tracing). Instead of placing a student on a scale, they ask, skill by skill, whether the student has learned it yet, and they keep the student practicing until the answer is very likely yes. In one well-known study of a programming tutor, 56% of the students who practiced this way reached 90% on the final test, compared with 24% of those who worked through a fixed set of exercises.

Put that together with IRT and you get five practical rules:

1. Practice at the right level. Most practice questions should be ones your child gets right most of the time, but not always. Several adaptive practice systems aim for about 75%. Too easy and nothing is learned; too hard and you get frustration and guessing.
2. Practice weak skills to mastery, then come back to them. One good day isn't mastery. Revisit old skills a few weeks later too, because skills that aren't used fade.
3. Study the wrong answers. A right answer tells you your child knew it, or guessed. A wrong answer tells you how they were thinking. On well-made tests, wrong answers are designed to attract specific mistakes, and the same few mistakes tend to repeat across dozens of questions.
4. Knowing a rule isn't the same as using it under pressure. Your child can explain what the main idea of a passage is, and still pick the wrong choice. Understanding comes first, but only repeated practice makes it automatic.
5. Use full practice tests as check-ins. They tell you where your child stands. By themselves, they don't teach much.

An adaptive test is, at heart, a model of your child: a best guess about their level, built from a few dozen answers and thousands of other students’ data, with a margin of error attached. It’s a good model, better in many ways than the old fixed test, but it’s still a model. As a critical thinker (and as a parent of a kid preparing for this very test), that’s how I’d want my own child to see it: something to understand and prepare for, rather than a verdict to fear.

And if the real test is built to measure, your practice should be built to teach. That’s why I made our free Baseline Test. Like the real SHSAT, it adapts to your child’s answers, but it doesn’t stop at a score. It shows you which skills are costing your child the most points, and which specific answer traps catch them again and again. That’s the information you need to decide what to practice next. You can do it here, for free.

Good luck.