Here’s what you should know about SHSAT sample tests

If you’re preparing your child for the SHSAT and you are using previously published tests, there are a few very important things you should know.

Not all previous tests are equally good for a successful prep. Some of them heavily repeat questions, while others are too similar to one another to event count as separate tests.

How do I know this?

I collected all previously published tests, scraped all of their passages, questions, and answer choices, and reverse-engineered it using statistical and computational linguistics methods.

I’ll share a few findings with you here.

What I analyzed

There are a total of 12 previous test samples published by the NYC Department of Education, between 2018 and 2024.

Together, they contain 1,368 questions, 114 per test, split evenly: 57 ELA and 57 math.

Most years had A and B tests. Here’s what’s out there:

To analyze all of these and derive a reliable insight, I cleaned the questions, setting aside some of them (about 6%) because they included images or text that was not comparable.

How long are the questions?

Your child will have 3 hours to complete the test (both ELA and math sections, computer based and now containing only 50 questions each). They will have the opportunity to choose whichever section they wish to start with.

So, the most natural first question is about the length of the questions. So, how long they are?

They can range anywhere between mere 13 to almost 800 characters. The averages are a bit different for each section. An average ELA question is about 157 characters long, while an average math question is 174.

There’s a great variance within each, because many ELA question are paired with a reading paragraph, which explains their somewhat shorter length.

Some math questions will be very short because they will ask for a simple computation, such as: 3.6 / 0.018 and similar.

Here’s the distribution of the question lengths:

This should give you some insight about how to prepare your child for timing the sections well.

Math is heavily reused

Out of total 624 math questions I analyzed, only 440 were distinct questions: others were reused.

This reuse runs in entire families of sample tests. For example, 2018 A, 2019 A, and 2022 A have many duplicate math questions. So do 2018 B, 2019 B, and 2022 B, as well as 2021 A and 2024 A.

Sample tests from 2018 and 2019 are almost the same math test.

The ELA section does not include that many repeated questions - only about 7%. However, some passages will repeat, sometimes more than once (more about that in a future article).

How can this help you?

Well, if you want to get a more accurate sense of how ready your child is for the test, you have to be more careful about which tests you use for practice. You should not trust a score on a test that repeats questions heavily. For example, if your child takes 2018 A and later 2019 A, their 2019 score will look better partly because they've seen most of the questions before. You may think they improved when the child simply remembered. A rising score only means progress only if the test (and all of its questions) was new to the child.

Here’s a full percentage breakdown of original and duplicated questions on each published test set:

What you should do?

I’m a parent too, so I know the worries and anxieties you might be facing with all of this.

But before you jump and spend a few thousand dollars for a test prep program, see if you can do some of the prepping yourself. Test prep companies charge crazy amounts of money for something dedicated parents can do themselves: have their children take sample tests, identify their weak points, and do a precision-based review and practice to build those exact skills.

However, if you’re too busy to do that by yourself, I can help.

Here you can find a free baseline test (made to reflect the precise distribution of official DOE questions) that will tell you exactly what are your child’s weak points, so you know what to focus on.

If you have any questions, feel free to reach out.

Good luck!