As many former test takers know, much of successful test prep comes down to quick pattern recognition: students who have practiced enough quickly recognize what the question is asking and start working on the problem immediately.

This is crucial for timed and challenging tests, like the SHSAT. With 100 questions in total, and only 3 hours to answer them, the race for a quick turnaround starts at the first minute.

The good news for prepping students is that the number of question templates is much lower than 100. Most of the questions fall into a set of fixed templates (basically buckets) that, once identified, can easily be memorized and recognized on the test day.

There are two ways to get a grasp of the SHSAT question templates:

• Do a bunch of tests and make note of the templates, or
• Run a data-mining analysis on the entire corpus of tests and identify templates using Natural Language Processing (NLP) methods.

I have tried both (even though I am not a student), but you won’t be surprised that I prefer the second method.

Here’s what I found out.

The data (and the method)

If you’ve followed my SHSAT research on this site (if you haven’t you should definitely subscribe so you get future research results straight into your inbox), you’ll know that I took 12 official SHSAT practice tests that the NYC Department of Education published between 2018 and 2024, and analyzed them thoroughly looking for patterns.

There are 1,368 questions in this dataset, and I processed each one of them carefully, looking for clues to sort them in a finite set of template buckets. To do that I used a text-mining tool, called regular expressions, to search for certain linguistic patterns (a specific wording) that might indicate common patterns shared by more questions.

I divided the work across ELA and math sections, and found several interesting things. Let’s start with ELA.

ELA: three questions in many costumes

About 72% of all ELA questions are reading comprehension questions and about 20% are editing (and/or revising) questions (the remaining 8% used wording too unusual to sort and I didn’t want to guess blindly).

In the reading comprehension (sub)group, three types of question occur most frequently:

• How does this part work? (For example, "How does paragraph 2 build on paragraph 1?")
• What's the central idea?
• Which sentence is the best evidence?

Taken together, that's 44% of the ELA section. Which means that nearly half the test is the same three questions asked about different passages, which on a 57-question section comes to roughly 25 questions.

Editing questions are spread much thinner. There are six kinds (combine sentences, fix an error, add a sentence, delete one, move one, revise for a transition), and none of them is more than about 7% of the section. Together they're worth around 11 questions per test, but each kind on its own is a small target.

Here’s the full distribution:

Math: a reading test with numbers in it

Math has the same shape. Three types of question make up about half the section:

• How many? (Questions involving some kind of counting)
• Solve for a value (Algebraic questions)
• Identify a measure (Questions about length, area, angle).

One caution about these labels: they describe what the question asks you to find, while the math skill underneath can be something else entirely. "How many pages will he still need to read?" lands under "how many," but what it really tests is fractions. Sorting questions by skill is a different job (and a future article).

The finding that surprised me most is the “wordiness” of math questions. Only about 2% of SHSAT math questions are bare calculations like "3.6 ÷ 0.018 =".

Everything else comes wrapped in sentences, and the most common types are also the wordiest: probability, geometry and "how many" questions run about 40 words each, while "solve for a value" questions run about 18.

Forty words doesn't sound like much (this paragraph is longer). But your child has to read those 40 words, figure out what's being asked, and turn it into arithmetic, under time pressure, again and again for the whole section.

Which means that getting used to the question pattern (template) is very important in prep. A child who calculates fast but reads slowly will lose time on the math section too, and no amount of times-table drilling will fix that.

Has the test changed?

The SHSAT has been running for a long times. It seems natural to ask whether (and how much) it changed over the years (including only those we have data for, of course). This may help when preparing for the current iteration of the test.

So, has it?

In math, barely, in either length or template. The typical math question has run 31 to 36 words every year since 2018, and the mix of question types hasn't moved in any steady direction. The one small change is that bare calculations show up only from 2021 on, two or three per test.

But, in ELA, something did shift.

In 2018, about 12% of ELA questions asked students to infer something ("What is the most likely reason…?") and only 6% asked how a part of the passage works. From 2021 on, inference questions dropped to 2–5%, while "how does this part work" questions have run between 14% and 25% every year since 2019. Central idea questions grew too, from 10% in 2018 to 17–19% in most later years.

Still, I try to be a responsible researcher, so I have to say this: with only one or two tests per year, I wouldn't bet my monthly salary on the exact numbers.

Part of the shift may also be wording, the same skill asked in different words. But the direction is consistent, and might matter for practice: a child working mainly from the older tests is over-practicing inference and under-practicing structure, compared with what recent tests ask.

So, if you’re a parent or a tutor, adjust accordingly.

What to do with this?

There are several takeaways from this analysis.

1. Learn the menu first. Go through the main question types with your child, one example each. By test day, they should be able to name the type of a question within a few seconds of reading it.
2. Practice by type, not only by test. Full tests still matter for stamina and timing. But an hour spent on the big three ELA types (how a part works, central idea, best evidence) pays back more than an hour spent on anything else in the section.
3. Tag every missed question. After each practice test, write the question type next to each wrong answer. After two or three tests, a pattern shows up, and it tells you what kind of help your child needs. A child who keeps missing "how does this part work" has a different problem from one who keeps missing "fix an error."
4. Treat math word problems as reading practice. Before solving, have your child say the problem back in one short sentence: what's given, and what's being asked.
5. Weight ELA practice toward recent tests. They match the current question style better. The 2023 tests are a good choice on two counts: they're recent, and their math questions don't repeat anywhere else in the official set (see here).

Finally — and most importantly — when you choose practice tests, make sure they reflect the exact distribution of the question templates as represented by the official DOE materials.

That’s why I built the School of Critical Thinking, reverse-engineered the SHSAT and created batteries of tests that are as precisely as (mathematically) possible reflective of the officially released samples. You can access the Baseline test here for free, and you’ll get a scientifically precise error analysis and feedback you can use for your prep.

In any case, good luck!