Jack Bishop of America’s Test Kitchen on Recipe Testing and a $600,000 Grocery Bill

Almost every food magazine has a test kitchen where they develop and refine recipes. But there’s nothing quite like America’s Test Kitchen, the powerhouse behind Cook’s Illustrated and Cook’s Country — along with, of course, the America’s Test Kitchen cooking show on PBS, dozens of cookbooks, numerous radio shows, and an ever-growing collection of online content.

We’ve been fans of their work for more than a decade. In the nine years that we wrote our cooking blog, The Bitten Word, countless readers wrote us to recommend different America’s Test Kitchen recipes that had become their “go to” for everything from meatloaf to chocolate chip cookies.

But what’s it like to actually work in America’s Test Kitchen? To find out, we reached out to Jack Bishop, the company’s chief creative officer. Jack gave us a behind-the-scenes look at ATK, including how a recipe gets developed, what it’s like to slice open a Yeti cooler, and why caramel is the bane of the test kitchen.

I was there when the company started 25 years ago. I was one of the original editors of Cook’s Illustrated, and started doing product reviews and recipe development. Before coming to America’s Test Kitchen, I was an English major and worked in book publishing.

I was at Cook’s because I loved to cook. I’m an anomaly in the test kitchen: I didn’t go to cooking school. I view that lack of professional training as an asset. In a lot of ways, I’m more like our readers. My knife skills are mediocre; I can’t chop three pounds of onions in three minutes.

The easiest statistic for people to get their heads around is our grocery bill. We’ll spend about $600,000 on groceries this year. That’s enough food for an awful lot of recipes. It’s the same ingredients the home cook is buying — just a whole heck of a lot more.

We have 35 people working in the test kitchen who are developing recipes, and another 35 who are editing, taking photographs, or art directing with content developers.

There is an insane amount of food at America’s Test Kitchen.

You can’t donate a half-eaten apple pie to a food pantry. One of the great employee perks of working here is that we have a “take-home” fridge that fills up over the course of the day. It’s nice to be able to take dinner home every night.

And on any given day, there are one or two product taste tests that we’re doing. It could be chocolate, it could be peanut butter, it could be — ugh — fish sauce. I mean, I like fish sauce, but you know …

For 25 years we were in a test kitchen — including our storage and photography areas — that was less than 4,000 square feet. We were literally shoulder to shoulder. If we wanted to shoot a video, we had shut down the entire kitchen. Someone can’t be using a blender while you’re trying to shoot a video!

Our new facility is 15,000 square feet, including four photo studios, two set kitchens, and four test kitchens.

Let’s walk through a Cook’s Illustrated example, for when we developed a recipe for lemon bars.

It begins with research. One of our five recipe developers will compile every lemon bar recipe we’ve ever made, as well as the recipes from other experts and chefs and maybe someone’s grandmother’s recipe. Typically the developer will read 80 to 100 lemon bar recipes.

We’ll choose five to seven of those recipes to make as is, choosing recipes that are different enough from each other and interesting.

We’ll taste them and then we’ll have a conversation about what we want to accomplish in our recipe. What do we want from the crust? What’s the ratio of crust to filling? We’ll identify problems: Can the crust be delicate but still sturdy enough to allow us to cut the bars into neat squares? Is the filling firm enough to slice, but not rubbery or gelatinous?

We’ll use that conversation to build a composite recipe. In the case of lemon bars, we’ll focus on crust, and then filling, and then the two together. But we’ll attack them separately. We may do a series of filling tests, with different amounts of lemon. Maybe it’s a quarter-cup of lemon juice. Let’s try a half-cup of lemon juice. Now a third-cup of lemon juice and two tablespoons of zest.

The next round will be all about eggs — it’s six eggs, it’s four eggs and two yolks, it’s six yolks. People also ask how we can test a recipe 50 to 60 times, but you can see how it adds up. The developer will keep making versions of a recipe for two to four weeks (she’s working on other recipes at the same time).

Then we’ll move to “abuse testing.” We send the recipe to the interns, and they’ll make it and we’ll see how it goes. Then we’ll change the recipe the way people at home might. We’ll sub low-fat milk in place of whole milk. We’ll bake at 325°F and at 350°F, because a lot of people’s oven temperatures are off. And then we’ll send it out to our community of volunteers — we have 30,000 friends of America’s Test Kitchen who have volunteered to road-test recipes before publication. They get an assignment from us once a month.

Once those volunteers make the recipes, they go online to tell us about their experience via a survey. We’ll ask them about the quantity, the equipment, the ingredients, and substitutions they made. We’ll ask what things they didn’t understand. If you have a lot of people who are confused by Step 4 in a recipe, you have to rewrite Step 4.

And most importantly, we’ll ask them if they would make the recipe again. If that score is not 80 percent or higher, we send it back to the test kitchen, and the developer has to see how we can make the recipe more appealing to the audience.

For any recipe, it’s generally one month in the kitchen, and one month of vetting. And then it goes off for photography and video

I’d say 20 percent of the recipes get sent back for some issue we want to address. Sometimes it’s just that the recipe needs artwork to explain the technique. Or sometimes it’s a matter of just writing the recipe more clearly.

We have. You should read Dan Souza’s editorial in the May/June issue of Cook’s Illustrated. It talks about his macaron recipe which was never published, despite many weeks he spent in the test kitchen working on it. Macarons are fickle — and ours were, too. Multiple versions went out for testing. We’re not doing fans any favors if a large percentage of them can’t make it.

Anything with sugar syrups, caramel, or fudge. One of our recipe developers made 130 batches of fudge, and he literally got a rotator cuff injury. People really have trouble with sugar syrups and caramel. In part, they’re problematic because people don’t have the right thermometers. And a lot of people use nonstick, even though we tell them that if you’re looking into a dark pan you can’t judge the color and you have no idea what’s going on. It’s a place people struggle.

It’s a complete sea change. Twenty-five years ago we had long discussions about whether people can find chipotle peppers or tahini. Now almost every supermarket has a wide range of ingredients [like that].

Tastes have also changed. Twenty-five years ago everyone wanted to make their grandmother’s food. Now everyone wants to make the food they had in a restaurant last week.

We’ll review more than 1,000 products this year, both ingredients and kitchen equipment. We buy everything that we test — we don’t accept freebies.

We’re testing skillets now. They’re 12-inch skillets, but not all of them have 12 inches of cooking surface, because of the way the sides are angled. We’re measuring and weighing the skillets, but we’re also searing chicken, sautéeing onions, and running the pans through a dedicated series of protocols about how we’re going to evaluate the product.

We talk to university professors and engineers for ideas on what to test. We come up with a mix of quirky tests and a straightforward look at how someone would use the product. When we were testing food storage containers, for example, we put them through the dishwasher 50 times. As we’ve all seen, bad things can happen to food storage containers in a dishwasher.

And we slice things open. We just sliced open a Yeti cooler so we can see what’s happening inside in terms of the insulation.

We had these two women doing it on Facebook Live, slicing into it with a turn saw. (I’m not sure if that’s what you call it; it was scary!) They didn’t know what they were going to find inside.

This interview has been edited and condensed.