Elizabeth Moore is a publishing associate in the Knopf Doubleday division of Penguin Random House, working primarily with paperback publication. Typically, the books that cross Moore’s desk were published in hardcover 12 to 18 months earlier. How they sold on their first circuit through the marketplace makes an enormous difference in what Moore and her team do next.

“Many books by successful house authors practically sell themselves,” Moore said. “When those books go to paperback we change hardly anything, just adding a stepback cover with all the praise and awards.”

Unfortunately, not every book published by Penguin Random House is a New York Times Best Seller—far from it. More often than not, hardcover books fail to reach sales targets. That’s when Moore and her team step in. Sometimes they’ll ask, “What went wrong?” Rethinking the cover art, book description, or marketing strategy to try and increase sales the second time around. Other times, they have to make the hard decision to let a book go out of print.

“Most books lose money, but a handful of books are wildly successful and that’s what keeps the lights on,” Moore explained. “It’s crazy. You just have to hope that your bestsellers will pull through.”

It’s not just Penguin Random House that struggles with inefficiencies. In a survey of publishing decision-makers conducted by Target Audience Insights in 2021, more than three-quarters (76%) of publishers reported that less than 36% of their titles met or exceeded their first 12-month revenue target; almost half said that fewer than one in five titles made budget. Those are rough statistics for any company, but particularly hard on smaller firms.

“I think this is why so many small presses have been acquired by massive publishing powerhouses,” she added. “If a small press goes a few years without landing that best seller, they just can’t maintain cash flow.”

In a world where savvy business decisions are nearly inevitably informed and shaped by data, the publishing industry has consistently lagged behind. And yet, slowly but inevitably, the publishing ecosystem is changing as publishers, agents, and authors alike are increasingly turning to target audience feedback to help them make better decisions.

“When I first started at Penguin Random House, I don’t remember much of any kind of market research being used,” said Moore. “It’s not that it wasn’t being done at all, but it just wasn’t super robust or built out. Just in the last couple of years [it’s started to change.] We now have a corporate consumer insights team that does research and puts together reports on what media consumers and book buyers are looking for. Like, what do consumers of romance novels want right now? How are people feeling about COVID? Politics? Are we into Valentines Day this year? Galentines day? It’s kind of a general vibe check on consumers.”

Even as the corporate team churns out broad consumer insights reports, a team in Moore’s specific division conducts deeper dive research into specific genres, books, and authors. When Knopf Doubleday wanted to rebrand a Pulitzer Prize-winning author’s backlist to introduce her to a younger generation of readers, the team researched that target market. Where could they find them? Where would these younger readers discover this author’s work? What are comparable contemporary authors that this audience is already reading? The results help determine everything from who writes the introduction to new paperback editions to cover art design to marketing campaign strategy.

In short, data provides an edge, helping improve a publisher’s “batting average” and make decisions based on what readers actually want.

This sort of data is being used more and more. According to TAI’s 2021 report, 43% of publishers report “sometimes” or “frequently” capturing target reader feedback before publishing a book. Nearly a third (31%) of publishers have used a third-party research vendor to capture target reader feedback on cover design, titles, initial reader interest in a book’s concept, or feedback on the content of the manuscript itself.

True, that is still less than half of publishers, but in an industry long-driven by instinct and seemingly allergic to data, those numbers may signal the beginning of a surge towards harnessing the transformative potential of consumer insights.

“There’s so much room for growth,” Moore said. “The way things used to be done, there wasn’t much room for data. It’s not that it didn’t exist, but publishing was much more relational and insular. But I think that right now we’re on this cusp where the old world of publishing is starting to fade out. Data usage and analysis is going to grow exponentially and become a much bigger part of publishing as younger people start moving into more senior level positions.”

In other words, things are changing. The new era of data-infused publishing isn’t just coming, it may already be here.