James Behnke, a 55-year-old executive at Pillsbury, greeted the men as they arrived. He was anxious but also hopeful about the plan that he and a few other food-company executives had devised to engage the C.E.O.’s on America’s growing weight problem. “We were very concerned, and rightfully so, that obesity was becoming a major issue,” Behnke recalled. “People were starting to talk about sugar taxes, and there was a lot of pressure on food companies.” Getting the company chiefs in the same room to talk about anything, much less a sensitive issue like this, was a tricky business, so Behnke and his fellow organizers had scripted the meeting carefully, honing the message to its barest essentials. “C.E.O.’s in the food industry are typically not technical guys, and they’re uncomfortable going to meetings where technical people talk in technical terms about technical things,” Behnke said. “They don’t want to be embarrassed. They don’t want to make commitments. They want to maintain their aloofness and autonomy.”

 The discussion took place in Pillsbury’s auditorium. The first speaker was a vice president of Kraft named Michael Mudd. “I very much appreciate this opportunity to talk to you about childhood obesity and the growing challenge it presents for us all,” Mudd began. “Let me say right at the start, this is not an easy subject. There are no easy answers — for what the public health community must do to bring this problem under control or for what the industry should do as others seek to hold it accountable for what has happened. But this much is clear: For those of us who’ve looked hard at this issue, whether they’re public health professionals or staff specialists in your own companies, we feel sure that the one thing we shouldn’t do is nothing.”

As he spoke, Mudd clicked through a deck of slides — 114 in all — projected on a large screen behind him. The figures were staggering. More than half of American adults were now considered overweight, with nearly one-quarter of the adult population — 40 million people — clinically defined as obese. Among children, the rates had more than doubled since 1980, and the number of kids considered obese had shot past 12 million. (This was still only 1999; the nation’s obesity rates would climb much higher.) Food manufacturers were now being blamed for the problem from all sides — academia, the Centers for Disease Control and Prevention, the American Heart Association and the American Cancer Society. The secretary of agriculture, over whom the industry had long held sway, had recently called obesity a “national epidemic.”

Mudd then did the unthinkable. He drew a connection to the last thing in the world the C.E.O.’s wanted linked to their products: cigarettes. First came a quote from a Yale University professor of psychology and public health, Kelly Brownell, who was an especially vocal proponent of the view that the processed-food industry should be seen as a public health menace: “As a culture, we’ve become upset by the tobacco companies advertising to children, but we sit idly by while the food companies do the very same thing. And we could make a claim that the toll taken on the public health by a poor diet rivals that taken by tobacco.”

“If anyone in the food industry ever doubted there was a slippery slope out there,” Mudd said, “I imagine they are beginning to experience a distinct sliding sensation right about now.”

Mudd then presented the plan he and others had devised to address the obesity problem. Merely getting the executives to acknowledge some culpability was an important first step, he knew, so his plan would start off with a small but crucial move: the industry should use the expertise of scientists — its own and others — to gain a deeper understanding of what was driving Americans to overeat. Once this was achieved, the effort could unfold on several fronts. To be sure, there would be no getting around the role that packaged foods and drinks play in overconsumption. They would have to pull back on their use of salt, sugar and fat, perhaps by imposing industrywide limits. But it wasn’t just a matter of these three ingredients; the schemes they used to advertise and market their products were critical, too. Mudd proposed creating a “code to guide the nutritional aspects of food marketing, especially to children.”

“We are saying that the industry should make a sincere effort to be part of the solution,” Mudd concluded. “And that by doing so, we can help to defuse the criticism that’s building against us.”

What happened next was not written down. But according to three participants, when Mudd stopped talking, the one C.E.O. whose recent exploits in the grocery store had awed the rest of the industry stood up to speak. His name was Stephen Sanger, and he was also the person — as head of General Mills — who had the most to lose when it came to dealing with obesity. Under his leadership, General Mills had overtaken not just the cereal aisle but other sections of the grocery store. The company’s Yoplait brand had transformed traditional unsweetened breakfast yogurt into a veritable dessert. It now had twice as much sugar per serving as General Mills’ marshmallow cereal Lucky Charms. And yet, because of yogurt’s well-tended image as a wholesome snack, sales of Yoplait were soaring, with annual revenue topping $500 million. Emboldened by the success, the company’s development wing pushed even harder, inventing a Yoplait variation that came in a squeezable tube — perfect for kids. They called it Go-Gurt and rolled it out nationally in the weeks before the C.E.O. meeting. (By year’s end, it would hit $100 million in sales.)

According to the sources I spoke with, Sanger began by reminding the group that consumers were “fickle.” (Sanger declined to be interviewed.) Sometimes they worried about sugar, other times fat. General Mills, he said, acted responsibly to both the public and shareholders by offering products to satisfy dieters and other concerned shoppers, from low sugar to added whole grains. But most often, he said, people bought what they liked, and they liked what tasted good. “Don’t talk to me about nutrition,” he reportedly said, taking on the voice of the typical consumer. “Talk to me about taste, and if this stuff tastes better, don’t run around trying to sell stuff that doesn’t taste good.”

To react to the critics, Sanger said, would jeopardize the sanctity of the recipes that had made his products so successful. General Mills would not pull back. He would push his people onward, and he urged his peers to do the same. Sanger’s response effectively ended the meeting.

“What can I say?” James Behnke told me years later. “It didn’t work. These guys weren’t as receptive as we thought they would be.” Behnke chose his words deliberately. He wanted to be fair. “Sanger was trying to say, ‘Look, we’re not going to screw around with the company jewels here and change the formulations because a bunch of guys in white coats are worried about obesity.’ ”

The meeting was remarkable, first, for the insider admissions of guilt. But I was also struck by how prescient the organizers of the sit-down had been. Today, one in three adults is considered clinically obese, along with one in five kids, and 24 million Americans are afflicted by type 2 diabetes, often caused by poor diet, with another 79 million people having pre-diabetes. Even gout, a painful form of arthritis once known as “the rich man’s disease” for its associations with gluttony, now afflicts eight million Americans.

The public and the food companies have known for decades now — or at the very least since this meeting — that sugary, salty, fatty foods are not good for us in the quantities that we consume them. So why are the diabetes and obesity and hypertension numbers still spiraling out of control? It’s not just a matter of poor willpower on the part of the consumer and a give-the-people-what-they-want attitude on the part of the food manufacturers. What I found, over four years of research and reporting, was a conscious effort — taking place in labs and marketing meetings and grocery-store aisles — to get people hooked on foods that are convenient and inexpensive. I talked to more than 300 people in or formerly employed by the processed-food industry, from scientists to marketers to C.E.O.’s. Some were willing whistle-blowers, while others spoke reluctantly when presented with some of the thousands of pages of secret memos that I obtained from inside the food industry’s operations. What follows is a series of small case studies of a handful of characters whose work then, and perspective now, sheds light on how the foods are created and sold to people who, while not powerless, are extremely vulnerable to the intensity of these companies’ industrial formulations and selling campaigns.

I. ‘In This Field, I’m a Game Changer.’

John Lennon couldn’t find it in England, so he had cases of it shipped from New York to fuel the “Imagine” sessions. The Beach Boys, ZZ Top and Cher all stipulated in their contract riders that it be put in their dressing rooms when they toured. Hillary Clinton asked for it when she traveled as first lady, and ever after her hotel suites were dutifully stocked.

What they all wanted was Dr Pepper, which until 2001 occupied a comfortable third-place spot in the soda aisle behind Coca-Cola and Pepsi. But then a flood of spinoffs from the two soda giants showed up on the shelves — lemons and limes, vanillas and coffees, raspberries and oranges, whites and blues and clears — what in food-industry lingo are known as “line extensions,” and Dr Pepper started to lose its market share.

Responding to this pressure, Cadbury Schweppes created its first spin­off, other than a diet version, in the soda’s 115-year history, a bright red soda with a very un-Dr Pepper name: Red Fusion. “If we are to re-establish Dr Pepper back to its historic growth rates, we have to add more excitement,” the company’s president, Jack Kilduff, said. One particularly promising market, Kilduff pointed out, was the “rapidly growing Hispanic and African-American communities.”

But consumers hated Red Fusion. “Dr Pepper is my all-time favorite drink, so I was curious about the Red Fusion,” a California mother of three wrote on a blog to warn other Peppers away. “It’s disgusting. Gagging. Never again.”

Stung by the rejection, Cadbury Schweppes in 2004 turned to a food-industry legend named Howard Moskowitz. Moskowitz, who studied mathematics and holds a Ph.D. in experimental psychology from Harvard, runs a consulting firm in White Plains, where for more than three decades he has “optimized” a variety of products for Campbell Soup, General Foods, Kraft and PepsiCo. “I’ve optimized soups,” Moskowitz told me. “I’ve optimized pizzas. I’ve optimized salad dressings and pickles. In this field, I’m a game changer.”

In the process of product optimization, food engineers alter a litany of variables with the sole intent of finding the most perfect version (or versions) of a product. Ordinary consumers are paid to spend hours sitting in rooms where they touch, feel, sip, smell, swirl and taste whatever product is in question. Their opinions are dumped into a computer, and the data are sifted and sorted through a statistical method called conjoint analysis, which determines what features will be most attractive to consumers. Moskowitz likes to imagine that his computer is divided into silos, in which each of the attributes is stacked. But it’s not simply a matter of comparing Color 23 with Color 24. In the most complicated projects, Color 23 must be combined with Syrup 11 and Packaging 6, and on and on, in seemingly infinite combinations. Even for jobs in which the only concern is taste and the variables are limited to the ingredients, endless charts and graphs will come spewing out of Moskowitz’s computer. “The mathematical model maps out the ingredients to the sensory perceptions these ingredients create,” he told me, “so I can just dial a new product. This is the engineering approach.”

Moskowitz’s work on Prego spaghetti sauce was memorialized in a 2004 presentation by the author Malcolm Gladwell at the TED conference in Monterey, Calif.: “After … months and months, he had a mountain of data about how the American people feel about spaghetti sauce… . And sure enough, if you sit down and you analyze all this data on spaghetti sauce, you realize that all Americans fall into one of three groups. There are people who like their spaghetti sauce plain. There are people who like their spaghetti sauce spicy. And there are people who like it extra-chunky. And of those three facts, the third one was the most significant, because at the time, in the early 1980s, if you went to a supermarket, you would not find extra-chunky spaghetti sauce. And Prego turned to Howard, and they said, ‘Are you telling me that one-third of Americans crave extra-chunky spaghetti sauce, and yet no one is servicing their needs?’ And he said, ‘Yes.’ And Prego then went back and completely reformulated their spaghetti sauce and came out with a line of extra-chunky that immediately and completely took over the spaghetti-sauce business in this country… . That is Howard’s gift to the American people… . He fundamentally changed the way the food industry thinks about making you happy.”

Well, yes and no. One thing Gladwell didn’t mention is that the food industry already knew some things about making people happy — and it started with sugar. Many of the Prego sauces — whether cheesy, chunky or light — have one feature in common: The largest ingredient, after tomatoes, is sugar. A mere half-cup of Prego Traditional, for instance, has the equivalent of more than two teaspoons of sugar, as much as two-plus Oreo cookies. It also delivers one-third of the sodium recommended for a majority of American adults for an entire day. In making these sauces, Campbell supplied the ingredients, including the salt, sugar and, for some versions, fat, while Moskowitz supplied the optimization. “More is not necessarily better,” Moskowitz wrote in his own account of the Prego project. “As the sensory intensity (say, of sweetness) increases, consumers first say that they like the product more, but eventually, with a middle level of sweetness, consumers like the product the most (this is their optimum, or ‘bliss,’ point).”

I first met Moskowitz on a crisp day in the spring of 2010 at the Harvard Club in Midtown Manhattan. As we talked, he made clear that while he has worked on numerous projects aimed at creating more healthful foods and insists the industry could be doing far more to curb obesity, he had no qualms about his own pioneering work on discovering what industry insiders now regularly refer to as “the bliss point” or any of the other systems that helped food companies create the greatest amount of crave. “There’s no moral issue for me,” he said. “I did the best science I could. I was struggling to survive and didn’t have the luxury of being a moral creature. As a researcher, I was ahead of my time.”

Moskowitz’s path to mastering the bliss point began in earnest not at Harvard but a few months after graduation, 16 miles from Cambridge, in the town of Natick, where the U.S. Army hired him to work in its research labs. The military has long been in a peculiar bind when it comes to food: how to get soldiers to eat more rations when they are in the field. They know that over time, soldiers would gradually find their meals-ready-to-eat so boring that they would toss them away, half-eaten, and not get all the calories they needed. But what was causing this M.R.E.-fatigue was a mystery. “So I started asking soldiers how frequently they would like to eat this or that, trying to figure out which products they would find boring,” Moskowitz said. The answers he got were inconsistent. “They liked flavorful foods like turkey tetrazzini, but only at first; they quickly grew tired of them. On the other hand, mundane foods like white bread would never get them too excited, but they could eat lots and lots of it without feeling they’d had enough.”

This contradiction is known as “sensory-specific satiety.” In lay terms, it is the tendency for big, distinct flavors to overwhelm the brain, which responds by depressing your desire to have more. Sensory-specific satiety also became a guiding principle for the processed-food industry. The biggest hits — be they Coca-Cola or Doritos — owe their success to complex formulas that pique the taste buds enough to be alluring but don’t have a distinct, overriding single flavor that tells the brain to stop eating.

Thirty-two years after he began experimenting with the bliss point, Moskowitz got the call from Cadbury Schweppes asking him to create a good line extension for Dr Pepper. I spent an afternoon in his White Plains offices as he and his vice president for research, Michele Reisner, walked me through the Dr Pepper campaign. Cadbury wanted its new flavor to have cherry and vanilla on top of the basic Dr Pepper taste. Thus, there were three main components to play with. A sweet cherry flavoring, a sweet vanilla flavoring and a sweet syrup known as “Dr Pepper flavoring.”

Finding the bliss point required the preparation of 61 subtly distinct formulas — 31 for the regular version and 30 for diet. The formulas were then subjected to 3,904 tastings organized in Los Angeles, Dallas, Chicago and Philadelphia. The Dr Pepper tasters began working through their samples, resting five minutes between each sip to restore their taste buds. After each sample, they gave numerically ranked answers to a set of questions: How much did they like it overall? How strong is the taste? How do they feel about the taste? How would they describe the quality of this product? How likely would they be to purchase this product?

Moskowitz’s data — compiled in a 135-page report for the soda maker — is tremendously fine-grained, showing how different people and groups of people feel about a strong vanilla taste versus weak, various aspects of aroma and the powerful sensory force that food scientists call “mouth feel.” This is the way a product interacts with the mouth, as defined more specifically by a host of related sensations, from dryness to gumminess to moisture release. These are terms more familiar to sommeliers, but the mouth feel of soda and many other food items, especially those high in fat, is second only to the bliss point in its ability to predict how much craving a product will induce.

In addition to taste, the consumers were also tested on their response to color, which proved to be highly sensitive. “When we increased the level of the Dr Pepper flavoring, it gets darker and liking goes off,” Reisner said. These preferences can also be cross-referenced by age, sex and race.

On Page 83 of the report, a thin blue line represents the amount of Dr Pepper flavoring needed to generate maximum appeal. The line is shaped like an upside-down U, just like the bliss-point curve that Moskowitz studied 30 years earlier in his Army lab. And at the top of the arc, there is not a single sweet spot but instead a sweet range, within which “bliss” was achievable. This meant that Cadbury could edge back on its key ingredient, the sugary Dr Pepper syrup, without falling out of the range and losing the bliss. Instead of using 2 milliliters of the flavoring, for instance, they could use 1.69 milliliters and achieve the same effect. The potential savings is merely a few percentage points, and it won’t mean much to individual consumers who are counting calories or grams of sugar. But for Dr Pepper, it adds up to colossal savings. “That looks like nothing,” Reisner said. “But it’s a lot of money. A lot of money. Millions.”

The soda that emerged from all of Moskowitz’s variations became known as Cherry Vanilla Dr Pepper, and it proved successful beyond anything Cadbury imagined. In 2008, Cadbury split off its soft-drinks business, which included Snapple and 7-Up. The Dr Pepper Snapple Group has since been valued in excess of $11 billion.

II. ‘Lunchtime Is All Yours’

Sometimes innovations within the food industry happen in the lab, with scientists dialing in specific ingredients to achieve the greatest allure. And sometimes, as in the case of Oscar Mayer’s bologna crisis, the innovation involves putting old products in new packages.

The 1980s were tough times for Oscar Mayer. Red-meat consumption fell more than 10 percent as fat became synonymous with cholesterol, clogged arteries, heart attacks and strokes. Anxiety set in at the company’s headquarters in Madison, Wis., where executives worried about their future and the pressure they faced from their new bosses at Philip Morris.

Bob Drane was the company’s vice president for new business strategy and development when Oscar Mayer tapped him to try to find some way to reposition bologna and other troubled meats that were declining in popularity and sales. I met Drane at his home in Madison and went through the records he had kept on the birth of what would become much more than his solution to the company’s meat problem. In 1985, when Drane began working on the project, his orders were to “figure out how to contemporize what we’ve got.”

Drane’s first move was to try to zero in not on what Americans felt about processed meat but on what Americans felt about lunch. He organized focus-group sessions with the people most responsible for buying bologna — mothers — and as they talked, he realized the most pressing issue for them was time. Working moms strove to provide healthful food, of course, but they spoke with real passion and at length about the morning crush, that nightmarish dash to get breakfast on the table and lunch packed and kids out the door. He summed up their remarks for me like this: “It’s awful. I am scrambling around. My kids are asking me for stuff. I’m trying to get myself ready to go to the office. I go to pack these lunches, and I don’t know what I’ve got.” What the moms revealed to him, Drane said, was “a gold mine of disappointments and problems.”

He assembled a team of about 15 people with varied skills, from design to food science to advertising, to create something completely new — a convenient prepackaged lunch that would have as its main building block the company’s sliced bologna and ham. They wanted to add bread, naturally, because who ate bologna without it? But this presented a problem: There was no way bread could stay fresh for the two months their product needed to sit in warehouses or in grocery coolers. Crackers, however, could — so they added a handful of cracker rounds to the package. Using cheese was the next obvious move, given its increased presence in processed foods. But what kind of cheese would work? Natural Cheddar, which they started off with, crumbled and didn’t slice very well, so they moved on to processed varieties, which could bend and be sliced and would last forever, or they could knock another two cents off per unit by using an even lesser product called “cheese food,” which had lower scores than processed cheese in taste tests. The cost dilemma was solved when Oscar Mayer merged with Kraft in 1989 and the company didn’t have to shop for cheese anymore; it got all the processed cheese it wanted from its new sister company, and at cost.

Drane’s team moved into a nearby hotel, where they set out to find the right mix of components and container. They gathered around tables where bagfuls of meat, cheese, crackers and all sorts of wrapping material had been dumped, and they let their imaginations run. After snipping and taping their way through a host of failures, the model they fell back on was the American TV dinner — and after some brainstorming about names (Lunch Kits? Go-Packs? Fun Mealz?), Lunchables were born.

The trays flew off the grocery-store shelves. Sales hit a phenomenal $218 million in the first 12 months, more than anyone was prepared for. This only brought Drane his next crisis. The production costs were so high that they were losing money with each tray they produced. So Drane flew to New York, where he met with Philip Morris officials who promised to give him the money he needed to keep it going. “The hard thing is to figure out something that will sell,” he was told. “You’ll figure out how to get the cost right.” Projected to lose $6 million in 1991, the trays instead broke even; the next year, they earned $8 million.

With production costs trimmed and profits coming in, the next question was how to expand the franchise, which they did by turning to one of the cardinal rules in processed food: When in doubt, add sugar. “Lunchables With Dessert is a logical extension,” an Oscar Mayer official reported to Philip Morris executives in early 1991. The “target” remained the same as it was for regular Lunchables — “busy mothers” and “working women,” ages 25 to 49 — and the “enhanced taste” would attract shoppers who had grown bored with the current trays. A year later, the dessert Lunchable morphed into the Fun Pack, which would come with a Snickers bar, a package of M&M’s or a Reese’s Peanut Butter Cup, as well as a sugary drink. The Lunchables team started by using Kool-Aid and cola and then Capri Sun after Philip Morris added that drink to its stable of brands.

Eventually, a line of the trays, appropriately called Maxed Out, was released that had as many as nine grams of saturated fat, or nearly an entire day’s recommended maximum for kids, with up to two-thirds of the max for sodium and 13 teaspoons of sugar.

When I asked Geoffrey Bible, former C.E.O. of Philip Morris, about this shift toward more salt, sugar and fat in meals for kids, he smiled and noted that even in its earliest incarnation, Lunchables was held up for criticism. “One article said something like, ‘If you take Lunchables apart, the most healthy item in it is the napkin.’ ”

Well, they did have a good bit of fat, I offered. “You bet,” he said. “Plus cookies.”

The prevailing attitude among the company’s food managers — through the 1990s, at least, before obesity became a more pressing concern — was one of supply and demand. “People could point to these things and say, ‘They’ve got too much sugar, they’ve got too much salt,’ ” Bible said. “Well, that’s what the consumer wants, and we’re not putting a gun to their head to eat it. That’s what they want. If we give them less, they’ll buy less, and the competitor will get our market. So you’re sort of trapped.” (Bible would later press Kraft to reconsider its reliance on salt, sugar and fat.)

When it came to Lunchables, they did try to add more healthful ingredients. Back at the start, Drane experimented with fresh carrots but quickly gave up on that, since fresh components didn’t work within the constraints of the processed-food system, which typically required weeks or months of transport and storage before the food arrived at the grocery store. Later, a low-fat version of the trays was developed, using meats and cheese and crackers that were formulated with less fat, but it tasted inferior, sold poorly and was quickly scrapped.

When I met with Kraft officials in 2011 to discuss their products and policies on nutrition, they had dropped the Maxed Out line and were trying to improve the nutritional profile of Lunchables through smaller, incremental changes that were less noticeable to consumers. Across the Lunchables line, they said they had reduced the salt, sugar and fat by about 10 percent, and new versions, featuring mandarin-orange and pineapple slices, were in development. These would be promoted as more healthful versions, with “fresh fruit,” but their list of ingredients — containing upward of 70 items, with sucrose, corn syrup, high-fructose corn syrup and fruit concentrate all in the same tray — have been met with intense criticism from outside the industry.

One of the company’s responses to criticism is that kids don’t eat the Lunchables every day — on top of which, when it came to trying to feed them more healthful foods, kids themselves were unreliable. When their parents packed fresh carrots, apples and water, they couldn’t be trusted to eat them. Once in school, they often trashed the healthful stuff in their brown bags to get right to the sweets.

This idea — that kids are in control — would become a key concept in the evolving marketing campaigns for the trays. In what would prove to be their greatest achievement of all, the Lunchables team would delve into adolescent psychology to discover that it wasn’t the food in the trays that excited the kids; it was the feeling of power it brought to their lives. As Bob Eckert, then the C.E.O. of Kraft, put it in 1999: “Lunchables aren’t about lunch. It’s about kids being able to put together what they want to eat, anytime, anywhere.”

Kraft’s early Lunchables campaign targeted mothers. They might be too distracted by work to make a lunch, but they loved their kids enough to offer them this prepackaged gift. But as the focus swung toward kids, Saturday-morning cartoons started carrying an ad that offered a different message: “All day, you gotta do what they say,” the ads said. “But lunchtime is all yours.”

With this marketing strategy in place and pizza Lunchables — the crust in one compartment, the cheese, pepperoni and sauce in others — proving to be a runaway success, the entire world of fast food suddenly opened up for Kraft to pursue. They came out with a Mexican-themed Lunchables called Beef Taco Wraps; a Mini Burgers Lunchables; a Mini Hot Dog Lunchable, which also happened to provide a way for Oscar Mayer to sell its wieners. By 1999, pancakes — which included syrup, icing, Lifesavers candy and Tang, for a whopping 76 grams of sugar — and waffles were, for a time, part of the Lunchables franchise as well.

Annual sales kept climbing, past $500 million, past $800 million; at last count, including sales in Britain, they were approaching the $1 billion mark. Lunchables was more than a hit; it was now its own category. Eventually, more than 60 varieties of Lunchables and other brands of trays would show up in the grocery stores. In 2007, Kraft even tried a Lunchables Jr. for 3- to 5-year-olds.

In the trove of records that document the rise of the Lunchables and the sweeping change it brought to lunchtime habits, I came across a photograph of Bob Drane’s daughter, which he had slipped into the Lunchables presentation he showed to food developers. The picture was taken on Monica Drane’s wedding day in 1989, and she was standing outside the family’s home in Madison, a beautiful bride in a white wedding dress, holding one of the brand-new yellow trays.

During the course of reporting, I finally had a chance to ask her about it. Was she really that much of a fan? “There must have been some in the fridge,” she told me. “I probably just took one out before we went to the church. My mom had joked that it was really like their fourth child, my dad invested so much time and energy on it.”

Monica Drane had three of her own children by the time we spoke, ages 10, 14 and 17. “I don’t think my kids have ever eaten a Lunchable,” she told me. “They know they exist and that Grandpa Bob invented them. But we eat very healthfully.”

Drane himself paused only briefly when I asked him if, looking back, he was proud of creating the trays. “Lots of things are trade-offs,” he said. “And I do believe it’s easy to rationalize anything. In the end, I wish that the nutritional profile of the thing could have been better, but I don’t view the entire project as anything but a positive contribution to people’s lives.”

Today Bob Drane is still talking to kids about what they like to eat, but his approach has changed. He volunteers with a nonprofit organization that seeks to build better communications between school kids and their parents, and right in the mix of their problems, alongside the academic struggles, is childhood obesity. Drane has also prepared a précis on the food industry that he used with medical students at the University of Wisconsin. And while he does not name his Lunchables in this document, and cites numerous causes for the obesity epidemic, he holds the entire industry accountable. “What do University of Wisconsin M.B.A.’s learn about how to succeed in marketing?” his presentation to the med students asks. “Discover what consumers want to buy and give it to them with both barrels. Sell more, keep your job! How do marketers often translate these ‘rules’ into action on food? Our limbic brains love sugar, fat, salt… . So formulate products to deliver these. Perhaps add low-cost ingredients to boost profit margins. Then ‘supersize’ to sell more… . And advertise/promote to lock in ‘heavy users.’ Plenty of guilt to go around here!”

III. ‘It’s Called Vanishing Caloric Density.’

At a symposium for nutrition scientists in Los Angeles on Feb. 15, 1985, a professor of pharmacology from Helsinki named Heikki Karppanen told the remarkable story of Finland’s effort to address its salt habit. In the late 1970s, the Finns were consuming huge amounts of sodium, eating on average more than two teaspoons of salt a day. As a result, the country had developed significant issues with high blood pressure, and men in the eastern part of Finland had the highest rate of fatal cardiovascular disease in the world. Research showed that this plague was not just a quirk of genetics or a result of a sedentary lifestyle — it was also owing to processed foods. So when Finnish authorities moved to address the problem, they went right after the manufacturers. (The Finnish response worked. Every grocery item that was heavy in salt would come to be marked prominently with the warning “High Salt Content.” By 2007, Finland’s per capita consumption of salt had dropped by a third, and this shift — along with improved medical care — was accompanied by a 75 percent to 80 percent decline in the number of deaths from strokes and heart disease.)

Karppanen’s presentation was met with applause, but one man in the crowd seemed particularly intrigued by the presentation, and as Karppanen left the stage, the man intercepted him and asked if they could talk more over dinner. Their conversation later that night was not at all what Karppanen was expecting. His host did indeed have an interest in salt, but from quite a different vantage point: the man’s name was Robert I-San Lin, and from 1974 to 1982, he worked as the chief scientist for Frito-Lay, the nearly $3-billion-a-year manufacturer of Lay’s, Doritos, Cheetos and Fritos.

Lin’s time at Frito-Lay coincided with the first attacks by nutrition advocates on salty foods and the first calls for federal regulators to reclassify salt as a “risky” food additive, which could have subjected it to severe controls. No company took this threat more seriously — or more personally — than Frito-Lay, Lin explained to Karppanen over their dinner. Three years after he left Frito-Lay, he was still anguished over his inability to effectively change the company’s recipes and practices.

By chance, I ran across a letter that Lin sent to Karppanen three weeks after that dinner, buried in some files to which I had gained access. Attached to the letter was a memo written when Lin was at Frito-Lay, which detailed some of the company’s efforts in defending salt. I tracked Lin down in Irvine, Calif., where we spent several days going through the internal company memos, strategy papers and handwritten notes he had kept. The documents were evidence of the concern that Lin had for consumers and of the company’s intent on using science not to address the health concerns but to thwart them. While at Frito-Lay, Lin and other company scientists spoke openly about the country’s excessive consumption of sodium and the fact that, as Lin said to me on more than one occasion, “people get addicted to salt.”

Not much had changed by 1986, except Frito-Lay found itself on a rare cold streak. The company had introduced a series of high-profile products that failed miserably. Toppels, a cracker with cheese topping; Stuffers, a shell with a variety of fillings; Rumbles, a bite-size granola snack — they all came and went in a blink, and the company took a $52 million hit. Around that time, the marketing team was joined by Dwight Riskey, an expert on cravings who had been a fellow at the Monell Chemical Senses Center in Philadelphia, where he was part of a team of scientists that found that people could beat their salt habits simply by refraining from salty foods long enough for their taste buds to return to a normal level of sensitivity. He had also done work on the bliss point, showing how a product’s allure is contextual, shaped partly by the other foods a person is eating, and that it changes as people age. This seemed to help explain why Frito-Lay was having so much trouble selling new snacks. The largest single block of customers, the baby boomers, had begun hitting middle age. According to the research, this suggested that their liking for salty snacks — both in the concentration of salt and how much they ate — would be tapering off. Along with the rest of the snack-food industry, Frito-Lay anticipated lower sales because of an aging population, and marketing plans were adjusted to focus even more intently on younger consumers.

Except that snack sales didn’t decline as everyone had projected, Frito-Lay’s doomed product launches notwithstanding. Poring over data one day in his home office, trying to understand just who was consuming all the snack food, Riskey realized that he and his colleagues had been misreading things all along. They had been measuring the snacking habits of different age groups and were seeing what they expected to see, that older consumers ate less than those in their 20s. But what they weren’t measuring, Riskey realized, is how those snacking habits of the boomers compared to themselves when they were in their 20s. When he called up a new set of sales data and performed what’s called a cohort study, following a single group over time, a far more encouraging picture — for Frito-Lay, anyway — emerged. The baby boomers were not eating fewer salty snacks as they aged. “In fact, as those people aged, their consumption of all those segments — the cookies, the crackers, the candy, the chips — was going up,” Riskey said. “They were not only eating what they ate when they were younger, they were eating more of it.” In fact, everyone in the country, on average, was eating more salty snacks than they used to. The rate of consumption was edging up about one-third of a pound every year, with the average intake of snacks like chips and cheese crackers pushing past 12 pounds a year.

Riskey had a theory about what caused this surge: Eating real meals had become a thing of the past. Baby boomers, especially, seemed to have greatly cut down on regular meals. They were skipping breakfast when they had early-morning meetings. They skipped lunch when they then needed to catch up on work because of those meetings. They skipped dinner when their kids stayed out late or grew up and moved out of the house. And when they skipped these meals, they replaced them with snacks. “We looked at this behavior, and said, ‘Oh, my gosh, people were skipping meals right and left,’ ” Riskey told me. “It was amazing.” This led to the next realization, that baby boomers did not represent “a category that is mature, with no growth. This is a category that has huge growth potential.”

The food technicians stopped worrying about inventing new products and instead embraced the industry’s most reliable method for getting consumers to buy more: the line extension. The classic Lay’s potato chips were joined by Salt & Vinegar, Salt & Pepper and Cheddar & Sour Cream. They put out Chili-Cheese-flavored Fritos, and Cheetos were transformed into 21 varieties. Frito-Lay had a formidable research complex near Dallas, where nearly 500 chemists, psychologists and technicians conducted research that cost up to $30 million a year, and the science corps focused intense amounts of resources on questions of crunch, mouth feel and aroma for each of these items. Their tools included a $40,000 device that simulated a chewing mouth to test and perfect the chips, discovering things like the perfect break point: people like a chip that snaps with about four pounds of pressure per square inch.

To get a better feel for their work, I called on Steven Witherly, a food scientist who wrote a fascinating guide for industry insiders titled, “Why Humans Like Junk Food.” I brought him two shopping bags filled with a variety of chips to taste. He zeroed right in on the Cheetos. “This,” Witherly said, “is one of the most marvelously constructed foods on the planet, in terms of pure pleasure.” He ticked off a dozen attributes of the Cheetos that make the brain say more. But the one he focused on most was the puff’s uncanny ability to melt in the mouth. “It’s called vanishing caloric density,” Witherly said. “If something melts down quickly, your brain thinks that there’s no calories in it … you can just keep eating it forever.”

As for their marketing troubles, in a March 2010 meeting, Frito-Lay executives hastened to tell their Wall Street investors that the 1.4 billion boomers worldwide weren’t being neglected; they were redoubling their efforts to understand exactly what it was that boomers most wanted in a snack chip. Which was basically everything: great taste, maximum bliss but minimal guilt about health and more maturity than puffs. “They snack a lot,” Frito-Lay’s chief marketing officer, Ann Mukherjee, told the investors. “But what they’re looking for is very different. They’re looking for new experiences, real food experiences.” Frito-Lay acquired Stacy’s Pita Chip Company, which was started by a Massachusetts couple who made food-cart sandwiches and started serving pita chips to their customers in the mid-1990s. In Frito-Lay’s hands, the pita chips averaged 270 milligrams of sodium — nearly one-fifth a whole day’s recommended maximum for most American adults — and were a huge hit among boomers.

The Frito-Lay executives also spoke of the company’s ongoing pursuit of a “designer sodium,” which they hoped, in the near future, would take their sodium loads down by 40 percent. No need to worry about lost sales there, the company’s C.E.O., Al Carey, assured their investors. The boomers would see less salt as the green light to snack like never before.

There’s a paradox at work here. On the one hand, reduction of sodium in snack foods is commendable. On the other, these changes may well result in consumers eating more. “The big thing that will happen here is removing the barriers for boomers and giving them permission to snack,” Carey said. The prospects for lower-salt snacks were so amazing, he added, that the company had set its sights on using the designer salt to conquer the toughest market of all for snacks: schools. He cited, for example, the school-food initiative championed by Bill Clinton and the American Heart Association, which is seeking to improve the nutrition of school food by limiting its load of salt, sugar and fat. “Imagine this,” Carey said. “A potato chip that tastes great and qualifies for the Clinton-A.H.A. alliance for schools … . We think we have ways to do all of this on a potato chip, and imagine getting that product into schools, where children can have this product and grow up with it and feel good about eating it.”

Carey’s quote reminded me of something I read in the early stages of my reporting, a 24-page report prepared for Frito-Lay in 1957 by a psychologist named Ernest Dichter. The company’s chips, he wrote, were not selling as well as they could for one simple reason: “While people like and enjoy potato chips, they feel guilty about liking them… . Unconsciously, people expect to be punished for ‘letting themselves go’ and enjoying them.” Dichter listed seven “fears and resistances” to the chips: “You can’t stop eating them; they’re fattening; they’re not good for you; they’re greasy and messy to eat; they’re too expensive; it’s hard to store the leftovers; and they’re bad for children.” He spent the rest of his memo laying out his prescriptions, which in time would become widely used not just by Frito-Lay but also by the entire industry. Dichter suggested that Frito-Lay avoid using the word “fried” in referring to its chips and adopt instead the more healthful-sounding term “toasted.” To counteract the “fear of letting oneself go,” he suggested repacking the chips into smaller bags. “The more-anxious consumers, the ones who have the deepest fears about their capacity to control their appetite, will tend to sense the function of the new pack and select it,” he said.

Dichter advised Frito-Lay to move its chips out of the realm of between-meals snacking and turn them into an ever-present item in the American diet. “The increased use of potato chips and other Lay’s products as a part of the regular fare served by restaurants and sandwich bars should be encouraged in a concentrated way,” Dichter said, citing a string of examples: “potato chips with soup, with fruit or vegetable juice appetizers; potato chips served as a vegetable on the main dish; potato chips with salad; potato chips with egg dishes for breakfast; potato chips with sandwich orders.”

In 2011, The New England Journal of Medicine published a study that shed new light on America’s weight gain. The subjects — 120,877 women and men — were all professionals in the health field, and were likely to be more conscious about nutrition, so the findings might well understate the overall trend. Using data back to 1986, the researchers monitored everything the participants ate, as well as their physical activity and smoking. They found that every four years, the participants exercised less, watched TV more and gained an average of 3.35 pounds. The researchers parsed the data by the caloric content of the foods being eaten, and found the top contributors to weight gain included red meat and processed meats, sugar-sweetened beverages and potatoes, including mashed and French fries. But the largest weight-inducing food was the potato chip. The coating of salt, the fat content that rewards the brain with instant feelings of pleasure, the sugar that exists not as an additive but in the starch of the potato itself — all of this combines to make it the perfect addictive food. “The starch is readily absorbed,” Eric Rimm, an associate professor of epidemiology and nutrition at the Harvard School of Public Health and one of the study’s authors, told me. “More quickly even than a similar amount of sugar. The starch, in turn, causes the glucose levels in the blood to spike” — which can result in a craving for more.

If Americans snacked only occasionally, and in small amounts, this would not present the enormous problem that it does. But because so much money and effort has been invested over decades in engineering and then relentlessly selling these products, the effects are seemingly impossible to unwind. More than 30 years have passed since Robert Lin first tangled with Frito-Lay on the imperative of the company to deal with the formulation of its snacks, but as we sat at his dining-room table, sifting through his records, the feelings of regret still played on his face. In his view, three decades had been lost, time that he and a lot of other smart scientists could have spent searching for ways to ease the addiction to salt, sugar and fat. “I couldn’t do much about it,” he told me. “I feel so sorry for the public.”

IV. ‘These People Need a Lot of Things, but They Don’t Need a Coke.’

The growing attention Americans are paying to what they put into their mouths has touched off a new scramble by the processed-food companies to address health concerns. Pressed by the Obama administration and consumers, Kraft, Nestlé, Pepsi, Campbell and General Mills, among others, have begun to trim the loads of salt, sugar and fat in many products. And with consumer advocates pushing for more government intervention, Coca-Cola made headlines in January by releasing ads that promoted its bottled water and low-calorie drinks as a way to counter obesity. Predictably, the ads drew a new volley of scorn from critics who pointed to the company’s continuing drive to sell sugary Coke.

One of the other executives I spoke with at length was Jeffrey Dunn, who, in 2001, at age 44, was directing more than half of Coca-Cola’s $20 billion in annual sales as president and chief operating officer in both North and South America. In an effort to control as much market share as possible, Coke extended its aggressive marketing to especially poor or vulnerable areas of the U.S., like New Orleans — where people were drinking twice as much Coke as the national average — or Rome, Ga., where the per capita intake was nearly three Cokes a day. In Coke’s headquarters in Atlanta, the biggest consumers were referred to as “heavy users.” “The other model we use was called ‘drinks and drinkers,’ ” Dunn said. “How many drinkers do I have? And how many drinks do they drink? If you lost one of those heavy users, if somebody just decided to stop drinking Coke, how many drinkers would you have to get, at low velocity, to make up for that heavy user? The answer is a lot. It’s more efficient to get my existing users to drink more.”

One of Dunn’s lieutenants, Todd Putman, who worked at Coca-Cola from 1997 to 2001, said the goal became much larger than merely beating the rival brands; Coca-Cola strove to outsell every other thing people drank, including milk and water. The marketing division’s efforts boiled down to one question, Putman said: “How can we drive more ounces into more bodies more often?” (In response to Putman’s remarks, Coke said its goals have changed and that it now focuses on providing consumers with more low- or no-calorie products.)

In his capacity, Dunn was making frequent trips to Brazil, where the company had recently begun a push to increase consumption of Coke among the many Brazilians living in favelas. The company’s strategy was to repackage Coke into smaller, more affordable 6.7-ounce bottles, just 20 cents each. Coke was not alone in seeing Brazil as a potential boon; Nestlé began deploying battalions of women to travel poor neighborhoods, hawking American-style processed foods door to door. But Coke was Dunn’s concern, and on one trip, as he walked through one of the impoverished areas, he had an epiphany. “A voice in my head says, ‘These people need a lot of things, but they don’t need a Coke.’ I almost threw up.”

Dunn returned to Atlanta, determined to make some changes. He didn’t want to abandon the soda business, but he did want to try to steer the company into a more healthful mode, and one of the things he pushed for was to stop marketing Coke in public schools. The independent companies that bottled Coke viewed his plans as reactionary. A director of one bottler wrote a letter to Coke’s chief executive and board asking for Dunn’s head. “He said what I had done was the worst thing he had seen in 50 years in the business,” Dunn said. “Just to placate these crazy leftist school districts who were trying to keep people from having their Coke. He said I was an embarrassment to the company, and I should be fired.” In February 2004, he was.

Dunn told me that talking about Coke’s business today was by no means easy and, because he continues to work in the food business, not without risk. “You really don’t want them mad at you,” he said. “And I don’t mean that, like, I’m going to end up at the bottom of the bay. But they don’t have a sense of humor when it comes to this stuff. They’re a very, very aggressive company.”

When I met with Dunn, he told me not just about his years at Coke but also about his new marketing venture. In April 2010, he met with three executives from Madison Dearborn Partners, a private-equity firm based in Chicago with a wide-ranging portfolio of investments. They recently hired Dunn to run one of their newest acquisitions — a food producer in the San Joaquin Valley. As they sat in the hotel’s meeting room, the men listened to Dunn’s marketing pitch. He talked about giving the product a personality that was bold and irreverent, conveying the idea that this was the ultimate snack food. He went into detail on how he would target a special segment of the 146 million Americans who are regular snackers — mothers, children, young professionals — people, he said, who “keep their snacking ritual fresh by trying a new food product when it catches their attention.”

He explained how he would deploy strategic storytelling in the ad campaign for this snack, using a key phrase that had been developed with much calculation: “Eat ’Em Like Junk Food.”

After 45 minutes, Dunn clicked off the last slide and thanked the men for coming. Madison’s portfolio contained the largest Burger King franchise in the world, the Ruth’s Chris Steak House chain and a processed-food maker called AdvancePierre whose lineup includes the Jamwich, a peanut-butter-and-jelly contrivance that comes frozen, crustless and embedded with four kinds of sugars.

The snack that Dunn was proposing to sell: carrots. Plain, fresh carrots. No added sugar. No creamy sauce or dips. No salt. Just baby carrots, washed, bagged, then sold into the deadly dull produce aisle.

“We act like a snack, not a vegetable,” he told the investors. “We exploit the rules of junk food to fuel the baby-carrot conversation. We are pro-junk-food behavior but anti-junk-food establishment.”

The investors were thinking only about sales. They had already bought one of the two biggest farm producers of baby carrots in the country, and they’d hired Dunn to run the whole operation. Now, after his pitch, they were relieved. Dunn had figured out that using the industry’s own marketing ploys would work better than anything else. He drew from the bag of tricks that he mastered in his 20 years at Coca-Cola, where he learned one of the most critical rules in processed food: The selling of food matters as much as the food itself.

Later, describing his new line of work, Dunn told me he was doing penance for his Coca-Cola years. “I’m paying my karmic debt,” he said.