While tennis definitely experienced a "boom" in the 1970s and remained very popular through the mid 1980s, its U.S. popularity had subsided a bit by the late '80s, and numbers started tailing off. However, they've been fairly stable now since the mid-'90s. It's actually primarily the '70s and part of the '80s that had elevated popularity, and if anything, some happenings in the late '80s and '90s may have contributed to the sport's decline in the U.S.
As for the causes of the original tennis boom in the '70s and into the early '80s, there was a perfect storm of factors at play in tennis that has not since been replicated, and may never be able to be. To understand the rise of professional tennis, we actually have to go back to 1968, when two huge things happened that changed the face of tennis in the U.S. and worldwide:
The first is that 1968 was year the "Open Era" started. Prior to that, professional players were not allowed to compete in any of the four Grand Slam tournaments, nor in Davis Cup play. This meant that, unlike in other professional sports, tennis' top titles did not feature all of tennis's top players. Once the Open Era came about, it made it possible for a player to make a living competing at tennis, and ushered in a level of professionalism and interest that hadn't existed before. Until then, tennis was a cloistered rich-man's country club sport that had only niche interest. But the Open Era allowed it to become a proper professional sport, like other major sports had been for decades. While Grand Slam titles had been contested for almost a century at that point, many of the best players in the world hadn't been competing, because the pros were out "barnstorming" in exhibitions.
The second thing that happened in 1968 is that the US Open was broadcast on television for the first time (by CBS). It would soon be followed in 1969 by the first Wimbledon broadcast on NBC, and eventually by the French Open on NBC in 1975. So the mid-'70s was the first time that the sport's most prestigious competitions were all available to be seen by the American public, sparking a new wave of interest among the general population rather than just among the country club set. Keep in mind that at this time, there were really only a few channels available on most poeple's TVs, so just being chosen for broadcast meant that millions of people would be exposed to the sport when they turned on their sets. Since that time, many other sports have achieved professional status, and essentially become competitors for tennis among U.S. audiences.
So now that we have touched on the tennis boom, and those two big happenings as its catalysts, we should consider a number of other factors that contributed to tennis's huge popularity, and its eventual decline.
1. A Cast of Characters: For a sport to be hugely popular, and appeal to audiences beyond the sport's active participants and avid fans, it usually needs recognizable stars with distinctive personalities. Characters whom the public can latch on to and remember in the absence of a nuanced understanding of the game. Someone to cheer for. Someone to cheer against. And tennis was lucky enough to have those in spades at the time of the boom. There was Bjorn Borg's icy mechanical demeanor in the face of incredible pressure, and his unprecedented athleticism. There was McEnroe's bratty mercurial brilliance and deft touch. There was Connors's competitive fire, and brash Americanness. There was Nastase's bad boy attitude and incredible hand skills. On the women's side, there was wholesome American sweetheart Chris Evert, feminist symbol Billie Jean King, and communist turncoat Martina Navratilova who redefined athleticism for women, as well as many others. Heroes and villains galore. Capitalizing on this newfound popularity, the media seized the chance to talk about tennis constantly, and tennis's stars were featured all the time in general interest newspapers, magazines and tv shows. Tennis players became celebrities, and were encouraged to express themselves openly and without fear of reprisal, making people want to tune in just to see what they'd do. Fans felt they knew the players and their different personalities. And that's what makes people pick a side. When the late 80s and early 90s rolled around, those characters were replaced at the top of the game with a dour quiet Czech, a German kid who let his racquet do the talking, and a bland Swede with no major weapons and a personality that lacked any real fire. It wasn't until Andre Agassi reached his potential in the mid-90s that tennis found another prominent character. And by then the damage had been done, and many casual viewers already lost. Agassi alone was responsible for some renewed interest in tennis in the 90s, but he wasn't a consistent enough contender to make Grand Slam finals consistently. Pete Sampras and Jim Courier, for all their great qualities, just weren't very interesting characters, nor were any of the players who followed them until Roddick, Federer, and Nadal came along years later.
2. Hometown Heroes: Many of the game's top stars and defining characters in the '70s and '80s were American. So Americans had skin in the game (and a reasonable chance that their favorite players would win the major titles). In fact, the year that tennis's television ratings started dropping consistently in the U.S. was 1985, the year that Ivan Lendl took over the top world ranking from John McEnroe, who never won another Grand Slam title again. If the guy you're interested in doesn't even make it to the final Sunday telecast, why watch? Americans are known to care mainly about sports in which they are competitive. We only need to look at the Olympics to see that the whole country will cheer for someone in an obscure sport if we have a chance to win it. But even the most popular world sports are slow to draw loyal fans in the U.S. if we don't have a chance to dominate the game (see soccer). By the time Agassi, Courier, and Sampras came along in the mid-'90s, the ratings had already fallen to about the current levels. Further, since the U.S. market drives tennis worldwide more than any other market, when the sport is popular here, it means greater success for the sport everywhere else.**
3. A Variety of Playing Styles: Not only were the personalities in tennis varied, but so were the playing styles that were employed effectively. Bjorn Borg used his incredible foot speed and endurance, combined with unprecedented topspin to wear down opponents like a tennis machine. McEnroe used the racquet like a giant hand to deftly drop volleys in seemingly inaccessible parts of the court, coming in behind every serve. Connors bashed the ball dead flat from all parts of the court. Lendl created howitzer-like modern topspin drives that overwhelmed everyone on both wings. Chris Evert won by hitting deep flat balls with precise placement, while Martina Navratlova and Billie Jean King served and volleyed their ways to the net whenever possible. Meanwhile, Andrea Jaeger and a host of other players moonballed their way to hundreds of victories. As a fan, you would choose to root for the player whose game you most admired or enjoyed watching, and each player had a distinctive style. Some would say that tennis became much less interesting to watch when Becker started winning with just his serve. Or that the power serving era of the 90s was bad for tennis, because points became so short, and long rallies so rare. Those things are true, but are made even more so when there's no contrast to them. And sure enough, we now find ourselves in an era where there is really only one playing style on each tour. Even the stroke production is limited now. A one-handed backhand is considered almost a novelty these days, and the backhand slice almost a relic apart from the occasional bailout or change of pace. Aside from a few players, volleys are only used to end points that have already been won strategically. This homogenization of playing styles makes the game much less interesting for the casual fan to watch, since they are less able to grasp the differences and nuances inherent to different players' games. Gone are the days when one player was much more fun to watch than another on a physical level.
4. A Variety of Playing Speeds: This is a bit of a circular problem, but the fact that the difference in court speed at the four majors is now much less than it was during tennis's heyday means that it's no longer possible to be a surface specialist or successfully employ a unique style of play. Even though McEnroe never won Roland Garros and Lendl never won Wimbledon, their different styles were both considered viable options on tour because of the different courts one had to succeed on. And because of these differences, the points looked different at each tournament and were fun to watch for different reasons, emphasizing the uniqueness of the characters and playing styles involved. Furthermore, there are those who would say that the value of winning a Grand Slam has now been diminished because it's not quite as complete a test of tennis mastery.
5. Increased Professionalism and Oversight: Tennis in the 1970s and early 1980s was really a bit like the wild west. Things were new, and tournaments, players, and officials were all still figuring out what professional tennis's standards would be. Although it's an unavoidable symptom of the increased money in the game, the added professionalism in tennis has arguably made it less compelling for the average fan. In the '70s and '80s, it was common to see Borg, Gerulaitis, or McEnroe out partying at a nightclub like Studio 54. Tabloids were rife with rumors of drug use and romantic intrigue among players. This all contributed to their popularity as characters (see #1) even if it risked hampering their performance the next day. As athletes became more mono-focused and professional, and top results became dependent on a much more ascetic lifestyle, players became less visible and their lives less interesting to outside observers. This meant that a lot of casual fans lost interest, because they no longer were being entertained as much and weren't as exposed to the players' lives except in controlled PR situations and photo ops. Furthermore, with the advent of athletic professionalism and the institution of the ATP as the tour's governing body, much greater pains were taken to control the images of the sport's participants. Whether it meant the tour covering up Agassi's drug use, or players at press conferences being discouraged from being candid with the press, the public lost access to the real personalities of the players. And rule changes in the Code of Conduct meant that a lot of the most entertaining and colorful behavior was forced out of the game. The new rules meant that players could no longer get away with the colorful behavior that made the sport's headlines until the mid 80s. While many would say that the tantrums thrown by McEnroe, or the fiery tirades of Connors, or the antics of Nastase were unseemly, they were big draws for the game. If you showed up, you never knew what you might see. Players made no bones about not liking each other, and this added to the rivalries on the court. Now, they're discouraged from saying anything negative about each other at all, and risk being censured if they're honest about each other or the tour's organization. In addition, many players self-edit for fear of risking lucrative sponsorship opportunities, many of which pay them more than their tour winnings. They're no longer real people or characters, but one-dimensional tennis-playing robots in the public's eyes, and the distinguishing differences are tiny subtleties apparent only to devoted fans.
6. The Societal Roles of Tennis: When tennis hit its boom, it wasn't just a sport, it was a symbol. It was the only major professional sport in the public eye in which both men and women competed (and even against each other in the case of mixed doubles). At a time when tennis was popular anyway, and women's liberation was a huge cultural phenomenon in the U.S., tennis was like a symbol of the era. Not only were women playing sports, but they were being paid for it. At a time of women's lib, the press latched on to this, and even made it viable to stage publicity stunts like the Battle of the Sexes (still one of the highest rated tennis events ever). Nowadays, women compete professionally in many sports, and the novelty has worn off.
7. The Changing Role of the Press: When tennis was booming, so was sports reporting. Most major newspapers had large sports departments, with pages devoted to each sport. As the internet took over and newspapers downsized, they eliminated secondary sports coverage. And in the U.S., tennis was still a secondary sport compared to football, baseball, and basketball. Many major newspapers that had a dedicated tennis writer, and featured tennis coverage in the daily paper were forced to cut them. A casual reader leafing through the sports section in the 70s and 80s was likely to come across tennis coverage at some point. But when papers started downsizing, they laid off many reporters and no longer sent anyone to tennis tournaments in order to have unique coverage. Instead, they just picked up syndicated coverage that was largely the same in every paper. Further, people started getting their news from the internet, where they would just seek out the specific sports they were following rather than reading through the sports pages in sequential/analog fashion. So the non-devoted tennis fans were coming across tennis writing and tennis scores much less often in the U.S. It was no longer top of mind.
8. Sports Life Cycles and Competition: Although it's not talked about much, many major spectator sports go through life cycles of popularity. With so many sports available now compared to the '70s and '80s, people pick up and drop sports of interest much more often. When a sport is newly popular, it gains a lot of casual viewers, who eventually stop watching it when something else more interesting comes along. After that happens, only the sport's devoted fans remain. Boxing was huge in the 70s, but it seemed to run its course, and dropped greatly in prominence. In boxing's case, the absence of compelling characters probably also contributed, like it did in tennis, as did the rise of MMA. Similarly, hockey in the U.S. was more popular then than it currently is. The sports with momentum get stronger, and the ones on the way down get dropped, eventually left with just the core audience (at least until another compelling character comes along). With tennis already weakened by many of the factors above, it lost momentum in the U.S. And while participation and spectatorship are separate factors, they do affect each other. One of the original barriers to tennis's acceptance was its perception as an elitist country club sport. While the popularization of the game during the 70s and 80s did a lot to combat this, sports associated with the wealthy took a hit after the 1980s.
9. Available Choices: It was mentioned above that just being on T V in the 70s and 80s meant that millions of Americans would watch tennis. But now, with hundreds of channels available in the U.S. as well as online options, people have a huge number of choices of what to watch. In addition, the U.S. has more major sports than many other countries. While baseball, football, basketball, hockey, soccer, and a host of other sports all provide considerable competition for tennis in the U.S., many other countries only have a few major pro sports. And many also don't have the infrastructure or population to support large team sports leagues. In the countries where people have far fewer channels, and far fewer team sports leagues, tennis is still quite popular, and (along with soccer and auto racing) is one of the most televised sports on general networks.