The film picks up almost directly after the end of The Karate Kid; John Kreese (Martin Kove), furious over his star pupil Johnny Lawrence's (William Zabka) second place finish in the All Valley Karate Tournament, viciously berates and humiliates Johnny in the parking lot, also nearly killing him by putting him in the headlock choking position. Despite pleads from Tommy, Bobby and the rest of Johnny's friends to let him go, Kreese refuses, hitting Johnny's friends away as they approach. Miyagi, who is leaving the venue with Daniel, rescues Johnny, passively immobilizes Kreese, then comically tweaks Kreese's nose instead of dealing him a fatal blow. Horrified by Kreese's behavior, Johnny and his friends quit the Cobra Kai dojo en masse, while Miyagi leaves with Daniel. When Daniel asks why Miyagi did not kill Kreese when he could have, Miyagi explains, "For person with no forgiveness in heart, living even worse punishment than death."
Six months later, Daniel is upset that his girlfriend Ali has left him for a football player from UCLA after the senior prom. To make matters worse, he learns that he and his mother are soon moving to Fresno for the summer. Miyagi surprises Daniel by telling him that he has made arrangements with Mrs. Larusso to have Daniel live with him for summer. Miyagi then receives a letter telling him his father is dying. He intends to return to Okinawa alone, but Daniel decides to accompany him. When Daniel asks Miyagi why he had left Okinawa in the first place, Miyagi answers that he loved a woman named Yukie, who was arranged to be married to Sato, son of the richest man in town, and Miyagi's best friend. Sato and Miyagi had studied karate together under Miyagi's father, in defiance of what was then the strict one-to-one father-to-son tradition of karate. One day, Miyagi had announced before the whole town that he wanted to marry Yukie. Sato had been insulted and had challenged Miyagi to a fight to the death. Rather than fight his best friend, Miyagi had left Okinawa.
When they arrive in Okinawa, Miyagi and Daniel are greeted by a young man, Chozen Toguchi (Yuji Okumoto), who is Sato's (Daniel Kamekona) nephew. Sato has neither forgiven nor forgotten his feud with Miyagi and once again demands to fight Miyagi. Again, Miyagi refuses, so Sato calls him a coward.
Miyagi and Daniel are welcomed to Tome village by Yukie (Nobu McCarthy) and her niece Kumiko (Tamlyn Tomita), to whom Daniel is immediately attracted. They learn that Sato has now become a rich industrialist, whose supertrawlers have destroyed the local fish population, impoverishing the other villagers, who have turned to small farming to survive. Worse yet, all the villagers are now forced to rent their property from Sato, who now actually owns the entire village. Yukie also reveals that because she truly loved Miyagi and carried a torch for him, she never married Sato.
Despite Miyagi's father's dying wish for his son and student to make peace with each other, Sato still insists on fighting Miyagi, though, after his sensei's passing, he gives Miyagi three days to mourn. Daniel comforts Miyagi, admitting that when his own father died, Daniel thought he had not been a very good son, but eventually realized that by being at his father's side when he was dying and getting to say goodbye to him was the greatest thing he could have done for him. Miyagi shows Daniel that the secret to his family's karate lies in a handheld drum that beats itself when twisted back and forth. This "drum technique," as Miyagi calls it, represents the block-and-defense that Daniel begins to practice diligently. Miyagi warns him that the powerful technique should only be used as a last resort. Later, Yukie and Miyagi perform the tea ceremony together, which, Kumiko explains to Daniel, is a sign that they are renewing their love.
Daniel inadvertently reveals that the grocery business of Chozen and his cronies, Taro and Toshio, has been defrauding the villagers with rigged weights. The outraged farmers set upon Chozen and demand appropriate compensation. Because of this, Chozen accuses Daniel of both insulting his honor and being a coward like his sensei. He and Daniel have a series of confrontations, first in the village, then later in Naha City, and at a '50s-themed dance. Chozen attempts to humiliate Daniel by demanding he demonstrate his karate skills by chopping through six blocks of ice, a seemingly impossible feat. However, Mr. Miyagi appears just in time to express confidence in Daniel by taking Chozen up on his bet at a dollar amount which Chozen cannot cover, but which Sato agrees to cover Chozen. Daniel successfully fulfills the challenge, which Chozen protests, but Sato honors the terms of the wager.
Meanwhile, Daniel and Kumiko begin to grow closer. She brings him to an old castle on the seacoast that Sato is allowing to deteriorate and be plundered. Both Daniel and Kumiko express incredulity at why Sato would allow a historical relic like the castle to suffer thus.
The feud between Daniel and Chozen eventually comes to a head when Sato, at the conclusion of the three-day mourning period, shows up to fight Miyagi. Because Miyagi is not present, Chozen and his cronies destroy the Miaygi family dojo and much of the garden, then Chozen viciously attacks Daniel when he tries to intervene. When Miyagi arrives, Chozen, Taro, and Toshio attack him, but Miyagi defeats them easily, even as Chozen wields a spear. Realizing that he has put Daniel in grave danger, Miyagi makes plans to return home.
Before they can leave Okinawa, however, Sato shows up with earth-mover machines and threatens to destroy and redevelop the village if Miyagi continues to refuse to fight. Miyagi reluctantly gives in, but only on the condition that no matter who wins, Sato must sign the titles to the villagers's homes back over to them. Sato agrees to this condition. On the day the fight is to take place, Daniel and Kumiko, like Yukie and Miyagi, perform the ancient tea ceremony, ending with a kiss. Meanwhile, a typhoon strikes the village. The villagers take cover at a storm shelter, but Sato is still at his family's dojo. When the Sato family dojo is leveled by the storm, trapping Sato inside, Miyagi and Daniel rush to rescue him. Sato believes that Miyagi has decided to unfairly defeat him while he is incapacitated, but Miyagi instead breaks a support beam that had pinned Sato down, freeing him.
After the three return to safety, Daniel goes out again, this time to rescue a child trapped in the bell tower. Sato orders Chozen to go help Daniel, but Chozen refuses, not wanting to cooperate with Daniel in any capacity. Just as Miyagi goes on to help Daniel, Sato insists in helping him instead as gratitude for saving his life. After the child is safe, Sato disowns Chozen for refusing to cooperate. Humiliated, Chozen later runs off into the storm in anger
The next morning, the villagers set about rebuilding the village, and Sato returns with the bulldozers, not to raze the village, but to help get rid of debris and repair storm damage. Sato hands over the titles to the villagers' homes, and also humbly asks Miyagi for forgiveness. Though Miyagi insists that there is nothing to forgive, he accepts his old friend's apology. Daniel asks Sato if the village may hold their upcoming O-bon festival on the castle grounds. Sato agrees, and grants them this right in perpetuity. Sato has one condition, however: that Daniel join him and the other villagers in the celebration.
At the O-bon festival, Kumiko is on stage performing a traditional dance when the now-deranged and vengeful Chozen evilly interrupts, taking her hostage at knifepoint. Sato tells Chozen that he was wrong to hate Miyagi and implores Chozen to similarly let go of his hatred for Daniel. But after being disowned, Chozen sees no point in listening to his uncle and refuses, saying that doing so will not give back his "honor".. Now thirsty for revenge and restoration of his honor, Chozen then threatens to kill Kumiko if Daniel does not step up to fight him to the death. Daniel agrees, in spite of Miyagi's warning that this time is no tournament, but instead very real. Chozen has turned from a cowardly bully to a hate-filled enemy who indeed wants to kill Daniel.
Daniel fights valiantly, but Chozen proves to be a much more formidable opponent than any other that he has faced before; he even deflects the crane kick Daniel used to win in the tournament. Just when Daniel is on the verge of defeat, Miyagi brings out his hand drum and beats it. The other villagers follow suit with their own drums, which allows Daniel to realize how he can win. As the puzzled Chozen closes in for the kill, Daniel successfully utilizes the drum technique to deflect Chozen's attacks and land a series of devastating counter-attacks. Daniel, realizing for the first time in his life that his karate skills are potentially capable of enabling him to kill another person, grabs the vanquished Chozen by his hair and cocks his hand back for the fatal blow, demanding of Chozen, "Live or die, man!" When Chozen responds with "die", Daniel responds the same way Miyagi did against Kreese; he fakes out Chozen before tweaking his nose and dropping him to the ground, unwilling to kill him. Daniel embraces Kumiko, while Miyagi looks on proudly.