The movie opens with Michael in his 50's or so getting Papal honors at his church (the same church as the Christening in Godather I.) After the ceremony, there's a big celebration and we meet Michael's grown children Mary and Anthony (Tony) and also Sonny's illegitimate son, Michael's nephew Vincent Mancini. We also meet Joey Zasa, another mafioso with whom Vincent has a big problem. Zasa send two thugs to kill Vincent that night and they don't succeed.
Michael's son Anthony doesn't want to go into the family business, which is FINALLY legitimate, instead he wants to be a singer. He gets a not-so-eager go-ahead from his father and soon gets a lead part singing in an opera in Sicily. Kay tells Michael how much she "dreads" him. Michael tells Kay he wants to change.
Tom Hagan has already passed away, and his son Andrew is a priest for whom Michael gets a position at the Vatican. Michael wants to take over a European company Immobiliere and he works out a deal to get the Vatican's shares in exchange for helping them out with their debt. They just have to sway some votes.
There's a meeting of the heads of the families for Michael to tell everyone that he's "out" and won't be working with any of them anymore, so he cashes everyone out on their stocks, and everybody gets shares based on what they put into the casinos. Vincent rejoices in telling Joey Zasa that he gets NOTHING. Joey leaves angry and threatens everyone that they're either with him or against him, and suddenly there's a hit-by-helicopter on everyone in the room, and most everyone dies except for Don Altobello (Connie's godfather), who cuts a deal with Zasa Vincent and Michael, and Al Neri.
Michael has problems with Diabetes, and he has a diabetic stroke and goes into a coma. It doesn't look good for him. He gets better, though.
Mary and Vincent fall in love, even though they're first cousins. Michael tells Vincent that if he wants to be in the family, things can't go on between him and Mary, because when "they come after you, they come after those you love." Vincent agrees.
The family travels to Sicily, even Kay. Michael takes Kay and shows her the sights. We see a wedding that takes place where Michael and Apollonia were wed in Godfather I. Michael has a lot of flashbacks of Apollonia in this movie. Michael is basically reviewing his life, knowing he won't live much longer since he's growing older, and wondering if it was all worth it, and if he's going to Hell, etc.
As the family witnesses a death of Michael's Mentor, Michael realizes he's unsafe in Italy. Don Luchese hires a professional assassin to kill Michael. After the pope, who was the main character in Immobiliare's vote dies, a pope who's not loyal to the Corleone family takes over. When Michael and his family attend his son's opera, it's at this point that Vincent breaks things off with Mary. The assassin hired to kill Michael arrives in New York, while Vincent's associates(He is proclaimed the new Godfather by Michael) kill the swiss banker, the pope, the archbishop, and Don Luchese. That "settles the Corleone Family's accounts". Yet, after the opera, before police crawl all over the place, The Italian Assassin shoots Mary, who gets shot by Vincent.