Yes and no. Without Rivera, Lovie still had 2 top 5 defenses. Also, his defenses after Rivera always ranked high in takeaways. 8th, 2nd, 12th,3rd, 5th, and 1st. Lovie's schemed changed over time as well. I used to do a ton of videos back in the day. We started using much more Cover 3 especially when we draft Danieal Manning. Manning struggled, but had the tools to play the deep 3rd of the field. People thought Lovie was strictly a Cover 2 guy and that was not the case.
People would get pissed at Lovie at giving up big plays like the Super Bowl TD that Wayne scored. Everyone wanted to blame Lovie for staying in Cover 2 but honestly that wasn't a bad call. The Colts back up tight end ran down the seam and Manning took too many steps towards him. Meanwhile Reggie Wayne is running a go route. Tillman passes him off to Manning. But Manning isn't there. Wayne is wide open. You see Chris Harris running into the shot from the TV angle but he was never supposed to Cover Wayne. He recognized what was happening and tried to run across the field to stop it. Mike Brown's injury and never being able to find his replacement hurt Lovie's defenses probably more than anything.....and Babich was trash.