"It's always fun," said Josh Reddick, whose RBI single in the 10th was the game-winner. "This is the first time for me starting in this series."
The victory gave the Red Sox sole possession of first place in the AL East, one game ahead of the Yankees, and clinched a win in the season series against New York for the first time since 2004.
The Yankees led 2-1 on homers by Eduardo Nunez and Brett Gardner before Rivera came in to close it out in the ninth. But Marco Scutaro led off with a double — his fourth hit of the game. He moved up on a sacrifice bunt and scored on a sacrifice fly to send the game into extra innings and hand Rivera his fifth blown save of the season.
"It's unusual," Yankees manager Joe Girardi said. "We're so used to seeing him do it, time after time after time."
In the 10th, Ortiz lined a one-out double off Phil Hughes (2-4) over the short fence in right field. Darnell McDonald pinch ran, and the Yankees intentionally walked Carl Crawford, who already had three hits.
Reddick then hit the first pitch into the corner in left, driving in McDonald. The Red Sox rookie would have had a double if he had merely touched second base, but instead he veered off to the outfield to avoid his teammates as they swarmed out of the dugout to congratulate him.
"I knew it was coming," said Reddick, who was all-too happy to be at the middle of the head-pounding and chest-bumping celebration. "Some of the guys get pretty banged up after walking off."
Reddick, who started the season at Triple-A, has been playing regularly since J.D. Drew struggled and then went on the disabled list. The 24-year-old outfielder is batting .338 with five homers in 45 games.
"He's been great, man," Scutaro said. "We have to give credit to all our guys. When our regular guys have gone down, guys have been good."
Daniel Bard (2-5) pitched a scoreless 10th for the win.
Boston lost the series opener but won the next two to snap the Yankees' eight-game winning streak and take the series. The Red Sox are 10-2 against New York this season with six games left.
Gardner had three hits for the Yankees, including the tiebreaking drive with two outs in the seventh. He also stole two bases, giving him 35 on the season.
Nunez tied it with his homer in the fifth — one batter after Russell Martin was thrown out by Reddick trying to stretch a single into a double.
Meanwhile, the Red Sox kept producing baserunners — and stranding them. They left the bases loaded in the second and sixth, and stranded a pair in the fourth.
NOTES: Mexican billionaire Carlos Slim was in the Red Sox clubhouse before the game, then posed for pictures with Ortiz. ... New Patriots wide receiver Chad Ochocinco was at the game, drawing a big cheer when he was shown on the scoreboard. ... Crawford hit safely in seven consecutive plate appearances — two on Friday, four on Saturday and his first at-bat on Sunday — before he grounded out in the fourth inning. ... Jacoby Ellsbury made a nice run to catch Nick Swisher's line drive in the triangle to end the fourth inning. ... Yankees SS Derek Jeter made a play to his right into shallow left field to get Gonzalez's grounder and force Dustin Pedroia out at second. More good defense: 1B Mark Teixeira's diving stop on Ellsbury, and Kevin Youkilis diving behind third on Jeter, both in the first inning. ... Neither starter, Josh Beckett (6 IP, 1ER) and Freddy Garcia (5 IP, 1 ER), received a decision. ... The Yankees are off on Monday, and Hughes was supposed to pitch against the Angels on Tuesday. The Red Sox play three against Minnesota, with Tim Wakefield facing Scott Baker in the opener Monday night. Wakefield will be going for his third try at win No. 200. He would be the 108th pitcher in major league history to reach the milestone. ... Rivera made his 1,023rd appearance, moving him into 10th on the career list; he had been tied with Jose Mesa and Lee Smith.