"They were the ones that provoked it," Phillies manager Charlie Manuel said. "I feel (Victorino) kind of did what he had to do. He had to stand up for himself."
The two-times All Star was hit in the back by a pitch from San Francisco's Ramon Ramirez and began walking toward the mound. That came after Philadelphia's Jimmy Rollins stole second base with a six-run Phillies' lead.
Both benches cleared and Victorino pushed the home plate umpire while trying to join in.
Victorino, Ramirez and Whiteside were ejected from the game, which the Phillies won 9-2 in a showdown between National League division leaders.
Victorino's aggressive actions prolonged Friday's incident, MLB said in a statement. All four players were fined undisclosed amounts by MLB.
Victorino, who declined to comment, finished Monday's game 3-for-5 and is batting .310 this season. He will be allowed to remain in the lineup for the Major League-leading Phillies (75-40) during the appeal process.
(Writing by Jahmal Corner in Los Angeles and Gene Cherry in Raleigh, North Carolina; Editing by Peter Rutherford)