BALTIMORE -- The Baltimore Orioles' fierce lineup has been especially effective against the New York Yankees.

One again, Baltimore found a way to beat its American League East rival.

Hyun Soo Kim hit a go-ahead RBI single in the seventh inning and the Orioles overcame a multi-run deficit for the second straight day to beat the Yankees 5-4 on Saturday.

The Orioles have now won 10 straight series against the Yankees at Camden Yards. New York has not won a series in Baltimore since 2013. The Orioles are also 4-0 for the second straight year.

"There are 158 games left," Orioles manager Buck Showalter said. "Every game you want to win. I'd rather win the first four than lose the first four. We'll take each challenge as it comes."

Trailing 4-1 in the fifth inning, the Orioles pulled within 4-3 on a double by Manny Machado and groundout by Mark Trumbo off Masahiro Tanaka. After loading the bases, Tanaka pitched out of the jam by striking out Kim and getting a popout from Jonathan Schoop.

Tanaka allowed the three runs and six hits with five strikeouts and four walks in five innings.

"I felt like I had good command of the ball going into the game, the first three innings, obviously," Tanaka said through an interpreter. "As you can see, the fourth inning, fifth inning, that command kind of starting getting out of sync. So it was probably the issue of command."

The Orioles tied the score 4-4 in the seventh on an RBI single by Trumbo off Dellin Betances (0-1). After Trumbo stole second, Kim followed with a blooper to center that proved to be the winning run.

"I was just trying to put the ball in play, trying to make good contact," Kim said through an interpreter. "He obviously throws very hard, so I was looking for something hard, but at the same time thinking about something slow as well. It worked out."

Mychal Givens (1-0) picked up the win with a scoreless seventh. Zach Britton posted this third save in as many chances.

The Yankees fell to 1-4 on the season. Nonetheless, Girardi is confident the team will bounce back.

"It's frustrating, but that's baseball," Girardi said. "It will turn around and we just have to keep fighting."

New York catcher Gary Sanchez had to leave the game in the fifth with a right biceps strain and was replaced by Austine Romine. Sanchez's status is uncertain. Kyle Higashioka will be recalled from Triple-A Scranton/Wilkes-Barre to take Sanchez's spot.

"It's an opportunity for other guys to step up," Girardi said. "Romine and (Higashioka) are going to have to step up in his absence. It's not what you want to see."

The Yankees took the lead in the second on a two-run single by Ronald Torreyes off Kevin Gausman. It was a particularly challenging inning for Baltimore catcher Welington Castillo, who couldn't get a tag down on Aaron Hicks for the second run and then had a throwing error on a steal by Brett Gardner.

Castillo made amends with a two-out RBI single that cut the margin to 2-1 in the fourth. Gausman, however, was called for a balk the following inning on a throw to third that allowed Austin Romine to score. Starlin Castro followed with an RBI single for a 4-1 lead that ended Gausman's day.

Gausman allowed four runs, eight hits and three walks with four strikeouts.

"I just couldn't throw a fastball down and away to save my life," Gausman said. "From the first pitch on, trying to figure out my mechanics, figure out what I needed to do with my front side, thought I was hitting a little too quick, but maybe I was hitting a little bit too slow."

NOTES: Orioles RHP Chris Tillman (right shoulder bursitis) threw between 25-30 pitches during live batting practice. ... Yankees 1B Greg Bird was out of the lineup because of a sore foot. ... With a first-inning single, Yankees DH Matt Holliday collected his 2,000th career hit.