The result means that Froome, Porte and Yates were all able to take a chunk of time back after each losing 51 seconds in the dramatic end to stage one on Saturday.
The Slovak cyclist has a six-second lead ahead of Gaviria and is another four seconds ahead of Colbrelli in the general classification.
Overall, Froome is in 18th place, 55 seconds off the pace and still 44 behind 2017 Giro d'Italia victor Dutchman Tom Dumoulin, who with Team Sunweb only lost seven seconds to the Briton on Monday.
Sagan would have been happy to hold on to it, but the yellow jersey isn't his priority for the Tour, and with the top 59 riders overall separated by just 16 seconds, it was going to take something special from Bora-Hansgrohe to keep him on top.
Froome's misfortune is a boost for France's Romain Bardet, seen along with Quintana as the four-time champion's main rival in this year's race, and also for Dutchman Tom Dumoulin, the 2017 Giro d'Italia victor.
It is a measure of just how much success Team Sky have enjoyed at recent Tours de France that when Geraint Thomas missed out on the yellow jersey yesterday by only three seconds he barely shrugged.
"It was a huge challenge for the team, it's been incredible for us", said the 33-year-old Briton who was jeered when his name was announced on the starting ramp.
The stage victory saw BMC Racing move their team leader Richie Porte further up the standings, having lost time in a crash on stage one.
"We rode a decent time trial and even if we still lost a lot of time I'm relieved", said Bardet.
Froome avoided a repeat of his first stage tumble, the Team Sky leader staying out of trouble as he goes in pursuit of a record-equalling fifth Tour triumph.
And though Sky missed out on victory - and a chance to put Geraint Thomas in the yellow jersey - Froome was able to eradicate the deficit to numerous other contenders. It would have been nice not to have thrown 51 seconds away on the first day.
Movistar lost 53 seconds, which leaves Colombian Nairo Quintana, twice runner-up, 1:13 behind Froome - a massive amount of time after only three days of racing.
"Fernando won yesterday and now I'm the one with the yellow jersey, it's been exciting".
The Tour resumes tomorrow with the 195 km flat stage four that runs between La Baule and Sarzeau.
Boris Johnson's resignation a selfish act, says Guto Bebb
May was asked by an opposition lawmaker Monday whether she would contest a vote of confidence if one came rather than resign. Top Brexiteer Michael Gove, an ally of Mr Davis , Mr Johnson and also Mrs May, said everybody was "in very good spirits".
Trump administration freezes risk adjustment payments
Eventually, experts say, some insurers could simply get fed up and exit ObamaCare markets, leading to fewer choices for consumers. The agency says the court ruling also prevents it from collecting additional risk-adjustment funds until the issue is resolved.
Princess Charlotte waves at christening
The Duke and Duchess of Cambridge arrived at St James's Palace cradling their 11-week-old son ahead of his baptism. Kate also remained loyal to her classic look , wearing an elegant McQueen dress once again for the occasion.
Rockets reportedly 'determined' to sign Carmelo Anthony
The 31-year-old averaged 7.5 points, 3.0 rebounds and 1.2 steals in 25.6 minutes per game across 61 games (15 starts) in Houston . The moves add to uncertainty for the Rockets around Swiss free agent center Clint Capela , who has not agreed to a new deal.