Trell Kimmons, Justin Gatlin and Ryan Bailey as well as Jeffery Demps and Darvis Patton, who ran a preliminary heat, will lose their medals after Gay was banned last year following a positive test for an anabolic steroid.
"As expected, following USADA's decision in the Tyson Gay case, the IOC today confirmed that the US team has been disqualified from the 4x100-metres race that was part of the athletics competition at the London 2012 Olympic Games," USOC spokesman Patrick Sandusky said in a statement.
"We will begin efforts to have the medals returned, and support all measures to protect clean athletes."
Twelve months ago, Gay received a one-year suspension by the US. Anti-Doping Agency (USADA) after testing positive for a banned anabolic steroid in 2013. All his results from July 2012 onwards were annulled and he returned his relay silver medal to the US Olympic Committee.
Athletics competitors must follow rules of the International Association of Athletics Federations in the Olympics but the International Olympics Committee has the final say on matters including medals. The IOC has varied in the past on stripping medals from relay squads.
All members of the US men's 4x400 meters relay at the 2000 Olympics lost their medals after doping cases involving Antonio Pettigrew and Jerome Young.
But the Olympic committee allowed U.. runners in the 2004 Olympic women's 4x400 meters relay final to keep their gold medals even though squad member Crystal Cox, who competed only in the preliminary round, was stripped of hers after admitting in 2010 she had used anabolic steroids.
The IOC had also stripped the US women's 4x100 and 4x400 relays of their medals at the Sydney Games after Marion Jones's doping admission but the Court of Arbitration for Sport reinstated medals for all but Jones after an appeal.
The IAAF rule in place in 2012, however, stated clearly that all relay members would lose their medals if there was a doping violation.
"Having the second place finish officially removed by the IOC is in accordance with the USADA ruling (on Tyson Gay) and has been expected since USADA issued their decision," said a USA Track & Field spokeswoman.