Japan trailed 12-0 at the half and 19-0 just a minute into the second half before Lote Tuqiri scored a try for Japan, and Katsuyuki Sakai converted as Japan made it 19-7.
Japan went down 25-19, 25-23, 25-16 to the world's No. 7 team at Tokyo Metropolitan Gymnasium, where the Serbs were led by 13 points from Nikic Milos and 10 each from Nikola Kovacevic and Marko Podrascanin.
Japanese No. 1 Jun Mizutani, ranked eighth in the world, lost both of his matches in straight sets -- to 10th-ranked Dimitrij Ovtcharov in the opening match and sixth-ranked Timo Boll in the fourth match.
There is no third-place contest between the losing teams of the semifinals.
It was the first time Japan lost a home World Cup qualifier since September 1997, when they were felled by South Korea on the road to France '98.