The Tokyo Metropolitan Police Department is planning to form a harbor squad to respond to emergencies during the 2020 Tokyo Olympic and Paralympic Games.
The Tokyo Bay area is expected to host the athletes' village, and be the site of many events.
The members of the Waterfront Response Team will be armed with submachine guns and use powered water bikes or rubber boats.
They will be tasked with thwarting terrorist attacks from the ocean and preventing suspects from escaping by sea.
Officials are also considering mobilizing the new team when foreign dignitaries visit Japan.
Tokyo police already have an emergency response unit armed with guns to deal with terrorist attacks on land.
Section chief Masaki Shigehisa said the mobile team on the water will help beef up security in the harbor area.