The strong typhoon made landfall near Aki, Kochi prefecture, on Shikoku island at around 6 am (2100 GMT on Saturday), an official at the Japan Meteorological Agency said.
Late Saturday, the agency issued its highest warning for Mie prefecture, some 300 kilometres (190 miles) west of Tokyo, as the outer bands of the storm were already lashing the region and other areas of southwestern Japan, the agency said.
The warning means that the storm poses a threat to life and could inflict massive damage, the meteorological agency said.
The agency said the levels of torrential rain monitored in Mie were already "unprecedented," which could trigger massive landslides or cause major floods.
Television footage showed high waves triggered by the typhoon splashing over breakwaters and muddy torrents roaring down a swollen river.
Storms and torrential rain earlier this week have left one dead and 25 injured, Japan's public broadcaster NHK said.
Local authorities mainly in western Japan issued evacuation advisories to nearly 1.5 million people in total, NHK said.
At least 143 flights will be cancelled on Sunday due to the typhoon, which came as Japan had just begun its annual "Obon" summer holiday, NHK said. On Saturday some 470 flights were grounded.
Typhoon Halong, packing winds of up to 180 kilometres per hour, was moving northeast at 20 kilometres per hour, the agency said.
Halong comes a month after Typhoon Neoguri killed several people and left a trail of destruction in southern Japan.
Last weekend, a man drowned in a raging river while more than half a million people were advised to evacuate as heavy rain from Typhoon Nakri lashed the country.