"We have gained a great chance by hosting the Tokyo Olympic and Paralympic Games," said Mr Abe, adding the government will aim for the 20 million mark for 2020.
Mr Abe asked cabinet members to review what restrictions and obstacles exist for foreign travelers in order to make Japan a more tourism-oriented country.
The Japan National Tourism Organisation said the same day that the number of foreign visitor arrivals in Japan rose to a record 10.36 million last year, up 24% from the year before, surpassing the previous record of 8.61 million in 2010.
The yen's recent depreciation against other currencies and the easing of visa requirements for visitors from Southeast Asian countries from July contributed to the increase, according to the organisation.
It said the number of arrivals from Taiwan, Hong Kong, Thailand, Singapore, Malaysia, Indonesia, Vietnam, India, Australia and France hit record highs.