There was scant public jubilation when the communist country's capital was chosen in 2012 to host the showpiece competition and public sentiment has since hardened against the move, with even tightly-controlled state-run newspapers criticising the move in recent weeks.
Vietnam will "quickly work with the Olympic Committee of Asia (OCA) for measures to pull out from the hosting of Asiad 18 in Hanoi", said a statement posted on the government's website after Prime Minister Nguyen Tan Dung convened a meeting to examine the issue.
A lack of experience in hosting large sporting events and "difficulties in the country's socio-economic situation" were cited as reasons.
"Vietnam will apply for the hosting of Asiad at an appropriate time," the statement said.
Hanoi in November 2012 won the right to host the 2019 Asian Games, beating Indonesia's second city Surabaya after Dubai in the United Arab Emirates pulled out just before the vote.
The victory sparked celebrations among Vietnamese officials, who said the decision would unlock millions of dollars in infrastructure investment.
But with its economy in crisis, a decaying sports infrastructure and an uninspiring athletics record, public sentiment never swung behind the official campaign.
Officials said hosting the 18th Asian Games could cost some US$150 million (4.87 billion baht) and would require the construction of new sporting stadiums and an athletes' village.
But experts have said the real cost of the games, which were scheduled to run for 16 days, could have been far higher.
Vietnam's Olympic Committee initially proposed a budget of around $300 million which the government slashed due to the country's dire economic predicament.
Vietnam has been hit by falling growth as the effects of corruption and inefficiencies in the state-run sector ripple out into the wider economy.
A lingering banking crisis also threatens some of the country's key financial institutions.
In recent weeks, state-run newspapers and popular blogs have run stories and comment pieces urging the government to withdraw from the Games and spend the money on other priorities.
This year's Asian Games will be held in the South Korean city of Incheon in September-October.