The gunmen late Saturday barged into a resort on Mabul Island in the state of Sabah, about 1,880 kilometres east of Kuala Lumpur, and fired randomly into the restaurant and reception area, according to state police chief Jalaluddin Abdul Rahman.
Another police source said a 32-year-old Marine corporal was killed in the attackm while a 26-year-old constable remained missing and believed to have been taken hostage by the attackers.
"The gunmen could have been looking for possible kidnap victims but had been thwarted by the presence of the police officers," he said.
The attack occurred three days after suspected al-Qaeda-linked Filipino Abu Sayyaf rebels freed Yang Zai Lin, 34, a Chinese fish farm manager kidnapped May 6 in the town of Kunak, also in Sabah.
On May 30, suspected Abu Sayyaf rebels freed two women - a Chinese tourist and a Filipino worker - seized in April from a resort in Semporna town in Sabah.
Abu Sayyaf militants are believed to be still holding two Malaysian hostages, a fish farm caretaker and a labourer.