A government official said the situation was tense, but denied any deaths.
Myanmar, a predominantly Buddhist country of 60 million people, has been grappling with sectarian violence for nearly two years. More than 240 people have been killed and another 250,000, mostly Muslims, forced to flee their homes.
Chris Lewa of the Arakan Project, an advocacy group that has been documenting abuses against members of the Rohinyga Muslim minority for more than a decade, said details about the violence that occurred early Tuesday morning in northern Rakhine state were still emerging, with many conflicting reports.
It is one of the most isolated regions in the country, both politically and geographically, and access to foreigners is denied or severely restricted. The death toll could range anywhere between 10 and 60, said Lewa, whose sources range from a village administrator to witnesses.
Tensions have been building in the region since last month, when monks from a Buddhist extremist movement known as 969 toured the area and gave sermons by loudspeaker advocating the expulsion of all Rohingya, who make up 90% of the population in northern Rakhine. It is the only place where the religious minority is in the majority.
A resident who spoke on condition of anonymity because he feared reprisals said an initial flare-up followed the discovery of three bodies in a ditch near Du Char Yar Tan village by several firewood collectors.
Believing they were among a group of eight Rohingya who went missing after being detained by authorities days earlier, they alerted friends and neighbors who returned with their cellphones to take pictures, said the man, who works as a volunteer English teacher.
That night, five police went to the village to confiscate the phones and check family lists, but the crowd turned on the officers, beating and chasing them off, he said. The police returned at 2am, saying one of their men had gone missing, he said.
That triggered a security crackdown.
Soldiers and police surrounded the village, breaking down doors and looting livestock and other valuables, the English teacher said. Almost all the men fled, leaving the women, children and elderly behind, he said.
Lewa said her sources reported that Rohingya women and children had been hacked to death, but the numbers varied widely. Some put the toll as low as five or 10, but one source who works for the administration in Maungdaw town, said it is widely believed 40 died, mostly women and children.
That some of the victims appeared to have been stabbed with knives, not shot or beaten, "would clearly indicate the massacre was committed by (Buddhist) Rakhine villagers, rather than the police or army," the Arakan Project wrote in a briefing Thursday.
The English teacher, who spoke by telephone, said 17 women and five children were killed.
Rakhine state spokesman Win Myaing said police had surrounded the village because they were looking for the policeman who went missing, but that he was not aware that anyone had been killed.
Khin Maung Than, a Muslim who lives in a neighbouring village, said he visited Du Char Yar Tan and had seen no evidence of violence or deaths there.
Tensions have been reported for days, but getting information is difficult. Northern Rakhine - home to 80% of the country's 1 million Rohingya - runs along the Bay of Bengal and is cut off from the rest of the country by a mountain range.
Some of the people there descend from families that have been there generations. Others arrived more recently from neighbouring Bangladesh. All have been denied citizenship, rendering them stateless.
For decades, they have been unable to travel freely, practice their religion, or work as teachers or doctors. They need special approval to marry and are the only people in the country barred from having more than two children.