The attacks appear to be an escalation of a tit-for-tat conflict that is one of the bloodiest examples of the complex political, social and economic tensions undermining Nigeria's nascent system of civilian rule.
"This is total ethno-religious killing and the state government is fully aware of what has happened," said Abdullahi Abdullahi, a community leader.
State authorities estimate 67 dead but local people said the true figure was much higher.
Mr Abdullahi said 650 people had died, although it was unclear whether that number included those who were missing. Lawal Waziri, a teacher, put the death toll at 550, adding that 250 were buried on Tuesday.
Mohammed Zaki, a transport worker, said the attackers went from house to house asking women to leave before killing men and all but the youngest boys.
Mr Abdullahi said security forces had left Yelwa before the attackers from the Tarok ethnic group and other peoples surrounded the town using weapons such as pistols and Kalashnikov rifles, some of which must have been obtained from the security forces.
Other local people said the attackers set light to three mosques and burned houses, adding that some of the assailants wore police and military uniforms. One man said he had seen paramilitary mobile police identity badges left behind by some of the attackers, although he did not have the evidence with him.
The outskirts of Yelwa, which is four hours' drive from Abuja, the capital, were filled with charred buildings and wrecked cars.
A truck containing members of Nigeria's paramilitary mobile police was parked almost opposite the Muktar Oil filling station, where petrol pumps had been ripped out and burned. One of the destroyed buildings belongs to the Brotherhood of the Cross and Star, a reminder of an attack on a Yelwa church in February in which 48 Christians were killed.
Local people said a main cause of the conflict was a dispute over ownership of the area's fertile farmland. The killings appear to be the latest in a series of reprisal attacks by Muslims and Christians in Plateau, a religiously mixed state with a Christian majority.
More than 1,000 people were killed in sectarian riots in Jos, the Plateau state capital, in September 2001.
Sunday Ehindero, deputy inspector-general of police, denied members of the security forces had been involved in the attacks and said he had no evidence that troops or police had withdrawn before the assault. He said the police had taken attackers into custody, but declined to give details.
Mr Ehindero said he doubted claims by some local people that the attackers had numbered 3,000 and had kidnapped community members. "In a situation like that some people run helter-skelter and some people may be missing," he said.
Umar Abdu Mairga, head of the Nigerian Red Cross team visiting Yelwa yesterday, said at least 500 people were believed killed.