The four-day visit by Zhang Zhijun, director of the Taiwan Affairs Office, is a further sign of warming ties between the former bitter rivals, despite vocal opposition from Taiwanese suspicious of closer ties with Beijing.
Zhang, who holds ministerial status, arrived at Taoyuan airport in the north of the island around noon and met Wang Yu-chi, chairman of Taiwan's Mainland Affairs Council, later in the day.
The pair had met in China's eastern city of Nanjing in February in the first government-to-government talks since Taiwan and the mainland split 65 years ago after a civil war.
Hailing the historic nature of his visit, Zhang said: "It took less than three hours to fly from Beijing to Taiwan, but the step took 65 years."
Ties between the former enemies have eased markedly since 2008 when Ma Ying-jeou of Taiwan's China-friendly Kuomintang came to power.
But many Taiwanese remain wary of closer relations. A planned pact to free up the services trade with China sparked an occupation of Taiwan's parliament and mass street protests in April.
At a press conference after Wednesday's talks, Wang said some "concrete progress" was made.
A proposal to set up liaison offices was floated last year. But the two sides have failed to reach agreement about giving representatives the right to visit nationals detained or jailed by the other side.
Wang indicated this issue would now be discussed.
"Regarding the issue of the two sides setting up liaison offices, today we formally agreed to list 'human rights visit' on the agenda of discussions in the future," he said.
Wang said the Chinese side also agreed to discuss a Taiwanese suggestion, that Chinese tourists be allowed to make transits in Taiwan while travelling to other destinations.
Ma Xiaoguang, spokesman for the Chinese delegation, said there was no discussion of a summit between Chinese President Xi Jinping and his Taiwanese counterpart Ma.
No official documents were signed at the meeting, but they were the highest-level talks between the two sides ever to take place in Taiwan.
Wang described the visit as "of significance". But he also called on Beijing to take into account the multiple views within Taiwan over its relationship with the mainland, urging Zhang to "listen carefully to the voices of Taiwan people, appreciate their lifestyle and respect their choices".
While Taiwan has a vibrant and growing business relationship with China, many of its people are concerned about the effect on their democracy should relations continue to warm with the one-party mainland.
- Angry protests -
Opponents in Taiwan have accused the government of trading Taiwan's national interests to Beijing in exchange for marginal economic benefits.
Demonstrators tried to break through security barriers outside the hotel where Wang and Zhang were to meet and clashed with riot police.
"We strongly oppose the Wang-Zhang meeting, which is illegal while the government has yet to come up with a law to supervise the contact," Lin Fei-fan, leader of an anti-Beijing protest group, said.
"Taiwan's future decided by Taiwan!" chanted dozens of demonstrators, mostly young students.
At the airport there were clashes between dozens of pro-independence and pro-unification activists.
The two sides split in 1949 after the Nationalists lost a civil war to the communists and fled to the island.
But Beijing still opposes the island taking part in international organisations as a sovereign state and considers it to be a part of China awaiting reunification -- by force if necessary.