Transport Minister Mohamed Ibrahim Habsade told the BBC that militiamen would eject government members by force if they did not withdraw peacefully.
Mr Habsade's militia have clashed with government troops in Baidoa recently.
Correspondents say the move is another setback for the government, which has already lost ground to Islamist forces.
"The government should vacate Baidoa peacefully. Or else we will eject it by force," Mr Habsade told the BBC.
Somalia's interim government was formed two years ago in an attempt to bring unity to a country which had been divided among clan-based militias since the fall of dictator Siad Barre in 1991.
Militia leaders have been given posts in government but some, like Mr Habsade, retain control over their own forces.
In another development, the two top officials of the Union of Islamic Courts (UIC) - the interim government's main rival for control of Somalia - flew to Libya where an African Union meeting is due to take place.
There is speculation that Libya's government may have invited Sheikh Hassan Dahir Aweys and Sheikh Sharif Sheikh Ahmed to Tripoli to try to arrange talks between them and the Somali government.
A long-delayed second round of peace talks began in Khartoum at the weekend between the government and the UIC, which controls a large part of southern Somalia including the capital, Mogadishu.
The transitional government controls only a small area around Baidoa.