Thousands of people lined the streets to catch a glimpse of the hearse as it carried his coffin from the military academy where it laid in state for 10 days.
Many of his supporters were wearing red, the colour of Mr Chavez's political movement.
Mr Chavez, who led Venezuela for 14 years, died of cancer last week.
His coffin was received by a military guard of honour.
Religious and political ceremonies were held at the military museum, attended by Mr Chavez's chosen successor Nicolas Maduro.
It is not yet clear what will happen to Mr Chavez's body in the longer term.
Mr Maduro asked the National Assembly to reform the constitution to allow Mr Chavez's body to be buried in the National Pantheon, together with the most important leaders in Venezuela's history.
Mr Chavez, for his part, had said he wanted to be buried in his hometown in Barinas.
After Friday's ceremonies, the country's Information Minister, Ernesto Villegas, said the government had dropped plans to embalm Mr Chavez for permanent display.
He said the decision was made at the advice of Russian experts who said Mr Chavez's body had not been properly prepared. The embalming process would take seven to eight months.
Earlier in the day, political and military authorities joined Mr Chavez's relatives for a ceremony at the military academy where his remains lay in state for 10 days.
"Thanks, comandante, for giving us back our fatherland," said one of Mr Chavez's daughters, Maria Gabriela, in an emotional eulogy.
"You have left us unexpectedly and have left an enormous vacuum in Venezuela," said one of Mr Chavez's former teachers at the military academy, Major General Jacinto Perez Arcay.
Tens of thousands of Venezuelans have visited the coffin of their former leader.
Shortly after his death was announced on 5 March, the government declared seven days of mourning, which was later extended to 10 days.
Source: BBC News