Ahmed Kathrada, who was jailed alongside Nelson Mandela, said that Mr Zuma’s resignation would end a “crisis of confidence” in the government.
Mr Zuma said on Friday he would repay the money but would remain in his post.
He has faced widespread calls for his resignation from South Africa’s opposition.
South Africa’s highest court ruled that Mr Zuma had failed to uphold the constitution by ignoring instructions to repay some of the $16m in state funds spent on renovations at his private residence.
behind its leader, but the intervention of 86-year-old Mr Kathrada, a party veteran, may carry weight.
Along with Mr Mandela, Mr Kathrada was among eight ANC activists sentenced to life imprisonment after being convicted of trying to overthrow the apartheid government during the 1963-1964 Rivonia Trial.
In an open letter published in South African media, Mr Kathrada said: “In the face of such persistently widespread criticism, condemnation and demand, is it asking too much to express the hope that you will choose the correct way that is gaining momentum, to consider stepping down?
“I know that if I were in the president’s shoes, I would step down with immediate effect.”
The scandal is arguably the biggest yet to hit Mr Zuma, who has fended off accusations of corruption and cronyism since before he took office in 2009.
The public protector, an anti-corruption body, ruled in 2014 that $23m of public money had been improperly spent on Mr Zuma’s rural home in Nkandla in KwaZulu-Natal province, and ordered him to reimburse part of the expense.
South African media reacts – BBC Monitoring
South African papers are abuzz with reactions to President Jacob Zuma’s apology for improper spending on his rural home, and speculation about what he will do next.
The Johannesburg Star‘s Janet Smith says Mr Zuma must be observing events in Brazil , where there are growing calls for the resignation of President Dilma Rousseff over corruption charges.
The Sowetan‘s report about Mr Zuma’s apology is overshadowed by the ongoing saga involving the president and the Indian-born Gupta family, which has allegedly been allowed undue influence. The paper’s front-page comment yesterday was headlined “It’s official: he’s a failed president”
In the Mail and Guardian, Carien du Plessis foresees difficulties for the ruling ANC if Mr Zuma were to resign, saying: “In a party split down the middle in more or less equal factions… things are more likely to fall apart.”
Business Day‘s reports about the apology includes opposition Democratic Alliance leader Mmusi Maimane’s description of it as an insult, and ANC secretary general Gwede Mantashe’s efforts to discourage calls for the president to step down.
The president claimed he had acted “in good faith” and “never knowingly and deliberately set out to violate the constitution”.
“Any action that has been found not to be in keeping with the constitution happened because of a different approach and different legal advice,” he said, before issuing an apology for the “frustration” caused.
The case had been brought by two opposition parties, the Economic Freedom Fighters (EFF) and the Democratic Alliance (DA).
They have launched impeachment proceedings against Mr Zuma, but these are unlikely to be successful because of the ANC’s large majority in parliament.