A statement said Bishop Richard Williamson must "unequivocally" distance himself from his statements to serve in the Roman Catholic Church.
The Vatican also said that the Pope had not been aware of the bishop's views when he lifted excommunications on him and three other bishops last month.
Earlier, a senior cardinal acknowledged the Vatican had mishandled the issue.
The Pope's decision, ending Bishop Williamson's excommunication on an unrelated matter, has caused a bitter row, as the bishop does not believe that Jews were gassed by the Nazis in World War II.
"Bishop Williamson, in order to be admitted to the Episcopal functions of the Church, must in an absolutely unequivocal and public way distance himself from his positions regarding the Shoah [Holocaust]," the Vatican statement said.
It said Bishop Williamson's positions on the Holocaust were "absolutely unacceptable and firmly rejected by the Holy Father".
On Tuesday, German Chancellor Angela Merkel urged the Pope to make a clearer rejection of Holocaust denials.
"This is not just a matter, in my opinion, for the Christian, Catholic and Jewish communities in Germany but the Pope and the Vatican should clarify unambiguously that there can be no denial," she said.
Cardinal Walter Kasper, who is in charge of relations between the Roman Catholic Church and Jewish leaders, admitted different parts of the Vatican administration had not talked enough to each other, and failed to check where problems could arise.
Bishop Williamson - who was ex-communicated 20 years ago on an unrelated matter - has apologised for stirring controversy, but not repudiated his views.
Last November, the British-born bishop angered Jewish people across the world when he told Swedish TV: "I believe there were no gas chambers [during World War II]."
He said he believed that up to "300,000 Jews perished in Nazi concentration camps but none of them by gas chambers".
He is one of four bishops, who are members of the Society of Pius X, whose excommunication was lifted last month by the Pope.
The Vatican's statement on Wednesday also said that the society must recognise the reformist Vatican II Council of 1962-65 and the popes who followed it.