In 1561 when the Protestants conquered the city, they took hold of the Church and used it as a Temple for a few years. In 1563 the Peace of Amboise returned it to Catholic worship.
Catherine Di Medici and Charles IX were received there in 1564.
In 1567 the Huguenots pulled down the spire and parts of the vaults, and turned the Church into a small fort. During the siege the bell tower was used as a observation point.
One can see, on the north and west sides of the tower that support the bell tower (rebuilt identically in 1740), the trace of the balls fired by the army of Louis XIII during the siege of 1621.
After the Moustier was destroyed, The Church of Saint James became the Cathedral from 1630 to 1739.