In 1109, a new diocese was created in England, at Ely, and Hervey was appointed to the bishopric created. While bishop, Hervey ordered the compilation of a house chronicle, which later became the Liber Eliensis. He supervised the construction of a causeway between Ely and Exning, which allowed easier access to Ely.
Hervey was a native of Brittany, and some sources state a chaplain of King William II of England, while others are less certain that he was a chaplain for the king. He was appointed Bishop of Bangor in 1092 by King William. Bangor at the time was in the Kingdom of Gwynedd, which had recently been overrun by the Normans, and following the killing of Robert of Rhuddlan had been taken over by Hugh d'Avranches, 1st Earl of Chester. The appointment of Hervey was probably intended to further consolidate the Norman hold on the area. Bangor was under the jurisdiction of the diocese of Canterbury, but Hervey was consecrated by Thomas of Bayeux, who was the Archbishop of York, since the see of Canterbury was vacant at the time.
Relations between Hervey and the Welsh appear to have been very bad. The Liber Eliensis described the situation as follows:
Since they did not show the respect and reverence due to a bishop, he wielded the sharp two-edged sword to subdue them, constraining them both with repeated excommunications and with the host of his kinsmen and other followers. They resisted him nonetheless and pressed him with such dangers that they killed his brother and intended to deal with him the same way, if they could lay hands on him.
Hervey was forced to rely on his own armed bands for protection. In 1094 a Welsh revolt against Norman rule in Gwynedd began under the leadership of Gruffudd ap Cynan, and by the late 1090s Hervey had been driven from his diocese by the Welsh. William of Malmesbury, however, states that the reason Hervey left Bangor was that the revenues of the see were too low. He remained nominally Bishop of Bangor until 1109. King Henry I of England tried to translate Hervey to the see of Lisieux in 1106, but the attempt was unsuccessful. The main opposition came from Archbishop Anselm of Canterbury, who was the metropolitan of Bangor, and refused to allow Hervey to go to a Norman bishopric. Anselm had the ability to decide the issue as Pope Paschal II had turned the decision of whether to translate Hervey to another see over to Anselm in 1102. While in exile, Hervey served as King Henry's confessor. Bangor itself remained vacant until 1120, when David the Scot was appointed.