He completed the construction of the Castle of Berkeley. He suffered during the wars between King Stephen and Empress Maud, at the hands of Walter, son of Miles, Earl of Hereford. In 1152 he was deprived of the Manor of Berkeley for refusing the recognise the authority of either Stephen or Maud, although he was soon after restored to the Honour of Dursley.