Joshua was one of the founders of the South Church, 1662. In 1687 he was, by commission from James II., chief justice of the court of common pleas for the province of Maine. He was the author of two tracts, "Old Men's Tears, " printed in 1691, and "Planting of the Massachusetts Colony, 1694." His house was in Sudbury street, and he died January 20, 1698, aged eighty-three.