Naomasa's son Naokatsu was still an infant when Naomasa died. So Naomasa chief retainer consulted with Shogun Tokugawa Ieyasu and it was decided to build a new castle at the present location. Construction started in 1603.