I would totally start here and then go up to the castle, as they will connect you with an app that has narration for the museum and also AR components for the castle. It also gives an overview of the castle and how it was made, the rationale behind the wall design, and other interesting insights but would be useful to have before going up to the castle itself. The rest of the museum is also lovely, with cultural, household, arts, crafts, religious, and historical contexts since the Jomon period. When I went today they also had a small art exhibit from their permanent collection, and a student art exhibit from area schools.