anime - Apprentice magician kidnaps people to turn them into human-shaped building blocks

I have a memory of watching a movie on TV when I was a kid. I wasn't familiar with anime at the time, but looking back I believe it was anime. I would have watched it sometime between 1982 and 1986 on broadcast TV in Cleveland, OH.

The plot focused on a man who was an apprentice magician or sorcerer. His master made him kidnap people, who would then be transformed into roughly human-shaped building blocks that the sorcerer then used to expand his castle. The apprentice knew what he was doing was evil, but went along with it to protect his little sister. He bides his time, covertly working to ultimately defeat his master and save all the people who have been taken and have used as building material.

That's the gist of the movie as I remember it, but I'm aware that memory is imperfect. Hopefully it's enough to be identifiable.

• Next star trek - How does the baby changeling transfer itself into Odo?
• Previous If Galadriel had claimed the One Ring, would Sauron have become her servant, or would have Galadriel become his servant?