The children’s father is believed to be a member of the far-right Reichsburger movement and has written a number of controversial books in which he denies the Holocaust happened.
Police have said their father, a 54-year-old Austrian, is a member of the far-right Reichsburger movement and is a known Holocaust denier.
Social services were called to the property in Obritz, near the Czech border, when locals became concerned for the children’s welfare.
But when they tried to get in, the father attacked them with pepper spray.
Police were called and the man was arrested. He has since been released on bail as an investigation is carried out.
The children – aged between seven months and five years – were taken with their mother to be checked out in hospital.
Police spokesman Stefan Loidl said they were “in a good health condition and were not neglected”.
The children are currently being looked after by social services.
Local authorities said they believed the family were living in the illegal hideout for several months but there had been complaints about them over the last few weeks.