Suggest to hire a car for approx 3 days to explore the island on wheels. A visit to the famous Skyrian horses in Kalamitsa, a dive in the past to the Palamari archaeological site, explore the secluded beaches of Pefkos, Agios Fokas and Agios Petros (all have fabulouse taverns too) ,
You dont need a car to visit the Falteits museum of Folklore , the castle, the monastery, the old town.
Also suggest to make a small cruise to Sarakino island with lots of swimming and lunch on the island.