The public transport in Malaysia is absolutely awful. Read online that the K2 bus comes every 20 minutes. Got to the bus station at about 10:30 and the bus didn't show up until 12:00, and then didn't leave until 12:15. The last bus back was also not at 18:00 like we'd read, but 15:20 (verified by the bus driver on our way there), leaving us with very little time to actually get to the fairy caves. RM4.50 each way per person. I believe a return ticket is cheaper but we didn't get one in case we missed it.
Arrived at about 13:30, wandered around trying to find a taxi (the first time since we started travelling that there hasn't been a taxi driver waiting at a bus stop). We were pointed in the right direction of where the taxis usually wait and managed to find one. He charged RM20 each way and waited at the fairy caves for us.
The caves themselves were amazing - absolutely massive and could hear the sound of bats and other animals much higher up. Didn't personally see any (saw some shapes on the rocks but that may have been little stalactites) so if anyone is worried about that, you shouldn't be.
There were a lot more stairs than I'd anticipated but it is worth it.
There were 2 of us and it cost us RM58 instead of RM100 if we'd gotten a grab. But it did take roughly 3 hours extra of waiting around in the heat so it's up to you which is worth it. My advice would be to go early - 08:00 or so. Try and find a group and split a grab - will save you several hours and a lot of stress.