Laugardalslaug is the heart of the city. It's where the locals go to relax and it's a wonderful facility.
Yes, the paint is flaking and yes, it doesn't look bright and shiny, but it has by far the best facilities in the city, and the prices are extremely reasonable.
It's essential that you follow the guidelines and shower properly before you go in. For people that prefer privacy there are individual cubicles available.
The best experience is had by alternating the hot tubs something like this: 42c, 5c, 42c, 5c, 44c, 5c, 43c salt. The 5c is amazing and well worth acclimatising to.
For those who are 'proper' swimmers, there are 50m outdoor and indoor pools and both are excellent. For families, the kids' areas and slides are great fun.
If you're looking for the full experience, after your visit turn right out of the door and walk 100m to the Pylsuvagninn (hot dog wagon) and have an Icelandic-style hot dog.