The official currency in Taiwan is called New Taiwan Dollar (TWD or NT/NTD). It is the only legal currency to purchase in Taiwan, other currencies, like US dollars or Renminbi, will not be accepted except in some duty-free shops.
Many banks in Taiwan allow you to exchange major currencies into TWD, many hotels and department stores do as well, but in a less preferred rate. So you usually don't need to worry about how to exchange money. It is worth noting, however, not every bank in the world exchanges TWD. If you want to bring some TWD bills back home, make sure you can exchange them back into your currency first!
Though many major banks in Taiwan allow you to cash Traveler's Cheques, please understand that Taiwanese people don't use Traveler's Cheques. So it's unlikely that you can use Traveler's Cheques in Taiwan as you may expected. You are advised to use credit card instead.
Most major credit cards can be accepted in Taiwan. You are still expected to pay cash at family-owned restaurents, but restaurents above medium level, hotels and shopping centers definitely accepts credit card. In some rare cases, you will be asked to pay an extra "process fee" when using a credit card. This usually means that the shop you are shopping at is not a honest tax payer. Avoid buying things from them.
It shouldn't be hard to find an ATM in Taiwan. Many of the 7/11 stores have them inside. Almost all bank offices you see on the street has a separate ATM area which operates 24 hours a day. They are also available in many other places like department stores. Some (but not all) of the ATMs supports internation bank networks like Cirrus. So you can withdraw cash with them easily. Almost all ATMs in Taiwan can display information in English.
Foreigh visitors are not allowed to open a bank account in Taiwan, unless the visitor is legally working or studying there.
10NT and 50NT coins has several versions. Most of them are still in circulation, though some versions are quite rare. Please don't be surprised if you received a strange looking coin. (But if you have doubt, you can ask them to give you a "normal one" instead.) If you hold any old bills which are not in circulation now, you can consult the Bank of Taiwan and see if you can exchange them into new bills.