We chose Angel's Cafe B & B mainly for its relative proximity to the Amtrak station, and what a delight it was! This historic building (1811, I think), served as the home of the Governor of New York in the early 19th century, in a neighborhood which later became the "Little Italy" area of the city. There are still some remnants of that that history in the form of Italian restaurants (we ate at Lombardo's--established 1919), just up the street. The neighborhood was then decimated in the 60's by the building of a huge interstate system and bridge, leaving this historic gem to decay.
Angela bought it in this form--windows broken, basically gutted--and renovated it as a really comfortable and attractive B and B, using the former hardware/candy store on the first floor as a cafe for the excellent breakfast--we got to choose our menu and time. It is beautifully furnished and features really wonderful and interesting art.
It is an easy walk to the riverfront, in one direction, and the excellent NY state museum in the other. I would note, though, that I would have been a little uncomfortable arriving late at night, if I didn't know where I was going. From Amtrak, using the 214 bus worked out great--about 8-10 minutes from the front of the station, and on the way back, we walked across the highway bridge, which was actually quite pleasant, but there is a confusing maze of parking lots and highway flyovers on the Albany side of the bridge.