The Paperback Exchange was on of my favorite places downtown in the 70s and 80s. It was not fancy, in fact it was a dump. My memory says that it was a little one story building on the East side of Assembly. The address was very easy to memorize, and is now occupied either by the former AT&T building or a the parking garage, I'm not sure which, so I'm including pictures of both.
The place had big glass windows on either side of a central door, and there was a wide wooden display shelf behind each window. I don't know what the building housed originally, but by the time The Paperback Exchange occupied it, this area was strewn with old magazines and comics yellowing in the afternoon sun.
The place was definitely a bit seedy, and porn was a good part of their stock in trade, along with men's "adventure" magazines like, um Argosy, Soldier of Fortune and the like. (Looking back, I'm a little surprised my mom would drop me off there sometimes while she shopped. Of course, she would have first look at whatever I bought...) There were never many customers when I was there, and I've wondered over the years if perhaps the place was a front of some sort though I never saw any indication of that at the time.
All that aside, what I went for was the Science Fiction rack (and later in the 80s, used comics). This was more or less in the center of the store and was, I believe, two double-sided wooden rack units. The books were in no particular order, but they did seem to turn over with fair regularity, and the place always seemed to have quite a few Ace Doubles. This was an interesting concept that Ace books pioneered in the 50s and 60s (though it lasted into the 70s) where the company would publish two books (often novellas by today's length standards) under the same cover, but upside-down to each other. Each book would have it's own front cover and there was (necessarily) no "back" cover. The books might be by the same author (Jack Vance: The Houses of Iszm with Jack Vance: The Son of the Tree for example) or different authors (Jaunita Coulson: The Singing Stones with E. C. Tubb Derai for another). Although I did not know this at the time, Donald A. Wollheim, who later founded DAW Books was the SF editor at Ace during a large part of this time, and since his tastes were often congruent with mine, I liked a lot of those old Ace Doubles. Anyway, I got a bit distracted there -- my point was going to be that at the time, at The Paperback Exchange, those doubles weren't yet collectible, they were just old and over the years I added many to my shelves.
The end came with development. As I said, I'm not sure exactly which of these two building occupies exactly the "1234" address, but between the two of them, they took out the entire contents of the first block of Assembly. I'm not saying The Paperback Exchange was any architectural treasure either, but in my opinion the AT&T building, at least, should not have been built there as it overshadows the Capitol... As far as I could tell, The Paperback Exchange never relocated to any other spot after it was evicted -- there were several other such operations in town at the time and perhaps they didn't have the margin to reestablish and compete. At any rate, I still have all those Ace Doubles.