A well-written introduction and shop guide pertaining mostly to the Tokyo area:
Record Shopping in Japan
In Osaka, I like browsing the Gorilla Records shop in the basement of the building next to Tower Records Shinsaibashi. The “Real Hip-Hop Americans from Lagos” on the street corners of Amemura, however, are sometimes more than one can bear.
Forever Records, between Namba Parks and DenDen Town (down the street from an improbably-located 7-11), used to have a good selection, but I haven’t been there in eons… Pawning my 1200s when I was a student was a necessary evil, but it still hurts to think about.
By far, the best deals on vinyl I have ever found are in second-hand stores. I once found a rare Beatles pressing worth several hundred dollars in a pile marked on sale for 100 yen.