Japan gets mixed marks in terms of accessibility, or what is called bariafurī (barrier free; バリアフリー) in Japanese. You'll find most service staff will go out of their way to be helpful, even if they don't speak much English. In cities, train stations usually have lifts and station staff will help you on and off the train with a temporary slope. Rural stations are harder to navigate.
Across the board, newer buildings are likely to have access ramps and wheelchair-accessible toilets. Major sights are often accessible, even if not obviously so: shrines and temples, for example, often have back entrances with ramps. That said, what is considered 'accessible' at many sights might still mean steep slopes or long gravel paths.
A fair number of hotels, from the higher end of midrange and above, offer a barrier-free room or two (book well in advance); note that what constitutes barrier free is not always consistent, so check the details carefully. Should you decide upon arrival that a wheelchair (車いす; kuruma isu) would be helpful, hotel staff can help you rent one.
Some downsides: many neighbourhoods in Japanese cities lack pavements, and restaurants are often too cramped to accommodate diners in wheelchairs. Look for shōtengai (商店街; market streets), which are often pedestrian-only, covered arcades; most cities have them.
Accessible Japan (www.accessible-japan.com) is the best resource; they also produce an ebook with lots of detail.
Download Lonely Planet's free Accessible Travel guide from http://lptravel.to/AccessibleTravel.