Most sights in Takayama can be covered easily on foot. You can amble from the train station across to Teramachi in 25 minutes.
The only place you may really need to take the bus is to Hida-no-Sato (¥200, 10 minutes, half-hourly).
Takayama is bicycle-friendly. Some lodgings hire out or lend cycles, or you can hire one from the convenience store Timely (34-1183; per hr/day ¥300/1200; 10am-5pm) next to the train station; or Hara Cycle (32-1657; per hr/day ¥300/1300) on Kokubun-ji-dōri.