There are two main bus providers running through the park between Xincheng-Taroko station and Tianxiang (NT$84, 40 to 45 minutes):
Taroko Bus (302) departs from Xincheng-Taroko every 50 minutes from 7.10am to 11.20am and 1pm to 5.10pm. The last return from Tianxiang is at 6.50pm.
Taroko Tourist Shuttle Bus (1133A) starts from Hualien train station roughly hourly from 7am to 3.10pm, picks up passengers at Qixingtan then Xincheng-Taroko station (35 minutes after Hualien), then takes a further 45 minutes to reach Tianxiang. Last return is 5pm from Tianxiang.
Each service includes stops in both directions at the National Park HQ, Swallow Grotto approach and at Lushui, plus westbound-only at the Shakadang trailhead and eastbound-only at Changchun (the viewpoint for Eternal Spring Shrine). Only the Tourist Shuttle detours to Bulowan (westbound only).
In addition there are four further Hualien Bus services including the once daily Hualien-Lishan bus (1141). East of Tianxiang, these accept passengers using the NT$250 Taroko Tourist Shuttle day pass, but Taroko buses require separate payment.
If you want to see the gorge by scooter, renting from the square outside Xincheng train station makes more sense than from Hualien as the shop here is less fussy over foreign licenses, rates are good and Xincheng is far nearer the national park.