Hold Stirling and you control Scotland. This maxim has ensured that a fortress of some kind has existed here since prehistoric times. You cannot help drawing parallels with Edinburgh Castle, but many find Stirling's fortress more atmospheric – the location, architecture, historical significance and utterly commanding views combine to make it a grand and memorable sight. This means it draws plenty of visitors, so it's advisable to visit in the afternoon; many tourists come on day trips, so you may have the castle to yourself by about 4pm.

The current castle dates from the late 14th to the 16th century, when it was a residence of the Stuart monarchs. The undisputed highlight of a visit is the fabulous, recently restored Royal Palace. The idea was that it should look brand new, just as when it was constructed by French masons under the orders of James V in the mid-16th century with the aim of impressing his new (also French) bride and other crowned heads of Europe. The suite of six rooms – three for the king, three for the queen – is a sumptuous riot of colour. Particularly notable are the fine fireplaces, the re-created painted oak discs in the ceiling of the king's audience chamber, and the fabulous series of tap­estries that have been painstakingly woven over many years. Based on originals in New York's Metropolitan Museum, they depict the hunting of a unicorn – an event ripe with Christian metaphor – and are utterly beautiful. Don't miss the palace exterior, studded with beautiful sculptures, or the Stirling Heads Gallery above the royal chambers. This has the original oak roundels – a real rogue's gallery of royals, courtiers and classical personalities. In the vaults beneath the palace is a kid-friendly exhibition on various aspects of castle life.

The other buildings surrounding the main castle courtyard are the vast Great Hall, built by James IV; the Royal Chapel, remodelled in the early 17th century by James VI and with the colourful original mural painting intact; and the King's Old Building. This is now home to the Museum of the Argyll & Sutherland Highlanders (donations appreciated), which traces the history of this famous regiment from 1794, including its famous defensive action in the Battle of Balaclava in 1854. Make sure you read the moving letters from the World Wars.

Until the last tapestry is completed, probably in late 2013, you can watch the weavers at work in the Tapestry Studio at the far end of the castle. It's fascinating to see. Other displays include the Great Kitchens, bringing to life the bustle and scale of the enterprise of cooking for the king and, near the entrance, the Castle Exhibition, which gives good background information on the Stuart kings and updates on current archaeological investigations. The magnificent vistas from the ramparts are stirring.

Admission includes an audioguide, and free guided tours leave regularly from near the entrance. Tours (£2 extra, free for HS members) also run to Argyll's Lodging, at the top of Castle Wynd. Complete with turrets, this spectacular lodge is Scotland's most impressive 17th-century town house. It's the former home of William Alexander, Earl of Stirling and noted literary figure. It has been tastefully restored and gives an insight into lavish, 17th-century aristocratic life. There are four or five tours daily (you can't enter by other means).