June is usually a very good moment to visit the Basque country. Days are long, usually warm (rainy cold days can happen anytime, though), and there are many celebrations and festivities in towns and smaller villages. About half an hour from Bilbo you can go to Gernika, this year commemorating the 80th anniversary of the bombing by the Nazi Condor-Legion, that inspired PIcasso for his famous painting. The museum of Peace is very interesting. A very short distance from Gernika you have the Biosphere reserve of Urdaibai, with beautiful beaches, the "painted forest", and interesting towns like Mundaka or Elantxobe. From there a curvy road goes all the way to the beautiful fishing town Lekeitio (great for walking, for foodies or for its beach). On your way to Donostia (S Sebastian), you could stop in Ondarroa, Mutriku, Zumaia, Getaria (I specially like this village, its white wine "txakolina", the museum dedicated to the fashion designer Balenciaga...), and Zarautz. Between Donostia and Baiona (Bayonne), Pasai Donibane and the road uphill to Hondarribia (instead of the motorway) has awesome views, too. And once Lapurdi (we call it Iparralde, the North, meaning the French part), a similar beautiful narrow road goes between Hendaia/Hendaye and Ziburu/Ciboure and Donibane Lohizune/St Jean de Luz. Inland, you can go to Sare/Sara, visit the Caves, or walk the so-called "smugglers way", a short walk now prepared for tourists that crosses the border to Zugarramurdi, a tiny village in Nafarroa/Navarre, with a long history in the Basque mitology and witchcraft.
Other interesting towns and villages inland: Donibane Garazi (St Jean-pie-de-Port), Elorrio and/or Oñati and the mountains above both, Azpeitia, Lesaka...