Someplace that is not crazy with day trippers and is quiet with all the things you imagine when you think of the Greek Islands.
That's how people imagine the Greek islands these days? LOL
Andros will be the least touristy of the 4 you mentioned, and is quite a gem. Corfu and Milos will be the most touristy I'd say, in proportion to their sizes. Though Naxos is up there too. Corfu is big enough that there should be some quiet corners left. Another island to consider, if you find a house rental, is Kythira. Also Kefalonia, whose tourism industry isn't as huge as its sister islands Corfu and Zakynthos. You should also consider the mainland; the mainland has a coast too, you know. A very, very long and scenic one. The Peloponnese region has a burgeoning tourism industry where you should be able to find rentals; consider flying to Kalamata, and exploring the southern part of the region. Perhaps also in Epirus region, on the region's beautiful coast. The lovely town of Parga is touristy but maybe outside of town. You can also side-trip to Corfu from here if you want, as well as the region's stunning interior (like the Zagoria national park and villages).