getting into bolivia but not through Puno?

Yes it is , although transport is much less frequent this way you can do it and it wold be an interesting way to go. Your starting point is Juliaca

I remember making enquiries re this a few years ago, there is transport that does the route but not much. If you decide to go for it check with immigration offices in Puno / La Paz first as Im not sure you can get entry/exit stamps at the border; would be a bugger if you got all the way there and had to return for bureaucratic reasons! Should be able to get the necessary in either of these towns in advance without problems.