By all means stay overnight (at least!!) in Toledo. There is a certain magic in that place that one feels the most at night time. During the day there are far too many people to enjoy that fascinating city.

We had a car so we stayed at the Parador. The view of Toledo from our room was just great.

Regards, Susana Sao Paulo, Brasil