I haven't been to Normandy, but I have spent time in Brittany. It's lovely and very easy driving. Quimper is nice, as is Dinan, Locronan, Cape Frehel, Concarneau. But my favorite is Carnac, with it's prehistoric megaliths and standing stones. There are some that are fenced off, and you have to look from a distance. but there are many in fields that are accessible. Also tombs on private property that you can go in. Lots to see and most incredible.
I wouldn't worry about driving in Brittany and I suspect that Normandy is much the same. Good roads, not a lot of traffic and easy to understand signs. I speak no French (except more wine please) and we had no trouble at all.
Callie in New Orleans