As a road cyclist, I've ridden in a wide variety of conditions over the years: hot, humid weather, below freezing, during rain, and while snowing.
I have spent time riding on endless flat roads, up and down rolling hills, and climbing up and then descending mountains on winding roads. The rides get interesting when the conditions start to combine, and especially when you are not expecting it.
One 85 mile ride in rural Virginia comes to mind when two other riders and I got caught in heat, then heavy rain, cold, then sleet--all while riding on rolling terrain while trying to get ahead of a thunderstorm that we could see coming up behind us. The storm eventually caught up with us and we stopped for a few minutes to get out of the rain. But, with the temperature dropping it was easier to stay warm by continuing to pedal so we got back on our bikes to finish the ride.
I find windy conditions in particular to be the most difficult, psychologically, to work through on a ride.
A strong head wind can be demoralizing and a wind that shifts from side to front and then back again isn't any easier. Wind is tough because it can make you feel like you are going backwards at times. When coming at an angle or from the side, it can alter your balance on the bike and it feels like there is nowhere to hide from it.
Riders handle the wind in one of two ways. They either power through it or settle into a manageable pace--knowing that they will pay for the effort later with their legs. There are certainly pros and cons with both approaches and a number of variables to consider.
How a rider chooses to handle the wind depends on the fitness of the cyclist, the specific conditions on the road, and ultimately what you are trying to achieve on that ride (race, training, touring).
The larger question, however, is do you want to run the risk of running your resources [legs] into the ground or use them more effectively to achieve long-term gain?