Pearl Pass is only open a few weeks a year. While not hard. It does have some ledge roads that will freak out many flat landers. Coming from the Aspen side, the last portion getting to Crusty Butt as the locals call it, is 20+ miles of road filled with basketball sized rocks. It gets a bit old, but it is pretty. Coming up to the top is like a different world. It is nothing but great bulwarks of scree covered with lichens. It is like you are on a different planet.
Personally, I love it.
If you go over, stop at the partially reconstructed ghost town, Ashcroft. If you hit Aspen, stop at the grocery store on the east side of town, they have an awesome deli and it is a great place to stock up.
A side trip to Maroon Bells is not to be missed. One of the most beautiful sights in the whole world.
A good return trail is Taylor Pass. While not horribly rugged, expect some Colorado Pinstriping from brushing up against the willows on the Taylor Park side.