There are some major sights in Chelsea, including the Chelsea Physic Garden, The Chelsea Pensioner's Hospital (with famous park, architecture, stables, chapel, etc.) and the Tate Britain. While you may be missing the big shows, as Mark said, you can always check for antique shows in Chelsea as there are many. A walk along the Chelsea Embankment on a nice day is lovely.

The King's Road is a lot of fun for people watching as well as shopping. Fulham Road and Old Brompton Rd, both nearby, have lots of eating options.

Definitely checkout some guidebooks that divide London attractions by neighborhood. City Secrets London does a nice job with Chelsea and environs.

Best of Luck, Debbie in Pittsburgh, anxiously awaiting the right winter airfare sale!