I've read that the contacts inside the cruise control switch on that stick sometimes can go bad / get finnickey ... this sounds like what is happening to you.
I don't have a link handy, but search through audiworld.com C5 forums - someone replaced theirs DIY and it wasn't too spendy
