Its pretty easy, there is a bolt that you losen, and then you can pop it right off. You will have to apply a good amount of pressure when pulling it out.
I had this same issue a few weeks ago, but after changing the bulbs the problem did not go away. I ended up taking it to the audi dealer and they said that the connector between the light and the car had some corrosion, so they cleaned it up and put it back and it worked without having to change any bulbs. The wierd thing is that it looked like only one bulb was out, but it ended up just being corrosion.