Why Use The Very Expensive 5w-40 Oil?

I own a 2000 A4 Avant 1.9tdi 115bhp. Unfortunately due to illness I can no longer afford to have my car dealer serviced and have decided to do all future work myself. I visited my local dealer to get all parts needed to carry out the service and asked why I have to use the very expensive 5W-40 oil and received a very lengthy answer which did not contain any mechanical reason and seem to be mainly based on Audi policy and guidelines.

Could anybody tell me if I need to use this oil and the reasons why? I am happy to carry on with using this oil if it is mechanically the best thing for me to do.