PM defends going to $10,000 dinner in Perth