The New South Wales Trade Minister Stuart Ayres has rejected calls to stand aside over his involvement in the recruitment process that appointed John Barilaro to a plum New York trade post.