FA Cup holders Arsenal were dumped out of the competition in the fourth round after a 1-0 defeat at Southampton on Saturday.