You might want to look up on the rather extensive literature on robust statistics. To do this in a mathematically correct way, the experts in this area have typically used mixture distributions to model outliers – ie p(\mathcal{N},Var) + (1-p)(\mathcal{N}, 10 \times Var) as an example. Then you could run your outlier tests on this synthetic data set where you are certain of the parameter values. John Tukey, Frank Hampel and Peter Huber have done the foundational work in this area.
The other scenario is to use inference techniques that are not sensitive to outliers. Classical nonparametrics would fall into this category.
A Bayesian method would specify a prior probability on the the possible distributions outliers could come from.
Related thread: