Thanks for the reply! I am aware of some robust statistical techniques which as you say will often be able to deal with influential outliers (those which specifically affect your model fit, standardized betas etc).
I have wanted to look at some semi parametric techniques but yes non parametric techniques remain valuable.
I know too little of priors to make use of Bayesian at this stage.
I think in general my lack of confidence is situated in the actual sample responses. I want to know how good those are. I can fit a model and see if I get results but that doesn’t answer my question regarding quality of sample. If they adhere to most assumptions required for parametric testing then it is just that - I am able to run certain techniques and yes in some cases prior literature or work should tell me what sort of relationships etc I should likely see.
However, I want to create scenarios where that sample is specifically constrained with false observations to see how well existing techniques can differentiate between actual sample and false sample. I understand that creating observations that are randomly generated values from the possible min and max per variable will lead to observations that could have been possible (i.e. they won’t be picked up by Mahalanobis or say robust Mahalanobis in the case where we know the suspected underlying distribution is not normal) so I do want some idea of proportions. Thinking about this as well I do wonder how much this “experiment” will end up testing the outlier technique more than actually “verifying” the sample. This is making me wonder if I should not look at split half reliability and other sort of techniques by which to rather gauge the sample in addition to Mahalanobis and other appropriate outlier techniques?