Hi Rick, thank you for your post.

I have a question: in your implementation, you used a reference distribution — in this case, N(59, 5).

If my reference distribution is some arbitrary distribution, say F(.), whose parameters need to be estimated from a sample, would the correct procedure be to re-estimate these parameters in each simulation replicate, in order to obtain p-values that reflect the uncertainty associated with parameter estimation in each simulated sample?

What I mean is that, as I understand it, the process would be as follows:

Fit the distribution F(.) to the observed sample;

Generate #replicates of samples of size n from the fitted distribution;

Re-estimate, in each replicate, the parameters of F(.);

Construct, in each replicate, the theoretical CDF using the re-estimated parameters;

Compute, in each replicate, the value of D between the empirical CDF and the fitted CDF;

Store each value of D.

The p-value would then be the proportion of cases where D_sim ≥ D_obs. This procedure accounts for the effect of parameter estimation on the distribution of D, and it typically results in larger (more conservative) p-values compared to the theoretical test without parameter estimation.

My question is: would this procedure be correct?

Thank you very much for your response!