Appendix H: Sampling from a simple prior distribution

To better understand the process of sampling from the joint prior distribution of the structural and anchor parameters , consider the following simplified example.

In this scenario, we consider the Bayesian analysis of structural parameters  only, but this can be easily extended to include anchor parameters.  In this example, the structural parameters that are treated as random variables statistically are the integral scale  and the sill  parameters of the variogram model.  Therefore, our goal is to define  and then sample it.

Perhaps based on similar sites or some “soft” local data about the geology, we want to define the integral scale to be uniformly distributed between 100 m and 300 m and the sill (of the natural log of hydraulic conductivity) to be distributed uniformly between 1 and 2 dimensionless. 

Now, we must extend these constraints into an explicit definition for .  Because the random variables are both defined as independent the joint distribution can be written as  and a complete sample of the structural parameter vector  can be obtained by sampling independently from the marginal distributions for integral scale  and sill .  The final step is simply to utilize the soft information to formalize  and , which can be easily done using the minima and maxima listed above.


Sampling these distributions can proceed in many ways: Latin Hypercube sampling, random sampling, stratified sampling, or even by providing a regular or variable mesh over the nonzero probability parameter values.  See Isukupalli, et al. [2006] for a review of many possible methods.

Note: this is a very simplistic prior distribution and there is a wide range of alternatives for defining joint prior distributions see Woodbury & Rubin [2000] or Hou & Rubin [2005] for more comprehensive review.

Finally, note that the objective of any joint prior distribution definition exercise is to be able to explicitly define the distribution  as we did above.  Once the prior distribution is defined, it needs to be sampled, and the realizations can then be processed utilizing MAD#.

For MAD#, we need to sample each of our  priors the same number of times. So in the above example, we would need N samples from  and N samples from . See How do I provide prior distributions? to see how to take these N samples and create a file readable by MAD#.

Now if we also had anchors, each anchor needs to be sampled M times, not necessarily N times. However, to create the file of samples that MAD# requires, these samples need to represent every combination of the priors and anchor priors which results in NxM samples. For each of the M anchor priors, the N  priors need to be repeated (or vice versa, the order in the file doesn’t matter). Once you have the NxM samples created, see page How do I provide prior distributions? of this manual to see how to take these NxM samples and create a file readable by MAD#.


(1)  Hou, Z., and Y. Rubin (2005), On minimum relative entropy concepts and prior compatibility issues in vadose zone inverse and forward modeling, Water Resour. Res., 41(12), W12425, doi:10.1029/2005WR004082.


(2)  Isukapalli, S. S., A. Roy, and P. G. Georgopoulos (2006), Stochastic response surface methods (SRSMs) for uncertainty propagation: Application to environmental and biological systems, Risk Anal., 18(3), 351–363, doi:10.1111/j.1539–6924.1998.tb01301.x.


(3)  Woodbury, A., and Rubin, Y. (2000), A full-Bayesian approach to parameter inference from tracer travel time moments and investigation of scale effects at the Cape Cod Experimental site, Water Resour. Res., 36(1), 159-171, doi: 10.1029/1999WR900273


Last edited Aug 15, 2014 at 9:43 PM by segej87, version 5


No comments yet.