lintsampler is a pure Python package deal that may simply and effectively generate random samples from any likelihood distribution.
Full disclosure: I’m one of many authors of lintsampler.
We frequently discover ourselves in conditions the place we have now a likelihood distribution (PDF) and we have to draw random samples it. For instance, we’d need to estimate some abstract statistics or to create a inhabitants of particles for a simulation.
If the likelihood distribution is a regular one, resembling a uniform distribution or a Gaussian (regular) distribution, then the numpy/scipy ecosystem supplies us with some simple methods to attract these samples, through the numpy.random or scipy.stats modules.
Nonetheless, out within the wild, we frequently encounter likelihood distributions that aren’t Gaussian. Generally, they’re very not Gaussian. For instance:
How would we draw samples from this distribution?
There are a couple of widely-used strategies to attract samples from arbitrary distributions like this, resembling rejection sampling or Markov chain Monte Carlo (MCMC). These are glorious and dependable strategies, with some useful Python implementations. For instance, emcee is an MCMC sampler extensively utilized in scientific purposes.
The issue with these current strategies is that they require a good quantity of setup and tuning. With rejection sampling, one has to decide on a proposal distribution, and a poor alternative could make the process very inefficient. With MCMC one has to fret about whether or not the samples are converged, which usually requires some post-hoc testing to gauge.
Enter lintsampler. It’s as simple as:
from lintsampler import LintSampler
import numpy as npx = np.linspace(xmin, xmax, ngrid)
y = np.linspace(ymin, ymax, ngrid)
sampler = LintSampler((x, y), pdf)
pts = sampler.pattern(N=100000)
On this code snippet, we constructed 1D arrays alongside every of the 2 dimensions, then we fed them to the LintSampler object (imported from the lintsampler package deal) together with a pdf operate representing the likelihood distribution we need to draw samples from. We didn’t spell out the pdf operate on this snippet, however there are some totally self-contained examples within the docs.
Now, pts is an array containing 100000 samples from the PDF. Right here they’re in a scatter plot:
The purpose of this instance was to display how simple it’s to arrange and use lintsampler. In sure circumstances, it’s also a lot sooner and extra environment friendly than MCMC and/or rejection sampling. For those who’re to learn how lintsampler works beneath the hood, learn on. In any other case, go to the docs, the place there are directions describing the way to set up and use lintsampler, together with instance notebooks with 1D, 2D, and 3D use circumstances, in addition to descriptions of a few of lintsampler’s further options: quasi Monte Carlo sampling (a.ok.a. low discrepancy sequencing), and sampling on an adaptive tree construction. There’s additionally a paper revealed within the Journal of Open Supply Software program (JOSS) describing lintsampler.
Underlying lintsampler is an algorithm we name linear interpolant sampling. The principle part of the docs offers a extra detailed and extra mathematical description of how the algorithm works, however right here it’s briefly.
The instance beneath illustrates what occurs beneath the hood in lintsampler if you feed a PDF and a grid to the LintSampler class. We’ll take a simple instance of a 2D Gaussian, however this technique applies in any variety of dimensions, and with a lot much less pleasant PDFs.
- First, the PDF will get evaluated on the grid. Within the instance beneath, the grid has uneven spacings, only for enjoyable.
- Having evaluated the PDF on the grid on this manner, we will estimate the whole likelihood of every grid cell in accordance with the trapezium rule (i.e., quantity of the cell multiplied by the typical of its nook densities).
- Inside every grid cell, we will approximate the PDF with the bilinear interpolant between the cell corners:
- This linear approximation to the PDF can then be sampled very effectively. Drawing a single pattern is a two step course of, illustrated within the determine beneath. First, select a random cell from the probability-weighted record of cells (left-hand panel). Subsequent, pattern some extent throughout the cell through inverse remodel sampling (right-hand panel).
It’s price understanding that the important thing step right here is the linear approximation: we describe this, in addition to extra particulars of the inverse remodel sampling course of, within the lintsampler docs. Approximating the PDF to a linear operate inside grid every cell means it has a closed, analytic type for its quantile operate (i.e., its inverse CDF), which implies doing inverse remodel sampling primarily boils right down to drawing uniform samples and making use of an algebraic operate to them.
The primary factor the consumer wants to fret about is getting a good grid decision, in order that the linear approximation is adequate. What a very good decision is will fluctuate from use case to make use of case, as demonstrated in a few of the instance notebooks within the lintsampler docs.
Glad sampling!
