MATLAB code for Sampling Gaussian distribution

function M = sample_gaussian(mu, Sigma, N)
mu = mu(:);
n=length(mu);
[U,D,V] = svd(Sigma);
M = randn(n,N);
M = (U*sqrt(D))*M + mu*ones(1,N);
M = M’;

To Generate Random Numbers from a Dirichlet Distribution

The following code snippet is copied from the MATLAB Topic Modeling Toolbox by Mark Steyvers and Tom Griffiths:

function r = drchrnd(a,n)
% take a sample from a dirichlet distribution
p = length(a);
r = gamrnd(repmat(a,n,1),1,n,p);
r = r ./ repmat(sum(r,2),1,p);

The following is an example that generates three discrete distributions from a symmetric Dirichlet distribution Dir( \theta ; [ 1 1 1 1 ] ):

>> A = drchrnd([1 1 1 1], 3)

A =

0.3889 0.1738 0.0866 0.3507
0.0130 0.0874 0.6416 0.2579
0.0251 0.0105 0.2716 0.6928

>> sum(A, 2)

ans =

1
1
1

Exponential, Power-Law and Log-normal Distributions

• For the standard form of exponential and power-law, refer to Wikipedia.
• For an intuitive introduction to power-law, refer to 幂律分布研究简史.
• For generative models for power-law and log-normal, refer to “A Brief History of Generative Models for Power Law and Lognormal Distributions“.
• For how power-law distributions can be generated from random processes, refer to “Producing power-law distributions and damping word frequencies with two-stage language models”.

Dirichelt Processes and Nonparametric Bayesian Modelling

r.t. An extremely good tutorial for DP. When reading it, take infinite Gaussian mixture model as ab example. After reading it, try to understand HDP.

Questions on DP, CRP and PYP

By Section 4 of [1], it seems that both CRP and PYP produces power-law distributions, where for CRP, the power-law exponent g=1, whereas for PYP it is 1+\alpha.

From the definition of DP, a generalization of Dirichlet distribution, DP should produce multinomial distributions.

But why is the relation between DP and CRP?

References:

1. Producing power-law distributions and damping word frequencies with two-stage language models.