Last time, I have introduced a parameterization of the half-Cauchy distribution as a hierarchical inverse gamma distributions that allows simple Gibbs sampler (Half-Cauchy distribution). I have just found another parameterization that also allows Gibbs sampler so I wanted to make a post about it.

So there are now two parameterizations of the half-Cauchy distribution:

\begin{array}{rcl}  X\,|\,a &\sim& \mathcal{IG}\left(\dfrac{1}{2},\dfrac{1}{a}\right)\\ a &\sim&  \mathcal{IG}\left(\dfrac{1}{2},1\right) \end{array}

\begin{array}{rcl}X^{2} &=& U / V\\ U,V &\overset{\text{i.i.d.}}{\sim}& \mathcal{G}a\left(\dfrac{1}{2},\dfrac{1}{2}\right)   \end{array}

In both cases, X follows a half-Cauchy distribution. The second reparameterization, however, expressed as a ratio of gamma random variables, yields a very unstable Gibbs sampler. Consider a linear regression model

y_{i} = \mathbf{x}_{i}'\boldsymbol{\beta}+\epsilon_{i},\quad \epsilon_{i}\sim \mathcal{N}\left(0,\sigma^{2}\right)

Let’s assign a horseshoe prior on $\beta_{j}$ which is expressed as a scale mixture of normal:

\begin{array}{rcl} \beta_{j}\,|\,\tau_{j}  &\sim& \mathcal{N}\left(0,\tau_{j}\right)\\ \tau_{j}^{1/2} &\sim&\mathrm{C}^{+}(0,1)  \end{array}

Then, we can set \tau_{j}=u_{j}/v_{j} where u_{j},v_{j}\overset{\text{i.i.d.}}{\sim} \mathcal{G}a\left(1/2,1/2\right) and derive expressions for u_{j} and v_{j} instead. Then, the Gibbs samplers are constructed as follows:

\begin{array}{rcl} u_{j}\,|\,\text{rest} &\sim& u_{j}^{-1}\exp\left(-\dfrac{1}{2}\left(\dfrac{v_{j}\beta_{j}^{2}}{u_{j}}+u_{j} \right) \right) \\ &\sim& \mathcal{GIG} \left(0,v_{j}\beta_{j}^{2},1\right)\\ v_{j}\,|\,\text{rest} &\sim& \exp \left(-\dfrac{1}{2}\left(\dfrac{\beta_{j}^{2}}{u_{j}}+1 \right)v_{j}\right)\\ &\sim&\mathcal{G}a\left(1, \dfrac{1}{2}\left(\dfrac{\beta_{j}^{2}}{u_{j}}+1 \right)\right)\end{array}

The issue here is that \mathcal{GIG}\left(0,0,c\right),\; c\neq 0 is not defined but we can expect that sometimes irrelevant \beta_{j} could degenerate to zero and in such cases, the sampler crashes. Perhaps, not use it? I don’t know how we can circumvent such issues.