Last time, I have introduced a parameterization of the half-Cauchy distribution as a hierarchical inverse gamma distributions that allows simple Gibbs sampler (Half-Cauchy distribution). I have just found another parameterization that also allows Gibbs sampler so I wanted to make a post about it.

So there are now two parameterizations of the half-Cauchy distribution:

$\begin{array}{rcl} X\,|\,a &\sim& \mathcal{IG}\left(\dfrac{1}{2},\dfrac{1}{a}\right)\\ a &\sim& \mathcal{IG}\left(\dfrac{1}{2},1\right) \end{array}$

$\begin{array}{rcl}X^{2} &=& U / V\\ U,V &\overset{\text{i.i.d.}}{\sim}& \mathcal{G}a\left(\dfrac{1}{2},\dfrac{1}{2}\right) \end{array}$

In both cases, $X$ follows a half-Cauchy distribution. The second reparameterization, however, expressed as a ratio of gamma random variables, yields a very unstable Gibbs sampler. Consider a linear regression model

$y_{i} = \mathbf{x}_{i}'\boldsymbol{\beta}+\epsilon_{i},\quad \epsilon_{i}\sim \mathcal{N}\left(0,\sigma^{2}\right)$

Let’s assign a horseshoe prior on $\beta_{j}$ which is expressed as a scale mixture of normal:

$\begin{array}{rcl} \beta_{j}\,|\,\tau_{j} &\sim& \mathcal{N}\left(0,\tau_{j}\right)\\ \tau_{j}^{1/2} &\sim&\mathrm{C}^{+}(0,1) \end{array}$

Then, we can set $\tau_{j}=u_{j}/v_{j}$ where $u_{j},v_{j}\overset{\text{i.i.d.}}{\sim} \mathcal{G}a\left(1/2,1/2\right)$ and derive expressions for $u_{j}$ and $v_{j}$ instead. Then, the Gibbs samplers are constructed as follows:

$\begin{array}{rcl} u_{j}\,|\,\text{rest} &\sim& u_{j}^{-1}\exp\left(-\dfrac{1}{2}\left(\dfrac{v_{j}\beta_{j}^{2}}{u_{j}}+u_{j} \right) \right) \\ &\sim& \mathcal{GIG} \left(0,v_{j}\beta_{j}^{2},1\right)\\ v_{j}\,|\,\text{rest} &\sim& \exp \left(-\dfrac{1}{2}\left(\dfrac{\beta_{j}^{2}}{u_{j}}+1 \right)v_{j}\right)\\ &\sim&\mathcal{G}a\left(1, \dfrac{1}{2}\left(\dfrac{\beta_{j}^{2}}{u_{j}}+1 \right)\right)\end{array}$

The issue here is that $\mathcal{GIG}\left(0,0,c\right),\; c\neq 0$ is not defined but we can expect that sometimes irrelevant $\beta_{j}$ could degenerate to zero and in such cases, the sampler crashes. Perhaps, not use it? I don’t know how we can circumvent such issues.