# Napkin Folding — probability

Posted by Cameron Davidson-Pilon at

I made a very interesting mistake, and I wanted to share it with you because it's quite enlightening to statistical learning in general. It concerns a penalizer term in maximum-likelihood estimation. Normally, one deals only with the penalizer coefficient, that is, one plays around with $$\lambda$$ in an MLE optimization like: $$\min_{\theta} -\ell(\theta) + \lambda ||\theta||_p^p$$ where $$\ell$$ is the log-likelihood and $$||\cdot||$$ is the $$p$$ norm. This family of problems is typically solved by calculus because both...

## Distribution of the last value in a sum of Uniforms that exceeds 1

Posted by Cameron Davidson-Pilon at

While working on a problem, I derived an interesting result around sums of uniforms random variables. I wanted to record it here so I don't forget it (I haven't solved the more general problem yet!). Here's the summary of the result: Let $$S_n = \sum_{i=1}^n U_i$$ be the sum of $$n$$ Uniform random variables. Let $$N$$ be the index of the first time the sum exceeds 1 (so $$S_{N-1} < 1$$ and $$S_{N} \ge 1$$). The distribution of $$U_N$$...

## Poissonization of Multinomials

Posted by Cameron Davidson-Pilon at

Introduction I've seen some really interesting probability & numerical solutions using a strategy called Poissonization, but Googling for it revealed very few resources (just some references in some textbooks that I don't have quick access to). Below are my notes and repository for Poissonization. After we introduce the theory, we'll do some examples. The technique relies on the following theorem: Theorem: Let $$N \sim \text{Poi}(\lambda)$$ and suppose $$N=n, (X_1, X_2, ... X_k) \sim \text{Multi}(n, p_1, p_2, ..., p_k)$$. Then, marginally, $$X_1, X_2, ..., X_k$$...

Suppose we interested in generating exponential survival times with scale parameter $$\lambda$$, and having $$\alpha$$ probability of censorship, $$0 \le \alpha < 1$$. This is actually, at least from what I tried, a non-trivial problem. I've derived a few algorithms: Algorithm 1  Generate $$T \sim \text{Exp}( \lambda )$$. If $$\alpha = 0$$, return $$(T, 1)$$.   Solve $$\frac{ \lambda h }{ \exp (\lambda h) -1 } = \alpha$$ for $$h$$.  Generate $$E \sim \text{TruncExp}( \lambda, h )$$, where $$\text{TruncExp}$$ is...