# Napkin Folding — probability

## A real-life mistake I made about penalizer terms

Posted by **Cameron Davidson-Pilon** at

I made a very interesting mistake, and I wanted to share it with you because it's quite enlightening to statistical learning in general. It concerns a penalizer term in maximum-likelihood estimation. Normally, one deals only with the penalizer coefficient, that is, one plays around with \(\lambda\) in an MLE optimization like: $$ \min_{\theta} -\ell(\theta) + \lambda ||\theta||_p^p $$ where \(\ell\) is the log-likelihood and \(||\cdot||\) is the \(p\) norm. This family of problems is typically solved by calculus because both...

## Distribution of the last value in a sum of Uniforms that exceeds 1

Posted by **Cameron Davidson-Pilon** at

While working on a problem, I derived an interesting result around sums of uniforms random variables. I wanted to record it here so I don't forget it (I haven't solved the more general problem yet!). Here's the summary of the result: Let \(S_n = \sum_{i=1}^n U_i \) be the sum of \(n\) Uniform random variables. Let \(N\) be the index of the first time the sum exceeds 1 (so \(S_{N-1} < 1\) and \(S_{N} \ge 1\)). The distribution of \(U_N\)...

## Poissonization of Multinomials

Posted by **Cameron Davidson-Pilon** at

Introduction I've seen some really interesting numerical solutions using a strategy called Poissonization, but Googling for it revealed very few resources (just some references in some textbooks that I don't have access to). So here it is: my notes and repository for Poissonization. Theorem: Let \(N \sim \text{Poi}(\lambda)\) and suppose \(N=n, (X_1, X_2, ... X_k) \sim \text{Multi}(n, p_1, p_2, ..., p_k)\). Then, marginally, \(X_1, X_2, ..., X_k\) are are independent Poisson, with \(X_i \sim \text{Poi}(p_i \lambda)\). [1] The proof is as follows. By...

## Generating exponential survival data

Posted by **Cameron Davidson-Pilon** at

Suppose we interested in generating exponential survival times with scale parameter \(\lambda\), and having \(\alpha\) probability of censorship, \(0 \le \alpha < 1\). This is actually, at least from what I tried, a non-trivial problem. I've derived a few algorithms: Algorithm 1 Generate \(T \sim \text{Exp}( \lambda )\). If \(\alpha = 0\), return \((T, 1)\). Solve \(\frac{ \lambda h }{ \exp (\lambda h) -1 } = \alpha \) for \(h\). Generate \(E \sim \text{TruncExp}( \lambda, h )\), where \(\text{TruncExp}\) is...