Spaces:
Sleeping
Sleeping
refine text on probability density functions and their properties.
Browse files
probability/16_continuous_distribution.py
CHANGED
|
@@ -14,7 +14,7 @@
|
|
| 14 |
|
| 15 |
import marimo
|
| 16 |
|
| 17 |
-
__generated_with = "0.
|
| 18 |
app = marimo.App(width="medium")
|
| 19 |
|
| 20 |
|
|
@@ -26,7 +26,9 @@ def _(mo):
|
|
| 26 |
|
| 27 |
_This notebook is a computational companion to ["Probability for Computer Scientists"](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/continuous/), by Stanford professor Chris Piech._
|
| 28 |
|
| 29 |
-
|
|
|
|
|
|
|
| 30 |
"""
|
| 31 |
)
|
| 32 |
return
|
|
@@ -38,20 +40,17 @@ def _(mo):
|
|
| 38 |
r"""
|
| 39 |
## From Discrete to Continuous
|
| 40 |
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
> Imagine you're running to catch a bus. You know you'll arrive at 2:15pm, but you don't know exactly when the bus will arrive. You want to model the bus arrival time (in minutes past 2pm) as a random variable $T$ so you can calculate the probability that you'll wait more than five minutes: $P(15 < T < 20)$.
|
| 44 |
|
| 45 |
-
|
| 46 |
|
| 47 |
-
|
| 48 |
|
|
|
|
| 49 |
- What's the probability the bus arrives at exactly 2:17pm and 12.12333911102389234 seconds?
|
| 50 |
-
- What's the probability
|
| 51 |
|
| 52 |
-
These questions
|
| 53 |
-
|
| 54 |
-
### Visualizing the Transition
|
| 55 |
|
| 56 |
Let's visualize this transition from discrete to continuous:
|
| 57 |
"""
|
|
@@ -150,44 +149,43 @@ def _(mo):
|
|
| 150 |
r"""
|
| 151 |
## Probability Density Functions
|
| 152 |
|
| 153 |
-
|
| 154 |
|
| 155 |
-
|
| 156 |
|
| 157 |
$$f(X=x) \quad \text{or simply} \quad f(x)$$
|
| 158 |
|
| 159 |
-
Where the lowercase $x$
|
| 160 |
|
| 161 |
### Key Properties of PDFs
|
| 162 |
|
| 163 |
-
|
| 164 |
|
| 165 |
-
1. The probability that $X$
|
| 166 |
|
| 167 |
$$P(a \leq X \leq b) = \int_a^b f(x) \, dx$$
|
| 168 |
|
| 169 |
-
2.
|
| 170 |
|
| 171 |
$$f(x) \geq 0 \text{ for all } x$$
|
| 172 |
|
| 173 |
-
3.
|
| 174 |
|
| 175 |
$$\int_{-\infty}^{\infty} f(x) \, dx = 1$$
|
| 176 |
|
| 177 |
-
4. The probability
|
| 178 |
|
| 179 |
$$P(X = a) = \int_a^a f(x) \, dx = 0$$
|
| 180 |
|
| 181 |
-
This last property
|
| 182 |
-
|
| 183 |
-
### Caution: Density ≠ Probability
|
| 184 |
|
| 185 |
-
|
| 186 |
|
| 187 |
-
|
| 188 |
|
| 189 |
-
|
| 190 |
-
|
|
|
|
| 191 |
"""
|
| 192 |
)
|
| 193 |
return
|
|
@@ -665,16 +663,18 @@ def _(fig_to_image, mo, np, plt, sympy):
|
|
| 665 |
# Detailed calculations for our example
|
| 666 |
_calculations = mo.md(
|
| 667 |
f"""
|
| 668 |
-
###
|
| 669 |
|
| 670 |
-
|
|
|
|
|
|
|
| 671 |
|
| 672 |
$$f(x) = \\begin{{cases}}
|
| 673 |
\\frac{{3}}{{8}}(4x - 2x^2) & \\text{{when }} 0 < x < 2 \\\\
|
| 674 |
0 & \\text{{otherwise}}
|
| 675 |
\\end{{cases}}$$
|
| 676 |
|
| 677 |
-
#### Expectation
|
| 678 |
|
| 679 |
$$E[X] = \\int_{{-\\infty}}^{{\\infty}} x \\cdot f(x) \\, dx = \\int_0^2 x \\cdot \\frac{{3}}{{8}}(4x - 2x^2) \\, dx$$
|
| 680 |
|
|
@@ -684,9 +684,9 @@ def _(fig_to_image, mo, np, plt, sympy):
|
|
| 684 |
|
| 685 |
$$E[X] = \\frac{{3}}{{8}} \\cdot \\frac{{32 - 12}}{{3}} = \\frac{{3}}{{8}} \\cdot \\frac{{20}}{{3}} = \\frac{{20}}{{8}} = {E_X}$$
|
| 686 |
|
| 687 |
-
#### Variance
|
| 688 |
|
| 689 |
-
|
| 690 |
|
| 691 |
$$E[X^2] = \\int_{{-\\infty}}^{{\\infty}} x^2 \\cdot f(x) \\, dx = \\int_0^2 x^2 \\cdot \\frac{{3}}{{8}}(4x - 2x^2) \\, dx$$
|
| 692 |
|
|
@@ -696,11 +696,11 @@ def _(fig_to_image, mo, np, plt, sympy):
|
|
| 696 |
|
| 697 |
$$E[X^2] = \\frac{{3}}{{8}} \\cdot \\frac{{20 - 64/5}}{{1}} = {E_X2}$$
|
| 698 |
|
| 699 |
-
Now we
|
| 700 |
|
| 701 |
$$\\text{{Var}}(X) = E[X^2] - (E[X])^2 = {E_X2} - ({E_X})^2 = {Var_X}$$
|
| 702 |
|
| 703 |
-
|
| 704 |
"""
|
| 705 |
)
|
| 706 |
mo.vstack([_img, _calculations])
|
|
@@ -779,7 +779,7 @@ def _(mo):
|
|
| 779 |
return
|
| 780 |
|
| 781 |
|
| 782 |
-
@app.cell
|
| 783 |
def _(mo):
|
| 784 |
mo.md(r"""Appendix code (helper functions, variables, etc.):""")
|
| 785 |
return
|
|
@@ -971,7 +971,6 @@ def _(np, plt, sympy):
|
|
| 971 |
1. Total probability: ∫₀² {C}(4x - 2x²) dx = {total_prob}
|
| 972 |
2. P(X > 1): ∫₁² {C}(4x - 2x²) dx = {prob_gt_1}
|
| 973 |
"""
|
| 974 |
-
|
| 975 |
return create_example_pdf_visualization, symbolic_calculation
|
| 976 |
|
| 977 |
|
|
|
|
| 14 |
|
| 15 |
import marimo
|
| 16 |
|
| 17 |
+
__generated_with = "0.12.6"
|
| 18 |
app = marimo.App(width="medium")
|
| 19 |
|
| 20 |
|
|
|
|
| 26 |
|
| 27 |
_This notebook is a computational companion to ["Probability for Computer Scientists"](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/continuous/), by Stanford professor Chris Piech._
|
| 28 |
|
| 29 |
+
Continuous distributions are what we need when dealing with random variables that can take any value in a range, rather than just discrete values.
|
| 30 |
+
|
| 31 |
+
The key difference here is that we work with probability density functions (PDFs) instead of probability mass functions (PMFs). It took me a while to really get this - the PDF at a point isn't actually a probability, but rather a density.
|
| 32 |
"""
|
| 33 |
)
|
| 34 |
return
|
|
|
|
| 40 |
r"""
|
| 41 |
## From Discrete to Continuous
|
| 42 |
|
| 43 |
+
Making the jump from discrete to continuous random variables requires a fundamental shift in thinking. Let me walk you through a thought experiment:
|
|
|
|
|
|
|
| 44 |
|
| 45 |
+
> You're rushing to catch a bus. You know you'll arrive at 2:15pm, but the bus arrival time is uncertain. If you model the bus arrival time (in minutes past 2pm) as a random variable $T$, how would you calculate the probability of waiting more than five minutes: $P(15 < T < 20)$?
|
| 46 |
|
| 47 |
+
This highlights a crucial difference from discrete distributions. With discrete distributions, we calculated probabilities for exact values, but this approach breaks down with continuous values like time.
|
| 48 |
|
| 49 |
+
Consider these questions:
|
| 50 |
- What's the probability the bus arrives at exactly 2:17pm and 12.12333911102389234 seconds?
|
| 51 |
+
- What's the probability a newborn weighs exactly 3.523112342234 kilograms?
|
| 52 |
|
| 53 |
+
These questions have no meaningful answers because continuous measurements can have infinite precision. In the continuous world, the probability of a random variable taking any specific exact value is actually zero!
|
|
|
|
|
|
|
| 54 |
|
| 55 |
Let's visualize this transition from discrete to continuous:
|
| 56 |
"""
|
|
|
|
| 149 |
r"""
|
| 150 |
## Probability Density Functions
|
| 151 |
|
| 152 |
+
While discrete random variables use Probability Mass Functions (PMFs), continuous random variables require a different approach — Probability Density Functions (PDFs).
|
| 153 |
|
| 154 |
+
A PDF defines the relative likelihood of a continuous random variable taking particular values. We typically denote this with $f$ and write it as:
|
| 155 |
|
| 156 |
$$f(X=x) \quad \text{or simply} \quad f(x)$$
|
| 157 |
|
| 158 |
+
Where the lowercase $x$ represents a specific value our random variable $X$ might take.
|
| 159 |
|
| 160 |
### Key Properties of PDFs
|
| 161 |
|
| 162 |
+
For a PDF $f(x)$ to be valid, it must satisfy these properties:
|
| 163 |
|
| 164 |
+
1. The probability that $X$ falls within interval $[a, b]$ is:
|
| 165 |
|
| 166 |
$$P(a \leq X \leq b) = \int_a^b f(x) \, dx$$
|
| 167 |
|
| 168 |
+
2. Non-negativity — the PDF can't be negative:
|
| 169 |
|
| 170 |
$$f(x) \geq 0 \text{ for all } x$$
|
| 171 |
|
| 172 |
+
3. Total probability equals 1:
|
| 173 |
|
| 174 |
$$\int_{-\infty}^{\infty} f(x) \, dx = 1$$
|
| 175 |
|
| 176 |
+
4. The probability of any exact value is zero:
|
| 177 |
|
| 178 |
$$P(X = a) = \int_a^a f(x) \, dx = 0$$
|
| 179 |
|
| 180 |
+
This last property reveals a fundamental difference from discrete distributions — with continuous random variables, probabilities only make sense for ranges, not specific points.
|
|
|
|
|
|
|
| 181 |
|
| 182 |
+
### Important Distinction: Density ≠ Probability
|
| 183 |
|
| 184 |
+
One common mistake is interpreting $f(x)$ as a probability. It's actually a **density** — representing probability per unit of $x$. This is why $f(x)$ values can exceed 1, provided the total area under the curve equals 1.
|
| 185 |
|
| 186 |
+
The true meaning of $f(x)$ emerges only when:
|
| 187 |
+
1. We integrate over a range to obtain an actual probability, or
|
| 188 |
+
2. We compare densities at different points to understand relative likelihoods.
|
| 189 |
"""
|
| 190 |
)
|
| 191 |
return
|
|
|
|
| 663 |
# Detailed calculations for our example
|
| 664 |
_calculations = mo.md(
|
| 665 |
f"""
|
| 666 |
+
### Computing Expectation and Variance
|
| 667 |
|
| 668 |
+
> _Note:_ The following mathematical derivation is included as reference material. The credit for this approach belongs to ["Probability for Computer Scientists"](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/continuous/) by Chris Piech.
|
| 669 |
+
|
| 670 |
+
Let's work through the calculations for our PDF:
|
| 671 |
|
| 672 |
$$f(x) = \\begin{{cases}}
|
| 673 |
\\frac{{3}}{{8}}(4x - 2x^2) & \\text{{when }} 0 < x < 2 \\\\
|
| 674 |
0 & \\text{{otherwise}}
|
| 675 |
\\end{{cases}}$$
|
| 676 |
|
| 677 |
+
#### Finding the Expectation
|
| 678 |
|
| 679 |
$$E[X] = \\int_{{-\\infty}}^{{\\infty}} x \\cdot f(x) \\, dx = \\int_0^2 x \\cdot \\frac{{3}}{{8}}(4x - 2x^2) \\, dx$$
|
| 680 |
|
|
|
|
| 684 |
|
| 685 |
$$E[X] = \\frac{{3}}{{8}} \\cdot \\frac{{32 - 12}}{{3}} = \\frac{{3}}{{8}} \\cdot \\frac{{20}}{{3}} = \\frac{{20}}{{8}} = {E_X}$$
|
| 686 |
|
| 687 |
+
#### Computing the Variance
|
| 688 |
|
| 689 |
+
We first need $E[X^2]$:
|
| 690 |
|
| 691 |
$$E[X^2] = \\int_{{-\\infty}}^{{\\infty}} x^2 \\cdot f(x) \\, dx = \\int_0^2 x^2 \\cdot \\frac{{3}}{{8}}(4x - 2x^2) \\, dx$$
|
| 692 |
|
|
|
|
| 696 |
|
| 697 |
$$E[X^2] = \\frac{{3}}{{8}} \\cdot \\frac{{20 - 64/5}}{{1}} = {E_X2}$$
|
| 698 |
|
| 699 |
+
Now we calculate variance using the formula $Var(X) = E[X^2] - (E[X])^2$:
|
| 700 |
|
| 701 |
$$\\text{{Var}}(X) = E[X^2] - (E[X])^2 = {E_X2} - ({E_X})^2 = {Var_X}$$
|
| 702 |
|
| 703 |
+
This gives us a standard deviation of $\\sqrt{{\\text{{Var}}(X)}} = {Std_X}$.
|
| 704 |
"""
|
| 705 |
)
|
| 706 |
mo.vstack([_img, _calculations])
|
|
|
|
| 779 |
return
|
| 780 |
|
| 781 |
|
| 782 |
+
@app.cell(hide_code=True)
|
| 783 |
def _(mo):
|
| 784 |
mo.md(r"""Appendix code (helper functions, variables, etc.):""")
|
| 785 |
return
|
|
|
|
| 971 |
1. Total probability: ∫₀² {C}(4x - 2x²) dx = {total_prob}
|
| 972 |
2. P(X > 1): ∫₁² {C}(4x - 2x²) dx = {prob_gt_1}
|
| 973 |
"""
|
|
|
|
| 974 |
return create_example_pdf_visualization, symbolic_calculation
|
| 975 |
|
| 976 |
|