Probability theory
Okay, I will create notes based on the topics you listed, drawing from S.C. Gupta and V.K. Kapoor's book (assuming "Fundamentals of Mathematical Statistics"). I will focus on key points, formulas, properties, and results, and include example problems where appropriate.
1. Sample Space and Events
- Sample Space (S): The set of all possible outcomes of a random experiment.
- Event (A): A subset of the sample space.
- Types of Events:
- Simple/Elementary Event: An event containing only one outcome.
- Compound/Composite Event: An event containing more than one outcome.
- Impossible Event: The null set ϕ, which contains no outcomes. P(ϕ)=0
- Sure Event: The entire sample space S. P(S)=1
- Mutually Exclusive Events: Events that cannot occur simultaneously. A∩B=ϕ
- Exhaustive Events: Events whose union is the entire sample space. A1∪A2∪...∪An=S
2. Axioms of Probability
- Axiom 1 (Non-negativity): For any event A, P(A)≥0.
- Axiom 2 (Certainty): P(S)=1.
- Axiom 3 (Additivity): If A and B are mutually exclusive events, then P(A∪B)=P(A)+P(B). More generally, for mutually exclusive events A1,A2,...,An, P(A1∪A2∪...∪An)=i=1∑nP(Ai).
3. Probability Function
A probability function P assigns a real number to each event in the sample space, satisfying the axioms of probability.
4. Laws of Addition
- General Addition Rule: For any two events A and B,
P(A∪B)=P(A)+P(B)−P(A∩B)
If A and B are mutually exclusive, P(A∩B)=0, and the rule simplifies to P(A∪B)=P(A)+P(B).
P(A∪B∪C)=P(A)+P(B)+P(C)−P(A∩B)−P(A∩C)−P(B∩C)+P(A∩B∩C)
Example Problem:
A card is drawn from a well-shuffled deck of 52 cards. Find the probability that it is either a king or a heart.
- Let A be the event of drawing a king. P(A)=524
- Let B be the event of drawing a heart. P(B)=5213
- A∩B is the event of drawing the king of hearts. P(A∩B)=521
- P(A∪B)=P(A)+P(B)−P(A∩B)=524+5213−521=5216=134
5. Conditional Probability
The probability of event A occurring given that event B has already occurred is denoted by P(A∣B) and is defined as:
P(A∣B)=P(B)P(A∩B), provided P(B)>0
6. Law of Multiplication
- P(A∩B)=P(A)⋅P(B∣A)=P(B)⋅P(A∣B)
7. Independence
Two events A and B are independent if the occurrence of one does not affect the probability of the occurrence of the other. Mathematically, A and B are independent if:
P(A∣B)=P(A) or P(B∣A)=P(B) or P(A∩B)=P(A)⋅P(B)
Example Problem:
A coin is tossed twice. Let A be the event that the first toss is a head, and B be the event that the second toss is a head. Are A and B independent?
- P(A)=21
- P(B)=21
- P(A∩B)=P(both heads)=41
- Since P(A∩B)=P(A)⋅P(B), the events A and B are independent.
8. Boole's Inequality
For any events A1,A2,...,An:
P(A1∪A2∪...∪An)≤P(A1)+P(A2)+...+P(An)
9. Bayes' Theorem
Let A1,A2,...,An be a set of mutually exclusive and exhaustive events (i.e., a partition of the sample space), and let B be any event. Then,
P(Ai∣B)=P(B)P(Ai∩B)=∑j=1nP(Aj)⋅P(B∣Aj)P(Ai)⋅P(B∣Ai)
Example Problem:
Bag I contains 4 white and 6 black balls, while Bag II contains 4 white and 3 black balls. One ball is drawn at random from one of the bags and it is found to be black. Find the probability that it was drawn from Bag I.
- Let A1 be the event that the ball is drawn from Bag I. P(A1)=21
- Let A2 be the event that the ball is drawn from Bag II. P(A2)=21
- Let B be the event that the ball drawn is black.
- P(B∣A1)=106 (Probability of drawing a black ball from Bag I)
- P(B∣A2)=73 (Probability of drawing a black ball from Bag II)
Using Bayes' Theorem:
P(A1∣B)=P(A1)⋅P(B∣A1)+P(A2)⋅P(B∣A2)P(A1)⋅P(B∣A1)=21⋅106+21⋅7321⋅106=106+73106=53+7353=3521+1553=353653=53⋅3635=127
10. Random Variables
- Random Variable (RV): A function that assigns a real number to each outcome in the sample space.
- Types of Random Variables:
- Discrete RV: A random variable that can take on only a finite number of values or a countably infinite number of values.
- Continuous RV: A random variable that can take on any value within a given range.
11. Distribution Function
The cumulative distribution function (CDF) of a random variable X is defined as:
F(x)=P(X≤x)
Properties of CDF:
- 0≤F(x)≤1
- F(x) is non-decreasing.
- x→−∞limF(x)=0
- x→∞limF(x)=1
- F(x) is right-continuous.
12. Probability Density Functions (PDF) - Continuous RVs
For a continuous random variable X, the probability density function f(x) satisfies:
- f(x)≥0 for all x
- ∫−∞∞f(x)dx=1
- P(a≤X≤b)=∫abf(x)dx
- F(x)=∫−∞xf(t)dt
- f(x)=dxdF(x)
13. Probability Mass Function (PMF) - Discrete RVs
For a discrete random variable X, the probability mass function p(x) satisfies:
- p(x)≥0 for all x
- x∑p(x)=1
- P(X=x)=p(x)
- F(x)=t≤x∑p(t)
14. Mathematical Expectation
- Expected Value (Mean):
- Discrete RV: E[X]=x∑x⋅p(x)
- Continuous RV: E[X]=∫−∞∞x⋅f(x)dx
- Variance:
- Discrete RV: Var(X)=E[(X−E[X])2]=x∑(x−E[X])2⋅p(x)=E[X2]−(E[X])2
- Continuous RV: Var(X)=E[(X−E[X])2]=∫−∞∞(x−E[X])2⋅f(x)dx=E[X2]−(E[X])2
- Standard Deviation: σ=√Var(X)
15. Moment Generating Functions (MGF)
The moment generating function of a random variable X is defined as:
MX(t)=E[etX]
- Discrete RV: MX(t)=x∑etx⋅p(x)
- Continuous RV: MX(t)=∫−∞∞etx⋅f(x)dx
Properties of MGF:
- The nth moment about the origin is given by: E[Xn]=MX(n)(0), where MX(n)(t) is the nth derivative of MX(t) with respect to t.
16. Cumulants
Cumulants are coefficients in the Taylor series expansion of the logarithm of the characteristic function. They provide an alternative way to describe the distribution of a random variable. The cumulant generating function is K(t)=ln(M(t)).
17. Characteristic Functions
The characteristic function of a random variable X is defined as:
ϕX(t)=E[eitX] where i is the imaginary unit.
- Discrete RV: ϕX(t)=x∑eitx⋅p(x)
- Continuous RV: ϕX(t)=∫−∞∞eitx⋅f(x)dx
18. Theoretical Distributions
- Binomial Distribution
- PMF: P(X=x)=(nx)px(1−p)n−x, where x=0,1,2,...,n
- n = number of trials, p = probability of success on a single trial
- Mean: E[X]=np
- Variance: Var(X)=np(1−p)
- MGF: MX(t)=(1−p+pet)n
- Poisson Distribution
- PMF: P(X=x)=x!e−λλx, where x=0,1,2,...
- λ = average rate of occurrence
- Mean: E[X]=λ
- Variance: Var(X)=λ
- MGF: MX(t)=eλ(et−1)
- Normal Distribution
- PDF: f(x)=σ√2π1e−21(σx−μ)2, where −∞<x<∞
- μ = mean, σ = standard deviation
- Mean: E[X]=μ
- Variance: Var(X)=σ2
- MGF: MX(t)=eμt+21σ2t2
19. Properties and Conditions of a Normal Curve
- Bell-shaped and symmetrical around the mean μ.
- Mean, median, and mode are equal.
- Area under the curve is equal to 1.
- The curve approaches the x-axis asymptotically.
- Points of inflection occur at x=μ±σ.
- Standard Normal Distribution: A normal distribution with μ=0 and σ=1. Denoted as Z∼N(0,1).
20. Test of Significance of Sample and Large Samples
- Null Hypothesis (H0): A statement about the population parameter that we want to test.
- Alternative Hypothesis (H1): A statement that contradicts the null hypothesis.
- Significance Level (α): The probability of rejecting the null hypothesis when it is actually true (Type I error). Common values are 0.05 and 0.01.
- Test Statistic: A value calculated from the sample data to test the null hypothesis.
- P-value: The probability of obtaining a test statistic as extreme as, or more extreme than, the one observed, assuming the null hypothesis is true.
- Decision Rule:
- If p-value ≤α, reject H0.
- If p-value > α, fail to reject H0.
21. Z-test
Used for testing hypotheses about the population mean when the population standard deviation is known, or when the sample size is large (n≥30).
- Test Statistic: z=σ/√nxˉ−μ (for testing the population mean)
where xˉ is the sample mean, μ is the population mean, σ is the population standard deviation, and n is the sample size.
22. Student's t-test
Used for testing hypotheses about the population mean when the population standard deviation is unknown and the sample size is small (n<30).
- Test Statistic: t=s/√nxˉ−μ
where s is the sample standard deviation.
- Degrees of Freedom: df=n−1
23. F-test
Used for testing the equality of variances of two populations. Also used in ANOVA (Analysis of Variance).
- Test Statistic: F=s22s12
where s12 and s22 are the sample variances.
- Degrees of Freedom: df1=n1−1 and df2=n2−1
24. Chi-Square Test
Used for testing hypotheses about categorical data, such as goodness-of-fit tests and tests of independence.
- Test Statistic: χ2=∑Ei(Oi−Ei)2
where Oi is the observed frequency and Ei
Loading Document