Instructor:	Claudio Campagnari
Email:	claudio@physics.ucsb.edu
Office:	Broida 5125.  Phone 805-893-8802
Office Hours:	By appointment, or just drop by

Modern Particle Physics experiments rely heavily on statistical analyses.  As the experiments and the analyses grow in complexity, so do the statistical methods and the tools that are applied.  Some of the tools have evolved into poorly documented complex software black boxes that are often used blindly.  This course is a practical introduction to statistics aimed at providing a foundation towards the understanding of the methods that are commonly applied.  The material is a little bit slanted towards the LHC experience, but students involved in neutrino or dark matter physics should also profit.

There will be no final. There will be some homework for you to do. 

Announcements

• 24 March: If you want to reach me by email with questions/comments about this class, please start the subject line with the string [Physics 226].  This will minimize the chance of me missing your email within the tsunami of emails that lands in my inbox every day.
  

Topics to be covered include:

• Probability, Probability Distribution (Mass) Functions (PDFs/PMFs). Moment Generating Functions (MGFs).
  
• Mean, variance, covariance, correlations.
• Bayesian vs. frequentist statistics. 
• Useful probability distributions for HEX: Poisson, Gaussian, Lognormal, Binomial, Beta, Gamma, Breit-Wigner, Voigtian, Landau, Cruiyff, Crystal Ball.
• Confidence/credible intervals. 
• Limits on number of events:  Bayesian,  Frequentist, Feldman-Cousins, Power Constrained. 
• Hypothesis testing, type I and type II errors, p-values. 
• Inclusion of systematics (nuisances)
• Central Limit Theorem.
• Estimators, Maximum Likelihood
  
• Fitting: chi-squared, log-likelihood, extended log-likelihood. 
• Barlow-Beeston method, morphing.
• Profile likelihood
• CLs method.
• Goodness of fit: Chi-squared, Kolmogorov-Smirnov, Saturated Model approaches
  
• How to use Minuit, how to write your own simple fitter. 
  
• Constrained fits
  
• MC generation methods. Markov Chains
• (Maybe) sPlots, unfolding

Resources

• When I taught the class two years ago, I wrote up my lecture notes.  They are here.  The notes may occasionally contain derivations that I skipped during  the lecture.
• Since the end of the course two years ago, I have been sporadically fixing typos, small mistakes, and broken links embedded in the text, but there are certainly more (and the sPlots chapter is missing).  If you find mistakes, let me know.  See also comment below about Jupyter notebooks.
  
• Occasionally I will show some slides.  These slides basically have pretty pictures from my lecture notes.  You can find them here.
  
• Best book for the class: Statistical Methods for Data Analysis (Luca Lista).  It is available for a free download (only from aithorized IPs, like UCSB's IPs, I think) from here
• The book has all the material I want to cover, but in some cases I will go a little bit deeper in the background theory. 
  
• Other good books:
  
• Roger Barlow's book
• Glenn Cowan's book
• Frederick James book
  
• The statistics review from the PDG is an excellent reference.
  
• Power Constrained Limits (Lecture Notes Chapter 12.3)
  
• Reproducibility of socials science studies.  Can you guess which ones were reproduced? (Lecture Notes Chapter 13.3)
• The argument for blind analyses (Lecture Notes Chapter 13.3)
• Hybrid Frequentist-bayesian methods (Lecture Notes Chapter 14.2)
• The original paper by Cousins and Highland
• See also Section 12.22 of Luca Lista's book
• Equivalence between hybrid CLs and Bayesian methods (Lecture Notes Chapter 14.2)
  
• Punzi's Figure of Merit (Lecture Notes Chapter 16)
• Morphing (Lecture Notes Chapter 25)
  
• Vertical morphing, Section 4 of this paper by John Conway. (Same paper as rhe one on Barlow-Beeston lite, below)
  
• Horizontal morphing
• Moment morphing
  
• Minuit (Lecture Notes Chapter 24)
• Minuit User Guide (C++)
  
• iminuit (Python)
• Interpreting Minuit errors
• Migrad: the Minuit minimizer
  
• The CCGV paper on CLs and Asymptotic CLs. (Lecture Notes Chapter 27)
• Combine Manual  and Combine Paper (Lecture Notes Chapter 27)
• Bob Cousins on GOF (Lecture Notes Chapter 28)
  
• Pitfalls of GOF tests using likelihood (Lecture Notes Chapter 28)
• Discrete Profiling (Envelope) Method (Lecture Notes Chapter 29)
• Barlow-Beeston method (Lecture Notes Chapter 30)
• Original paper
• Barlow-Beeston lite.  Section 5 of this paper by John Conway. (Same paper as the one on morphing above)
  
• A recent improvement.
• Feldman Cousins paper (Lecture Notes Chapter 32)
• Look Elsewhere Effect (Lecture Notes Chapter 33)
• A foundational paper for the crossing method by R. Davies in Biometrika
  
• Applications to HEX
  
• Talk by Gavin Salam about theoretical uncertainties (Lecture Notes Chapter 34)

Some of the Jupyter notebooks to make figure/plots/calculations mentioned in the Lecture Notes are linked below.
(The actual source code is also embedded in the pdf of the Lecture Notes)

However: while everything worked perfectly in 2024, since then there have been changes in the various libraries that may have broken backward compatibility here and there.   I will check/fix possible problems as we proceed through the class.   Again, let me know what problems you find.

• Jupyter notebook to calculate frequentist limits. Chapter 8.1 and 8.1.1, Figures 5 and 6.  Here.
• Jupyter notebook to calculate the effect of systematic on Bayesian limits.   Chapter 14.1 and 14.1.1, Figures 31 and 32.  Here.
• Jupyter notebook to generate random variables using the hybrid acceptance-rejection / CDF inversion method.  Chapter 15.2 and 15.2.1, Figure 36.  Here.
• Jupyter notebook for MCMC example.  Chapter 15.3 and 15.3.1, Figure 37.  Here.
• Jupyter notebook for fit using linear algebra techniques. Chapter 20.3, Figures 40,41, 42.  Here.
• Jupyter notebook for Minuit example. Chapter 24. Here.
• Also in pdf format
  
• Jupyter notebook for Bayesian and CLs Toy methods. Chapter 27.  Here.
• Jupyter notebook for Asymptotic CLs method. Chapter 27. Here.
  

What is covered in each lecture