Home

A Unified Approach to Interpreting Model Predictions Introduction Explanation model Viewing any explanation of a model’s prediction as a model itself. Application of game theory Game theory results guaranteeing a unique solution apply to the entire class of additive feature attribution methods and propose SHAP values as a unified measure ...

XGBoost: A Scalable Tree Boosting System Introduction Designs introduced scalability to XGBoost A novel tree learning algorithm is for handling sparse data. A theoretically justified weighted quantile sketch procedure enables handling instance weights in approximate tree learning. Tree Boosting In a Nutshell R...

Chapter 1 Statistical Models, Goals, And Performance Criteria 1.3 The Decision Theoretic Framework Decision Theoretic Framework Basic elements of a decesion problem Estimation Estimating a real parameter $\theta\in\Theta$ using data $X$ with conditional distribution $P_\theta$. Testing Given data $X\sim P_\theta$, ...

Chapter 1 Statistical Models, Goals, And Performance Criteria 1.2 Bayesian Models Bayesian Framework Statistical Model A random variable $X$ $\mathcal{X}$: Sample Space = {outcomes x} $\mathcal{F}_X$: sigma-field of measurable events $P(\cdot)$: probability distribution defined on $(\mathca...

Regularization Overview Definition In general: Any method to prevent overfitting or help the optimization. Specifically: Additional terms in the training optimization objective to prevent overfitting or help the optimization. Overfitting Key: empirical loss and expected loss are different. Smaller the data set, larger the difference...

Back Propagation Perceptrons A toy model of perceptrons The perceptron training rule Initialize weights $w=0$ Iterate through training instances until convergence Calculate the output for the given instances. \(o = \begin{cases} 1 \,\, if\,\,w_0 + \sum_{i=1}^n w_ix_i>0 \\ 0 \,\,o.w. \end{cases}\) Update each w...

Naive Bayes Basic Ideas Supervised Learning Problem Setting: Set of possible instances: $X$ Unknown target function (concept): $f: X\rightarrow Y$ Set of hypotheses (hypothesis class): $H = {h\lvert h:X\rightarrow Y}$ Given: Training set of instances of unknown target function $f$, $(x^{(1)}, y^{(1)})$, $(x^{(2)}, y^{(2)})$, … , $(x...

Chapter 1 Statistical Models, Goals, And Performance Criteria 1.1 Data, Models, Parameters and Statistics Definition of Statistical Model Random experiment with sample space $\Omega$. Random vector $X=(X_1, \ldots, X_n)$ defined on $\Omega$. $\omega \in \Omega$: outcome of experiment $X(\omega)$: data observations ...