Courses / HEC NSCT – National Skill Competency Test: Complete Preparation for Computing Graduates
8.3 Supervised Learning and Model Evaluation
Area 8 – AI, Machine Learning & Data Analytics (10%) · 55 min read · 41 MCQs to check yourself
Supervised learning learns from labelled data (input → known output).
| Task | Output | Algorithms |
|---|---|---|
| Regression | A number (house price) | Linear regression, polynomial regression, random forest regressor |
| Classification | A category (spam / not spam) | Logistic regression, decision tree, random forest, k-NN, Naive Bayes, SVM, neural networks |
- k-NN: classify by majority vote of the k nearest points (needs scaling).
- Decision tree: splits on features using information gain / Gini; easy to interpret, can overfit.
- Ensembles: bagging (random forest – many trees on bootstrapped samples) and boosting (AdaBoost, XGBoost – trees fix previous errors).
- SVM: finds the widest-margin boundary; kernels handle non-linear data.
- Naive Bayes: probability with Bayes' theorem, assumes independent features; good for text.
Splitting and validation
Train / validation / test split (e.g. 70/15/15). k-fold cross-validation averages performance over k splits. Never test on training data – data leakage gives falsely high scores.
Overfitting vs underfitting
| Training score | Test score | Fix | |
|---|---|---|---|
| Underfitting (high bias) | Low | Low | More complex model, more features |
| Overfitting (high variance) | High | Low | More data, regularization (L1/L2), simpler model, dropout, early stopping, pruning |
Metrics
Confusion matrix: TP, FP, FN, TN.
| Metric | Formula | Use when |
|---|---|---|
| Accuracy | (TP + TN) ÷ total | Classes balanced |
| Precision | TP ÷ (TP + FP) | False positives are costly (spam filter) |
| Recall (sensitivity) | TP ÷ (TP + FN) | False negatives are costly (cancer, fraud) |
| F1-score | 2PR ÷ (P + R) | Balance of P and R, imbalanced data |
| ROC-AUC | Area under ROC curve | Ranking quality across thresholds |
| Regression metrics: MAE, MSE, RMSE, R² (fraction of variance explained, 1 = perfect). | ||
| Tune hyperparameters (k in k-NN, tree depth, learning rate) with grid/random search on validation data. |
The confusion matrix (StatQuest)
This lesson ends with a quiz of 41 MCQs. Open the lesson on PrepOasis to take it and save your progress.
Previous: 8.2 Data, Preprocessing and Exploratory Data Analysis · Next: 8.4 Unsupervised Learning