Courses / HEC NSCT – National Skill Competency Test: Complete Preparation for Computing Graduates

8.3 Supervised Learning and Model Evaluation

Area 8 – AI, Machine Learning & Data Analytics (10%) · 55 min read · 41 MCQs to check yourself

Supervised learning learns from labelled data (input → known output).

Task Output Algorithms
Regression A number (house price) Linear regression, polynomial regression, random forest regressor
Classification A category (spam / not spam) Logistic regression, decision tree, random forest, k-NN, Naive Bayes, SVM, neural networks

Splitting and validation

Train / validation / test split (e.g. 70/15/15). k-fold cross-validation averages performance over k splits. Never test on training data – data leakage gives falsely high scores.

Overfitting vs underfitting

Training score Test score Fix
Underfitting (high bias) Low Low More complex model, more features
Overfitting (high variance) High Low More data, regularization (L1/L2), simpler model, dropout, early stopping, pruning

Metrics

Confusion matrix: TP, FP, FN, TN.

Metric Formula Use when
Accuracy (TP + TN) ÷ total Classes balanced
Precision TP ÷ (TP + FP) False positives are costly (spam filter)
Recall (sensitivity) TP ÷ (TP + FN) False negatives are costly (cancer, fraud)
F1-score 2PR ÷ (P + R) Balance of P and R, imbalanced data
ROC-AUC Area under ROC curve Ranking quality across thresholds
Regression metrics: MAE, MSE, RMSE, R² (fraction of variance explained, 1 = perfect).
Tune hyperparameters (k in k-NN, tree depth, learning rate) with grid/random search on validation data.

The confusion matrix (StatQuest)

This lesson ends with a quiz of 41 MCQs. Open the lesson on PrepOasis to take it and save your progress.

Previous: 8.2 Data, Preprocessing and Exploratory Data Analysis · Next: 8.4 Unsupervised Learning

All lessons in HEC NSCT – National Skill Competency Test: Complete Preparation for Computing Graduates