Math Editor

Decision Tree Report

Classification Comparison (Decision Tree Models)

Table 1: Classification Models Performance

Justification:

• Best Model: Random Forest achieved the highest accuracy (96.49%) and F1-score (0.97). Its ensemble approach (aggregating predictions from multiple trees) reduces overfitting and improves generalization.
• Training Time: Basic and manually tuned Decision Trees trained quickly, while GridSearchCV added minimal overhead. Random Forest, though slower due to 100 trees, delivered the best performance.

Regression Comparison (Decision Tree Models)

Table 2: Regression Models Performance

Justification:

• Best Model: Random Forest Regressor outperformed others with the highest R² score (0.7472) and lowest MSE (0.3297). Its ensemble method effectively captures non-linear relationships in housing data.
• Training Time: The limited-depth tree trained fastest but underperformed. The basic tree suffered from overfitting (high MSE), while Random Forest balanced speed and accuracy

Comparison with SVM Results

Classification (SVM vs. Decision Tree):

• SVM Advantages: Higher accuracy (98.25%) and faster training (0.0036s). The RBF kernel captures complex boundaries more effectively.
• Decision Tree Trade-off: Slightly lower accuracy but provides interpretability (e.g., feature importance).

Regression (SVM vs. Decision Tree):

• Decision Tree Advantage: Higher R² score (0.7472 vs. 0.7130) due to better handling of non-linear patterns in regression.
• Training Time: Both models are comparable, but Random Forest is marginally faster.

Conclusion

1. Classification:
   • SVM (RBF Kernel) is superior for high-accuracy, fast predictions.
   • Random Forest is ideal if interpretability and robustness are priorities.
2. Regression:
   • Random Forest Regressor outperforms SVM, offering higher R² and lower MSE.
3. Trade-offs:
   • SVM excels in classification speed and accuracy.
   • Decision Trees (especially ensembles) are better for regression and model transparency.

Visualization