Bayesian Optimization of Lasso and XGBoost Models for Comparative Analysis in Housing Price Prediction

Pamplin College of Business, Virginia Polytechnic Institute and State University, 24060, Blacksburg, the United States

^* Corresponding author: runze@vt.edu

Abstract

Fluctuations in housing prices have a profound impact on the broader economy and people's livelihoods. Accurate housing price predictions contribute to enhanced market transparency and the formulation of evidence-based policies. This paper focuses on optimizing two machine learning models, Lasso Regression and XGBoost, using Bayesian optimization for predicting housing prices. By leveraging economic features such as Average Earnings, Gross Domestic Product (GDP), Mortgage rates, Population, and Unemployment Rate, the models aim to improve prediction accuracy in the housing market. The Lasso model, known for its feature selection capability through L1 regularization, was fine-tuned using Bayesian optimization to minimize mean squared error (MSE). The XGBoost model, designed for handling large-scale, non-linear datasets, was also optimized using the same method. After optimization, the Lasso model achieved an MSE of 240,498,369.05 and an R² score of 0.977, while the XGBoost model showed superior performance with an MSE of 80,273,332.19 and an R² score of 0.9914. SHAP analysis was used to interpret the models, revealing that Average Earnings and GDP were the most influential features in both models. The results demonstrate that while both models perform well, XGBoost's ability to handle non-linearity and high-dimensional data makes it more effective in housing price predictions.