HYBRID PORTFOLIO OPTIMIZATION WITH PCA, CLUSTERING, AND THE BARZILAI-BORWEIN METHOD

Kai Shin Chiew1, Wei Yeing Pan2*, Hong Seng Sim3, Jia Hou Chin4 and Yap Jia Lee5

1,2*,3,5Department of Mathematics and Actuarial Science, Lee Kong Chian
Faculty of Science, University Tunku Abdul Rahman, Sungai Long
Campus, Malaysia.
4School of Accounting and Finance, Faculty of Business and Law,
Taylor’s University, 47500 Subang Jaya, Selangor, Malaysia.

1This email address is being protected from spambots. You need JavaScript enabled to view it., 2*This email address is being protected from spambots. You need JavaScript enabled to view it., 3This email address is being protected from spambots. You need JavaScript enabled to view it.,
4This email address is being protected from spambots. You need JavaScript enabled to view it., 5This email address is being protected from spambots. You need JavaScript enabled to view it.



ABSTRACT

Portfolio optimization aims to balance risk and return by identifying an effective mix of assets. In this study, we integrate principal component analysis (PCA) and hierarchical clustering for stock selection with the Barzilai–Borwein (BB) gradient method for portfolio optimization. Forty-eight U.S. stocks from the Kaggle fundamental stock dataset were initially collected, and 42 stocks were retained after preprocessing. Financial ratios from 2006 and adjusted closing prices from 2016–2017 were analysed, with one representative stock from each cluster selected using the Sharpe ratio. The BB method was then applied to determine optimal weights, ensuring full capital allocation without short selling. Among the tested approaches, the Barzilai–Borwein gradient method 1 (BB1) step size achieved strong performance, producing an annual return of 25.6% while maintaining relatively low volatility. The portfolio also generated a Jensen’s alpha of 1.55, confirming the presence of positive abnormal returns beyond market expectations. These results suggest that combining PCA-based clustering with the BB optimization method offers a practical and efficient way to construct diversified portfolios. The study highlights the BB algorithm’s potential as a lightweight yet effective alternative to more complex optimization techniques in financial decision-making.


Keywords: Barzilai-Borwein Gradient Method, Hierarchical Clustering, Portfolio Optimization, Principal Component Analysis

Published On: 1 April 2026

 

Full Download