Can Artificial Intelligence Replace the Human Stock Picker?
While AI might be trendy among the general public as of late, the investment industry has been aware of it (and its limitations) for around 3-4 decades now.
My colleague Piero will be participating in “Il Salone del Risparmio” this month which happens to be one of the largest events for the asset management industry in Italy. He asked me to help prepare for an event he’s preparing to participate in. The theme: the role of AI in stock-picking. Here’s my background research/opinion that I offered.
The earliest roots in practical predictive machine learning algorithms is the “Classification and Regression Tree” (CART) proposed by Breiman, Friedman, Olshen, and Stone in 1984. In this model, the distribution of a variable y, given a vector of predictors x, is determined via a binary tree which recursively partitions the predictor space until the distribution of y is more homogenous. This was a seminal work that went on to be studied quite closely in the investment professionals since, at least in theory, virtually every traded instrument is supposed to have at least some bearing on a host of underlying factors.
As early as December 1994, Joseph J. Mezrich - a Director at the dearly-departed Salomon Brothers - highlighted in the Financial Analysts Journal how investing into the S&P 500 index can be optimized using CART such that it produced an average incremental return of 18 basis points a month over the market return and 22 basis points a month over routine index buy-writes.
Note: A “buy-write” is an options trading strategy where an investor buys a security, usually a stock, with options available on it and simultaneously “writes” (sells) a call option on that security. The purpose is to generate income from option premiums.
In the article, three strategies are compared:
Investing into the S&P 100 index options, which go by the ticker “OEX” in the CBOE as a portfolio (denoted as the “Market”);
Investing into the OEX while regularly selling near-term, two-strike, out-of-the-money calls on the portfolio value at the beginning of each month (denoted as the “Buy-Write”);
Investing in a strategy that differs from (2) only in that the regression tree rules dictate whether to sell calls each month (denoted as “Classification Tree” or “Regression Tree”).
Month-wise, the “Classification Tree” doesn’t always pay off.
Overall, it can be seen that while outperformance over the “Market” was frequent, outperformance over “Buy-Write” was achieved less often. In fact, more often than not, the performance was similar. However, in the occasion that they do pay off, the tendency (rare they may be) is that the pay-off is substantial.
Overall market conditions in the period observed indicate a pretty high degree of variability, with some very extreme changes:
The article also identifies this via a dot plot between the S&P 500’s monthly changes and implied volatility. The latter has a substantial impact on an option’s premium, thus impacting the payout of a “buy-write” strategy.
Another frontier for artificial intelligence has been Bayesian estimation methods which stands distinct from a frequentist assumption of probability in that allows for the expression of a degree of belief in the factors influencing an outcome derived from prior observations. Bayesian inferences have proven to be very popular in the financial risk management; it has proven to be a highly acceptable foundation for estimating an expected shortfall on market portfolio and the setting aside of regulator-mandate capital for such a scenario.
An example of this comes from my own archives; specifically from 2007 when I was pursuing my Master’s Degree. One project - constructed without consulting other research and with only an academic background in Bayesian principles - had a rather simple proposition: the programmatic estimation of the average monthly level of the Euro/US Dollar rate on the movement of 101 variables. Of these, 100 variables are economic figures published on a monthly basis across the U.S. and Europe while the 101st was a second-order auto-regressive variable introduced to reinforce the dependent variable’s behaviour on itself in prior iterations as well.
The result:
While the magnitude of monthly changes weren’t estimated very accurately most of the time, the constructed model proved to be directionally accurate more often than not.
Given these two approaches, the natural query that arrives would be: can the two prediction philosophies be melded. The answer is a conditional “Yes”. As early as 1996, a “Bayesian CART” modelling framework, wherein a prior induces a posterior distribution that will guide the stochastic search process towards a stronger CART model, had been proposed by Chipman, George and McCulloch. The vintage, however, doesn’t imply that work on this methodology is exhausted: as recently as 2019, a Working Paper at the Becker Friedman Institute for Economics in the University of Chicago shows that practically used “Bayesian CART” priors tends to lead to a reduction of “white noise”. The Paper also holds the view that “Bayesian CART” and its framework of recursive partitioning is an integral part of the “Bayesian Additive Regression Trees” methodology (“BART”). BART, despite being in its infancy, is considered to be one of the most effective general approaches to predictive modeling under minimal assumptions in the present.
Thus, it should be clear that the foundations of artificial intelligence/machine learning (AI/ML) has never been beyond the radar of the investment industry. The new millennium happened to bring into the forefront the likes of the “scikit-learn” library package which, when integrated with the freeware Python programming environment, enabled a very time- and labour-efficient means of applying many of these techniques on datasets which hitherto were in the domain of the theoretician. For instance, CART is a foundation for the “Random Forest” ensemble technique for prediction within machine learning. With greater awareness among the public came a greater interest in AI/ML. The recent trends in commentary on AI entering financial markets is didn’t necessarily treading new grounds; it’s just now more cost-efficient and scalable.
NVIDIA’s survey of the industry for 2022 estimates that portfolio optimization and trading had the highest number of respondents stating to have a use case for 2021. In 2022, the focus had shifted/increased to fraud detection and conversational AI. In fact, adoption of AI has seen a net increase across the board.
While AI adoption trends might suggest that human stock-pickers are an imperiled species, the simple fact remains that capital markets are dynamic, subject to sudden changes and tend to be driven by underlying factors not necessarily driven by rules. Mezrich’s analysis from nearly 30 years proves that while the language surrounding the ongoing evolution of BART suggests that predictive techniques still have a long way to go. What does tend to do well with automation via AI, however, are rules-driven aspects of the investment industry such as KYC and Compliance.
One benefit of the rapid processing capabilities brought about by the advent of GPUs and the subsequent explosion of deep-learning is that rules-driven processing of the potential inherent in a vast universe of choices in the market becomes increasingly simpler for the stock-picker. Thus, instead of considering the inherent possibility (and almost limitless risk) of replacing the Human Stock-Picker, it might be more germane to consider the vast inherent value in harnessing AI in supplementing said Stock-Picker’s efforts.
Of course, if the machines start getting “smarter”, one might consider hiring a “Killswitch Engineer” like OpenAI (an AI research laboratory founded by the likes of Elon Musk, among others) did:
Note: It’s not known if the job posting (which immediately went viral) was a joke or not. However, OpenAI’s founder Sam Altman - who also helped create ChatGPT - did confirm that their AI indeed are equipped with a “kill switch”.