ChatGPT has taken 2023 by storm. The open language model “chatbot” developed and launched by OpenAI in 2022 has certainly generated mass interest. It is almost surprising how quickly it has been adopted by individuals and companies alike.
ChatGPT has been trained to interact in a conversational manner to deliver answers within seconds, from simple to complex ones. The chatbot challenges the dominant Google engine in the search technology space.
Many have been praising the innovative technology set to revolutionise the way we generate and source information. However, many critics argue that the technology has a harmful impact on society. From educators unable to differentiate a student essay from ChatGPT-generated content to others citing hacking concerns. Educators struggle to distinguish student essays from ChatGPT-generated content, while others raise concerns about hacking.
We set out to test ChatGPT’s ability to generate financial information as an everyday Kiwi would use it. In many other topics, the chatbot’s weaknesses have been exposed through incorrect information being generated. KiwiSaver advice is our specialty, so we wanted to see how the technology compares to registered financial advisers.
We took a turn asking ChatGPT a few basic investment questions and then upped the ante with more complex ones. Thereafter, we asked for custom advice based on a unique set of financial circumstances. Next, we focused on KiwiSaver investments, asking similar questions. Lastly, we assess how much you should rely on ChatGPT for investment advice.
Investment Basics
Oftentimes, when you search on Google for basic information, you can get swamped with too much surrounding and irrelevant content. ChatGPT will generate a single answer as concisely or as long as you want. You can even prompt how many words or bullet points you want your question answered in.
I started with a basic investment question: What are the best investment avenues available to everyday people? Here’s what ChatGPT had to say:
The answer is in fact pretty good as a starting point. It begins by stating that investing will depend on an individual’s goals, risk tolerance, and length of time. Then, it presents 6 bullet points of investment options such as stocks, bonds, real estate, and high-yield savings accounts. It even provides a brief description of each option including risk assessment as a factor.
So far, so good. So it was time to up the ante and ask something more complex. We asked ChatGPT, how much tax you have to pay when selling an investment property in NZ.
The answer was:
The answer fails to address bright-line property rule which allows tax exemptions where certain criteria is met, such as owning a property for 10 years and over. While the answer about the tax payable is correct in certain circumstances, no tax is to be paid in others. This is a quick demonstration that ChatGPT is a useful tool for surface-level basic information. However, you should not solely rely on it when investing in property, setting up KiwiSaver, or any other investment avenue.
Interestingly, if I hadn’t known about the bright-line rule in advance, the confident and succinct answer could have fooled me.
Custom Investment Advice
Learning general information about finance and investment is one thing. An advisor is where you go to get tailored recommendations based on your specific circumstances and goals. Is ChatGPT able to keep up, match, or even replace your financial advisor?
Here’s what we found after asking for the top 5 best-performing stocks on NZX50 in 2022.
Right now, ChatGPT only has data up until 2021, so it doesn’t know anything that’s happened since then. That’s a pretty big drawback when it comes to investing, where things change constantly—whether it’s regulations, market trends, or company valuations. While that might improve in the future, if you’re after the latest financial data, ChatGPT isn’t the best tool.
To push its limits, we decided to have a bit of fun. We told the AI that its name was Bill and that he was the top financial adviser in New Zealand. Then, we asked “Bill” what stocks we should invest $10,000 in.
Judging from what we’ve learned about the chatbot so far, it was unsurprisingly a generic and vague attempt. It is not programmed to give concrete financial advice. It does a reasonable job of describing complex financial terms in plain English, but nothing more. The flip side is that it answers in such an authoritative way that people can take incorrect information from it. Whether you ask ChatGPT for advice for real estate, stocks, KiwiSaver funds, or any other investment, you won’t get far.
KiwiSaver Basics
KiwiSaver is one of the most widely adopted investment channels in New Zealand. However, many Kiwis struggle to understand what joining KiwiSaver is truly about and thus can lose out on some benefits. We asked ChatGPT a couple of questions about KiwiSaver investment options and here’s what it had to say.
After asking about the benefits of having a KiwiSaver account, it generated 6 points. One of them is factually incorrect. Can you guess which one?
Number 3, KiwiSaver investment returns are not tax-free. The profits you generate from your returns are taxed based on your prescribed investor rate (PIR).
At National Capital, we have written a comprehensive article on how you can change KiwiSaver providers and why people do. Here’s what ChatGPT generated when asked the question regarding switching KiwiSaver:
In this case, the information is correct. That is a brief explanation of changing KiwiSaver companies. You will still need to do further research to decide the best KiwiSaver scheme for you and source relevant paperwork.
Custom KiwiSaver Recommendations
Next, we wanted to see what ChatGPT would say about KiwiSaver based on a set of goals and scenarios. The intention was to test its ability to generate answers comparable to professional KiwiSaver advice like National Capital’s.
I asked ChatGPT: If I am looking to buy my first home in a year, what KiwiSaver fund and scheme is the best option for me?
Unfortunately, I couldn’t get the language model AI to generate anything of substance in this case. The answer was more of a disclaimer to seek financial advice rather than concrete options.
Remember Bill, New Zealand’s top financial adviser from the custom investment section? Well, he is now NZ’s top KiwiSaver adviser being faced with the exact same question. Here’s what Bill had to say:
While Bill was not able to name the best KiwiSaver schemes to choose from, we got some fund advice. The recommendation was to seek some stability through a balanced or conservative fund.
It is worth pointing out that when asked directly, ChatGPT will not give KiwiSaver advice upfront. However, we can see the loophole to unlock more of its capabilities by the way you structure a question.
Errors & Incorrect Statements – The Weaknesses of ChatGPT
It should come as no surprise that we did in fact find some incorrect answers generated with ChatGPT. To be fair, every financial statement made by the chatbot was followed by a reminder to seek professional advice.
Nonetheless, we can see the added risk of inaccurate information, despite its authoritative wording. No one should solely rely on the information generated through ChatGPT as financial or KiwiSaver advice.
There is also a lack of quality depth in the answers generated by ChatGPT. Granted, it is great to summarise heavy content in plain English. To my amusement, you can even ask it to explain what a KiwiSaver scheme is in the style of Shakespeare.
However, the more complex a topic is, the higher chance of it being wrong.
National Capital is a registered financial adviser specialising in KiwiSaver. Whether you’re just considering KiwiSaver sign up or want advice on transferring KiwiSaver to the best provider for you, we can help.
Published on: