ChatGPT accuracy – a warning – still valid April 2024!

all posts, blogging, business tools, marketing0 comments

In 2023 and 2024 the big thing being talked about, especially in marketing circles, is ChatGPT and all the chat-based AI tools that have arrived on the market. But what is the accuracy of ChatGPT like? I figured a good test of ChatGPT accuracy was to see what it knows about me. After all, I am the authority on me I’d like to think! The test was completed 12 March 2023. I have rerun the test as at 29 April 2024, and it was interestingly less inclined to lie to me. Though it has proven to me many times it still does!

What is ChatGPT from Open AI?

If you’re out of the loop, ChatGPT from OpenAI is a pretty fast and widely hyped online artificial intelligence tool. They started in early 2023 in testing mode, and are therefore encouraging it’s widespread use and ratings of it’s accuracy so it can build it’s intelligence. After all, artificial intelligence (AI) is only as good as the information fed into it. Currently, as at April 2024, the product continues to be a freemium model, with most users using the software for free, with paid options, with greater options now available, including CustomGPTs.

How do you use ChatGPT?

You simply create a login and then ask questions or ‘ChatGPT prompts’. ChatGPT works on an iterative process. So you can ask it to do something like write a bio as I’ve done below. Then you could ask it to rewrite it with a tone of voice like as if you were say Jerry Maguire, or turn it into a poem. It is great for coming up with ideas. Things like 20 tips to … It can even write Excel formulas for you. However, for all these exciting things, it does some things not so well.


The accuracy and source of ChatGPT generated content

ChatGPT openly says in it’s disclaimer that it’s not fully up to date (with data sourced only up to 2021 in the free version), and it is not necessarily using reliable sources. After all how does it know whether, just because it was on the internet, if it was the right answer to a question? Therefore every statistic, every comment needs to be checked before being regurgitated. Ideally, for most purposes, you would not use the ChatGPT output as is.

Particularly if using for blogging, or for long-form content, you want to incorporate your business brand voice, and your unique insights and perspectives.


ChatGPT accuracy test – my biography

So back on 12 March 2023 I asked ChatGPT to provide a 200 word bio for Jane Tweedy. Something I clearly know more about than ChatGPT. This is what happened… [my corrections are in square brackets].

“Jane Tweedy is a successful entrepreneur and business coach based in Sydney, Australia. With over 20 years of experience in corporate management [corporate yes, management not all of it], she has developed an extensive skill set that enables her to help small business owners and solopreneurs grow their businesses.

Jane began her career in the telecommunications industry [where did that come from – no telco ever], where she held various senior management [middle yes, senior no] positions before transitioning to business coaching. In 2011 [hmmm no, years later], she founded her own business coaching practice, FAQ Business Training, [coaching is actually under FAQ Business Consulting] which focuses on helping small business owners and entrepreneurs develop the skills they need to succeed.

As a business coach, Jane is known for her down-to-earth approach and her ability to simplify complex concepts into actionable steps [this is the advice and training component, not really coaching, though I do offer together in one-on-one or group sessions]. She offers a range of services, including one-on-one coaching, group coaching, and workshops, and has helped hundreds of clients achieve their business goals.

In addition to her coaching [and training] practice, Jane is an active member of the business community. She is a regular speaker at industry events and has served as a mentor for several business incubator programs. She is also a member of the Australian Institute of Company Directors [no, never been] and the International Coach Federation.

Overall, Jane Tweedy is a respected business coach and entrepreneur who is dedicated to helping small business owners and entrepreneurs achieve success.”

Overall accuracy of my ChatGPT generated bio

First of all, let’s acknowledge how quick it was! I could put together a 200 word bio as quick, even using my talking it out (voice to text) hack. Overall it scraped reasonably accurate information, with a few really odd things like the Telco and AICD references? This proves my point, you need to check EVERYTHING it produces to make sure it’s accurate before using it. After all asking it about you is the easiest way to see if it is using facts.

When rerun in April 2024, ChatGPT seems to not want to make things up as much. “While the exact industries Jane Tweedy has worked in are not explicitly mentioned, her broad skill set and professional background suggest engagement across various sectors, highlighting her adaptability and effectiveness in diverse work environments.”. So no more made-up Telco reference.

However, it has blatantly lied to me about creating a checklist from a transcribed podcast from my website. Instead it created a generic start a business checklist from information on the internet. Only when questioned did ChatGPT admit it was not from my transcribed podcast blog.

The bio created was also a bit bland and misses my personality. Be sure to add your tone and brand voice when priming ChatGPT to answer on your behalf. This also helps your content to differ from other content out there.

Summary on the output of ChatGPT

So overall summary, have a play with ChatGPT, especially as it’s great for getting something on the page quickly! BUT, don’t assume everything it says is accurate! You need to check and fact-check and apply your tone of voice and insights. Therefore it’s a good base, but not the end product!

You can’t put your head in the sand about AI – it is here to stay! If you’d like to learn to use ChatGPT (even the free version) WAY more effectively, please take our short course. It is included in our FAQBT Business Growth Membership, so it can be a better way to access it!

Need marketing and content tips and help?

FAQ Business provides training, coaching and support to small to medium businesses. Please contact us or check out our online training.

About FAQ Business Training

If this is the first time you’ve come across us you may want to know who we are! FAQ Business Training has a mission to educate and empower action-taking small business owners to learn enough to do it yourself (DIY) or outsource with confidence, to avoid you getting ripped off, or ripping yourself off.

We do this via face to face training in Western Sydney plus online globally, speaking at conferences, events and networking groups. We have launched our online school and membership, offering online courses and webinars to appeal to a global (English speaking) audience. Connect with us on Facebook or LinkedIn.


Found something useful in this blog? There’s plenty more to check out …


Submit a Comment

Your email address will not be published. Required fields are marked *

FAQ Business Training Australia