CLOUDHow accurate is generative AI and is it enterprise...

How accurate is generative AI and is it enterprise ready?

It seems you can’t escape having a conversation today without the mention of generative AI. But how accurate is the information coming out of generative AI algorithms? 

While the origins of generative AI date back to the 1960’s, the interest has exploded in the past two years. Generative AI is built on large language models (LLMs) and companies are taking different approaches as to how they leverage data, analyze, and present it. 

That’s where many of the similarities end. Interestingly, the results can vary depending on what you ask and how you ask it. Even asking the same question can yield varied results. The point is that there is variability even beyond delving into prompt engineering. 

Taking generative AI for a spin and seeing the varied results

I asked both OpenAI’s ChatGPT and Google Bard the same question: “Who are the top cloud providers ranked?” You would think that this is a fairly pedestrian question. Yet the results had variance in them. 

OpenAI’s ChatGPT response started with a caveat that it does not have access to realtime data or access to the latest rankings. However, here is ranked list as of September 2021. 

  1. Amazon Web Services (AWS)
  2. Microsoft Azure
  3. Google Cloud Platform (GCP)
  4. IBM Cloud
  5. Alibaba Cloud
  6. Oracle Cloud

How accurate is generative AI and is it enterprise ready?

When I asked Google’s Bard the same question, it stated that the results were from 2023 and referred to results based on data from Gartner. 

  1. Amazon Web Services (AWS) 34%
  2. Microsoft Azure 22%
  3. Google Cloud Platform (GCP) 9.5%
  4. Alibaba Cloud 7%
  5. Oracle Cloud Infrastructure (OCI) 4%
  6. IBM Cloud 3.5%
  7. Tencent Cloud 3%
  8. Salesforce 2.5%
  9. Digital Ocean 2%
  10. VMware Cloud on AWS 1.5%

What’s interesting is that using the same question, Bard provided more context, details and a source for the data. This is important as trust in data is more critical than ever. Also note the shift in ranking for IBM Cloud compared with Alibaba Cloud and Oracle Cloud. As a reminder, the same exact question was asked of both services. 

Even within a single provider, the results can vary. I asked very similar questions to Google’s new Search with AI and here are the results. 

I then waited 5-10 minutes and asked a similar question and got yet a slightly different result….and format

Note that within a single provider, the rankings are similar, but the percentages are slightly different. One could argue that 1% or 0.5% isn’t much to argue about and ‘close enough’. However, when you are talking enterprise data and certain industries, 1% can make a big difference. 

What can we learn from these examples?

Does this mean that generative AI is not ready for production yet? No. It can be and should be evaluated. At the same time, understand a few things before leveraging the results. 

First, know your data. Understanding the source is key to building trust. Even if a link to the source data is not provided, mentioning the source(s) provides context and confidence. 

Second, understand that variability exists. As these models evolve, so will the results that come from them. You may have to ask the question multiple times to correlate a more complete answer. 

Lastly, and probably the most important point is that we are still in the experimental phase. While the potential opportunities from generative AI are massive, we still need to spend more time understanding how best to leverage these new tools. 

Does this mean we should not use or trust these tools? No, we should use them. They should be exercised and leveraged. But as with most technology, know where best to leverage it. Getting guidance, creating boiler plate content sound like safe options. Using output to make critical business decisions or to provide granular details is probably less of a good idea for now unless you can correlate the data and insights.


Discover more from AVOA

Subscribe to get the latest posts sent to your email.

Latest news

The DNS zone file is invalid

Learn how to fix the “cPanel Error: The DNS zone file is invalid”. Our cPanel Support team is here...

How to Install Docker on Ubuntu 18.04

Container technology is...

HPE paints a fresh vision that addresses the enterprise demand

HPE invited me to join them for the HPE Open House at their new headquarters in San Jose. During...

Building a Smart City – Compare the Cloud

If you ask me how I picture the future, I would imagine a city where everything worked together as...

Five things critical to the future of SAP and their co-CEOs

Just a few days ago, SAP announced earnings and a significant change to their CEO leadership. After a good...

Must read

Top 10 CIO Trends for 2019

As we get ready to close out 2018 and...

Are the cloud wars over or just getting started?

One of the biggest opportunities for enterprises large and...

You might also likeRELATED
Recommended to you