At present, ChatGPT does not have live access to the internet and cannot directly read content located at a specific URL.
This is despite the fact that A) ChatGPT will sometimes falsely claim that it has the ability to access URLs directly and B) when given URLs will confidently make up information AND stick by its answers when called out.
Yet another case for why human expertise and oversight is still a must when using AI.
While ChatGPT has a reputation for making up stuff, this is usually counteracted by the tendency to be upfront about its limitations and backtrack quickly when called out. Not so in this case…
So let’s try a specific example:
Seems convincing right? But that article contains nothing about an acquisition of MGM Studios. So where is this coming from?
Well it’s probably using the information contained in the URL itself as a prompt, and then just guessing the rest in a semi-plausible way. (This makes a bit more sense when you understand a little of how ChatGPT works)
You can see this ruse for what it is when you enter a completely non-descript URL. Like so:
That’s just an excerpt, but all you need to see that it got it completely wrong. The article is actually about building telescopes on the moon.
With no info from the URL itself, ChatGPT has to go with a complete fabrication.
AND after a bunch of followup questions, it just kept doubling down:
So, in conclusion:
ChatGPT cannot access URLs and has a tendency to completely makeup information about the present state of the internet, including articles, existing webpages, twitter accounts etc.
Some of this problem may be due to ChatGPT being set up to always go for the “zero-shot” answer: trying to get it right the first time, which often means taking liberties with context and user intent. You ask for specific information and ChatGPT, with nothing else to go on, treats it like a creative writing challenge.
Has anyone else spotted any interesting yet completely plausible and confident fictions pushed by ChatGPT?