In the above example, it was also interesting that it defaulted to two US-based news outlets and US-centric headlines, though it did include BBC and Aljazeera. It didn’t include something like the Times of India or Deutsche Welle, for example. As an Australian, news outlets like the ABC, the Sydney Morning Herald, or the Herald Sun would have been more applicable (something that would have come up with an actual Bing or Google search). NYT is paywalled, so there’s also some interesting questions there.
In any case, ChatGPT doesn’t always exercise what I’d call “expert-level Google-fu” (Or in this case, Bing-fu) when it comes to looking for results. For instance, it might look around Twitter for an answer, but utterly ignore other avenues of research, resulting in incorrect answers that a bit more diligence would have solved.
This sort of problem could be solved with a bit more targeted prompt engineering, or sequential queries. I’m sure this is to limit the tokens expended per query; if ChatGPT does a search that’s too thorough, there’s likely less tokens left to respond to you.
However, if you’re working with a fairly simple query, it doesn’t need to do multiple searches. For instance, here’s what happened when we asked it to check the latest blog post on Pluralsight.com.
Previously, ChatGPT would give you an answer and not tell you what its sources were at all. In fact, it famously hallucinates some of these and makes up books and papers that don’t even exist. However, when using ChatGPT, suddenly the chatbot was all about cross-checking. There were little hyperlink numbers above a statement that told me exactly what domain it had gotten its information from!
What was more impressive is that it even apologized if it couldn’t cross reference something, shared with me the source that it had used, and assured me it was credible.
