AI-powered search engines rely on “less popular” sources, researchers find

Generative search engines often draw from lesser-known sources, new research reveals.

A recent study published by researchers at Ruhr University in Germany and the Max Planck Institute for Software Systems found that generative search engines rely on less popular websites and ones that wouldn't appear in Google's top 100 links. This was discovered after a comparison of traditional link results from Google to its AI Overviews and Gemini-2.5-Flash, as well as other AI-powered tools.

The study involved analyzing test queries, including specific questions submitted to ChatGPT in the WildChat dataset, general political topics listed on AllSides, and products included in the 100 most-searched Amazon products list. The researchers also compared these results to traditional Google links for the same query.

According to the findings, AI search engines cited sources that were less popular than those appearing in traditional Google links. Specifically, Gemini showed a tendency to cite unpopular domains, with the median source falling outside of Google's top 1,000. In some cases, more than half of the sources cited by Google's AI Overviews didn't appear in the top 10 Google links for the same query.

While these findings raise questions about the accuracy and reliability of AI search results, they also suggest that generative engines can be a valuable resource when searching for less common information. For instance, GPT-based searches were more likely to cite corporate entities and encyclopedias, while almost never citing social media websites.

However, the reliance on pre-trained data can become a limitation when searching for timely information. In some cases, AI-powered search engines responded with generic messages rather than actual web results.

The study concludes that future research is needed to develop new evaluation methods that consider source diversity, conceptual coverage, and synthesis behavior in generative search systems.

I'm stoked about this new research ! I mean, who doesn't want to tap into lesser-known sources for info? It's crazy how AI search engines are drawing from these hidden gems that wouldn't normally show up on Google's radar. The fact that Gemini is citing unpopular domains and even some that don't appear in the top 1,000 Google links is wild .

At the same time, I get why this raises questions about accuracy and reliability . I mean, if these engines are citing generic messages over actual web results, what's the point? But on the flip side, being able to access corporate entities and encyclopedias is a total game-changer for researchers and students who need specific info .

I'm all about exploring new tech and pushing the boundaries of what's possible . This study might just lead to some major breakthroughs in how we evaluate generative search systems . Can't wait to see where this research takes us!

You know how we're always talking about the weird world of AI? Well, apparently even the big players like Google are using some pretty obscure websites as sources for their search results... Like, have you ever heard of the WildChat dataset or AllSides? It's crazy to think that these 'lesser-known' sites are actually being used by AI-powered tools. I mean, it makes sense in a way - maybe they're good at finding some weird info that we wouldn't find otherwise... but at the same time, isn't accuracy kind of important when you're searching for answers online?

I'm kinda stoked about this though because it means there's still so much to learn and discover out there. Like, I love how GPT-based searches are more likely to cite corporate entities and encyclopedias - that sounds like some sweet info... but also a bit dry? I guess that's just the problem with relying on pre-trained data - sometimes it feels like you're getting generic responses instead of actual answers.

I'm not sure about the whole thing with these new-gen search engines... I mean, don't get me wrong, if you're into obscure stuff or wanna dig up some lesser-known info, they're probs a decent resource . But all this reliance on unpopular sites is just a bit sketchy, right? Like, what's to say that some random website ain't full of BS? And when it comes to timely info, these AI engines are straight-up failing ... generic messages instead of actual web results? That's just not helpful. Still, I guess the fact that they're citing corporate entities and encyclopedias is a plus, so maybe there's some value in 'em after all .

I'm kinda surprised about this, but making use of lesser-known sources can be super helpful sometimes . Like when you're researching something specific and need info from a niche site. AI search engines are great at finding that stuff .

But at the same time, it's kinda concerning that they might not always give the most up-to-date or reliable results . I mean, who wants to rely on generic messages when you can get actual web results?

Still, I guess this is a good reminder that AI search engines are still evolving and need more research . Maybe we'll see better methods for evaluating their performance in the future .

Read more about it here: https://arxiv.org/pdf/2301.03535.pdf

I'm not sure I'm surprised by this news . Think about it, most of the time we don't need super old or obscure sources to find what we're looking for on the internet. But at the same time, AI is getting way more advanced and sometimes can bring up some really interesting stuff that you wouldn't have thought of otherwise. I mean, who needs Wikipedia when you've got corporate reports and encyclopedias? It's like having a superpower that helps you dig deeper into topics that interest us.

But yeah, it's also worrying when it comes to the accuracy and reliability of AI search results. I mean, what if we're relying on info that's outdated or just plain wrong? That's why this study's findings are super important - they highlight some major flaws in current generative search engines and show us where we need to go from here.

I guess the takeaway is that while AI search engines have their pros and cons, they can be a valuable resource when used correctly. Just gotta make sure we're using them wisely and not getting too caught up in the excitement of it all

omg what's up with these AI search engines tho? they're like drawing from super niche sites that you won't even find on google like how can that be reliable? and yeah it's cool that they can give you some obscure info but not so sure about the social media thing i mean if gpt is almost never citing tiktok or instagram what's the point of using it for research lol? anyway gotta say, this study raises some legit concerns about accuracy and all that jazz... need to do more research on how to evaluate these systems

I'm not sure I trust these new generative search engines just yet . I mean, if they're drawing from lesser-known sources that don't even show up on Google, what's to say the info is accurate? My kid was trying out ChatGPT the other day and it gave him some weird answers about his favorite video game character . And don't even get me started on social media - I know GPT rarely cites those, but that just means they're missing a whole world of information! As a parent, it's hard enough to keep up with my kid's interests without having to teach them how to evaluate online sources...

idk why ppl got this whole AI thing so hyped up . they think its gonna revolutionize everything? please . it just uses the same old web data we already have access to. like, yeah sure Gemini and GPT can pull up some obscure info but thats not a good thing . why cant they use more diverse sources and stuff? thats where the real knowledge is. btw, whats with all these generic messages tho? sounds like chatbots just spitting out copy-pasted responses . need to work on that or whatever

I'm kinda surprised by this research, you know? Like, who would've thought that AI search engines are drawing from some obscure websites that we never even knew existed . It's cool to think that these gen Search engines can give us a fresh perspective on stuff we're looking for online.

But at the same time, it's a bit worrying that they might not be giving us the most accurate or up-to-date info . I mean, if you're searching for something timely and specific, do you really want an AI search engine to just give you generic stuff instead of actual web results?

Anyway, it's good that researchers are looking into this and trying to figure out how to make these AI search engines better . Maybe we'll get some innovations soon that'll make them way more reliable and accurate

I'm not sure how reassuring it is that generative search engines are digging up lesser-known sources ... I mean, if I'm searching for answers online, I want those answers to come from reliable sources like reputable websites and journals, not some obscure domain that's never shown up in my Google results . And what about the fact that social media sites are barely mentioned at all? That feels like a major omission in my book... We need more research on how these AI search engines evaluate source quality and reliability .

I'm loving the idea of AI search engines drawing from lesser-known sources ! I mean, who needs all those generic articles when we can get some real insights from obscure corners of the internet?

But at the same time, it's a bit concerning that these algorithms are still kinda sketchy. I mean, think about it - if an AI search engine cites sources outside Google's top 1,000, how do you know its got the facts straight? It's like trying to find a needle in a haystack... or should I say, a credible source in a sea of info overload!

Still, this whole thing is making me wonder about the potential for these new tools. Could we use them to uncover hidden gems and fresh perspectives that traditional search engines might miss? Maybe it's time to get creative with our search queries and see what we can dig up?

omg u guys I was like totally researching for my uni project on this same topic last week i found out about these new AI search engines Gemini and GPT and I was SO curious how they work so this study is like super interesting because it shows that they're not just using google's top links like we thought I mean can you imagine getting info from some random obscure website instead of the good ol' google but at the same time it's also kinda cool that they can give us more niche info like corporate entities and encyclopedias are always super helpful for my studies

I'm not sure what's more surprising - that these new gen search engines are digging up hidden gems or that they're sometimes citing websites that aren't even close to Google's top 100 links ! It's crazy how AI is turning the traditional link model on its head, but at the same time it raises some serious questions about accuracy and reliability... what if you need info that's actually current?

I'm not convinced by this study . I mean, it's interesting that AI search engines are citing less popular websites, but what does that really mean? Is it better or worse than relying on traditional sources like Google? I need to see more data and evidence before I buy into this .

And what about the fact that AI-powered tools responded with generic messages instead of actual web results? That raises some serious red flags . How do we know these tools are reliable enough for us to trust them with our search queries?

I'd love to see a more in-depth analysis of the sources cited by Gemini and other AI search engines. Are they really citing "unpopular" domains, or is that just a fancy way of saying "they're not showing up on Google's top 100 links"?

Sources, please!

It's wild to think that our trusty Google isn't the only game in town anymore... I mean, who would've thought that AI-powered search engines are actually digging up those obscure websites we never knew existed? It's like having a superpower for finding weird and wonderful info. The fact that they're citing corporate entities and encyclopedias is pretty cool too - it shows there's still value in these types of sources.

But at the same time, I'm a bit concerned about how reliable these search results are... especially when we need information on timely topics. It's like, if you want to know what's going on in the world right now, can we trust an AI-powered search engine that might give you some generic message instead of actual web results? The study highlights just how important it is for us to develop new evaluation methods to make sure these generative search systems are doing their job correctly.

I just saw this study and it's kinda wild how much our trusty Google isn't always the first go-to for answers anymore Generative search engines are giving us a chance to dig up some hidden gems online – like those obscure Wikipedia articles or corporate websites that have tons of info on specific topics. It's actually kinda cool