General Discussion
Related: Editorials & Other Articles, Issue Forums, Alliance Forums, Region ForumsSomething Disturbing Happens When You "Learn" Something With ChatGPT
https://futurism.com/artificial-intelligence/learning-with-chatgpt-disturbing“When people rely on large language models to summarize information on a topic for them, they tend to develop shallower knowledge about it compared to learning through a standard Google search,” study co-lead author Shiri Melumad, a professor at the Wharton School of the University of Pennsylvania, wrote in an essay for The Conversation about her work.
The findings are based on an analysis of seven studies with more than 10,000 participants. The gist of their experiments went like this: the participants were told to learn about a topic, and were randomly assigned to either only use an AI chatbot like ChatGPT to do their research, or a standard search engine like Google. At the end, the participants were asked to write advice to a friend about what they learned.
A clear pattern emerged. The participants who used AI to do their research wrote shorter advice, with generic tips and less factual information, while the people who used a Google search produced more detailed and thoughtful tips. The pattern held even after controlling for factors like the information the users saw during their research by showing each group the same facts, or what tools they used.
“The findings confirmed that, even when holding the facts and platform constant, learning from synthesized LLM responses led to shallower knowledge compared to gathering, interpreting and synthesizing information for oneself via standard web links,” Melumad wrote.
-snip-
This new study was published in the journal PNAS Nexus: https://academic.oup.com/pnasnexus/article/4/10/pgaf316/8303888
The essay one of the study's authors wrote: https://theconversation.com/learning-with-ai-falls-short-compared-to-old-fashioned-web-search-269760
Sadly, despite the study showing that using chatbots for research dumbed users down, the study's authors hope to find ways to make chatbots force their increasingly dumbed-down users to work a bit harder and thus learn a bit more.
The obvious solution is really the only smart one: Step away from the chatbot.
= new reply since forum marked as read
Highlight:
NoneDon't highlight anything
5 newestHighlight 5 most recent replies
hlthe2b
(112,308 posts)-proceed to go to established sources to check it. Even if predominantly correct, the wording and/or examples presented by HUMAN language are reinforcing. This AI crap is anything but.
But, that most will not do as I described--especially younger learners (who may not have a 'bone to pick,' ) well, it is going to be disastrous. Second only to relying on those idiotic, know-nothing YouTube INFLUENCERS. 
FakeNoose
(39,688 posts)However I don't think that was the goal of this experiment. They did this hoping it would score zingers for the Chatbots, and it didn't on every level. There aren't any shortcuts to gaining worthwhile knowledge.
RockRaven
(18,502 posts)and "I did my own research"? That is the threshold which is being under-shot.
Society is doomed if this keeps up.
chia
(2,731 posts)some variation of that, and I'm already so tired of it.
Iris
(16,795 posts)And seemingly without a clue about the true costs of just "running this by"
ultralite001
(2,329 posts)ALWAYS... ALWAYS... ALWAYS...
Check the sources that they cite...
Their responses look pretty raunchy when they cite highly questionable or AI-generated sources...
which may not be noticeable if one is not accustomed to doing research... + when so many once
reputable sources have been compromised...
Also, when doing web-browser-based searches -- turn off AI features so the information returned isn't
peppered w/ AI-tainted references... Sometimes you can add "-AI" or "no AI" or "minus AI" to
the end of the search query. What a difference that small change can make.
allegorical oracle
(6,054 posts)
erronis
(22,117 posts)Problem with linking to web sources is the links become stale and the 'source' may have been taken over by some malicious actor (and we all know who many of these are.)
FakeNoose
(39,688 posts)This Chatbot junk is many levels lower than Cliff Notes.
Goonch
(4,136 posts)Differences between Google Search and Google AI Search:
Google Search provides a list of links to web pages, while Google AI Search (like
AI Mode or AI Overviews) generates direct, synthesized answers to queries, often with summaries, comparisons, and interactive tools. Google Search relies on keyword matching, whereas AI Search uses large language models to understand intent and context, leading to a more conversational experience that can be personalized and may include zero-click answers.
Detailed differences between ChatGPT and Google AI Search:
ChatGPT (OpenAI)
and AI on Google Search (powered by the Gemini family of models, Search's AI Overviews) have key differences. These include their primary purpose, real-time data access, and integration with external ecosystems, though both are powerful AI models.
Core Purpose and Interaction
ChatGPT: This AI is primarily designed for conversational assistance. It generates human-like text responses. It excels at creative tasks and complex discussions. It follows detailed instructions to produce in-depth, paragraph-style answers.
AI on Google Search: The models (like Gemini) are integrated into a wider range of services. The core function is leveraging Google's robust search infrastructure. AI Overviews in Google Search distill information. They use a vast index of real-time web pages to provide summaries and relevant source links for a user's query.
Data and Information Access
ChatGPT: Traditionally, this AI relies on a pre-trained dataset. Its knowledge is limited to a certain cutoff date. Paid versions can browse the web. It processes large amounts of diverse internet text for training.
AI on Google Search: This AI has an advantage due to its real-time web crawling and indexing capabilities. It uses a custom Gemini model with Retrieval-Augmented Generation (RAG). This scans the current web for the most reliable sources. This ensures the information is generally up-to-date.
Ecosystem Integration
ChatGPT: This AI offers a rich user-driven plugin and API ecosystem. Developers can integrate its capabilities into various third-party services and platforms.
AI on Google Search: This AI is deeply integrated into the Google ecosystem. It has built-in extensions that draw data from Google products. These include Gmail, Docs, Drive, Maps, and YouTube. This streamlines workflows for users of those services.
Summary Table of Differences
Feature
ChatGPT (OpenAI) AI on Google Search (Gemini, AI Overviews)
Primary Goal Conversational AI, detailed content generation Search enhancement, information retrieval, task assistance
Data Source Primarily pre-trained data (with web browsing available in paid tiers) Real-time web crawling, Google Knowledge Graph
Response Style Detailed, paragraph-style responses, often narrative Concise summaries with numerous linked sources
Ecosystem Third-party plugins and broad API access Deep integration with Google apps (Docs, Gmail, Maps, etc.)
Multimodal Handles text, images, and audio natively Understands and combines text, code, images, audio, and video
highplainsdem
(59,167 posts)to the unreliability of chatbots.
Please don't post AI slop here. Even if it was meant as a joke.
Jack Valentino
(4,096 posts)highplainsdem
(59,167 posts)here, DU will fill up with what will effectively be discussions between chatbots even if humans are prompting the chatbots.
This site has people constantly telling others not to
post all caps
post abbreviations
post videos
post videos without a summary
post messages without whatever background the reader wants even if half the messages on the board are on the same topic
and so on. You can agree or disagree with any of those, but people post messages like that all the time. Including about AI slop. Because I'm not the only one who doesn't want AI slop here.
Jack Valentino
(4,096 posts)I make my own personal points personally, and only use AI, which I clearly identify as such,
to provide some background on what I am talking about, saving myself much time--
as opposed to those who provide no background at all...
I won't apologize for availing myself of such time-saving technology,
and I won't be taking orders from anyone who tells me not to use it here.
Don't like it? Then don't recommend it. No problem.
highplainsdem
(59,167 posts)you don't know at this point that genAI is trained illegally on stolen intellectual property, so your use of it just shows you put your convenience over any concern about that and all the other harm done by genAI.
Jack Valentino
(4,096 posts)above any other considerations whatsoever,
and anyone who would attempt to "unilaterally disarm" me from their use
is unwittingly, or perhaps NOT untwittingly,
standing between me and Trump!
We can argue about the faults of AI after the bigger battle is won,
and many of those faults of AI I would agree with you,
but now is not the time for some kind of 'ideological purism',
in my humble opinion, and I won't be silenced by any such arguments,
even if I was a Bernie backer in 2016, but not 2020....
SheltieLover
(75,402 posts) highplainsdem
(59,167 posts)SheltieLover
(75,402 posts)
Wounded Bear
(63,594 posts)The concept that using AI is worse shouldn't be surprising.
rog
(907 posts)I do the research myself, *without* AI assistance, then once I have read, listened to, watched, etc, all the sources, NotebookLM is pretty darn good at creating summaries in various formats based only on the data I've supplied. i.e., It never strays from the data I've already found and linked as a source. It can create an alphanumeric outline of key points, create technical reports or reports directed to lay people ... and all of this can be edited if you should find some discrepancies or if you find some things are left out or glossed over.
Edited to add that you can use Google search operators like 'site:' to limit your search to mostly reliable sources. For example, 'site:.edu' restricts your search to academic websites. 'site:.edu OR site:.gov' gives results only from university sites or government sites, such as nih.gov, if you're looking for studies or papers. I've prompted ChatGPT to do that, but for one reason or another, it cannot. NOTE: Actually, I have to take that back. Recently I did use ChatGPT, but prompted it to use 'only peer reviewed academic sources', and some good stuff did come back, with annotations that I could follow up.
I've been using NotebookLM to help me create profiles of my doctor appointments. You can upload an mp3 file of the visit, and NotebookLM can do a pretty darn good (not perfect) word-for-word transcription that you can follow along as you listen back. The summaries it creates are pretty spot on ... and you can upload a pdf of the doctor's summary and include that as a second source to get a more complete overview.
I still think 'artificial intelligence' is an oxymoron, but I think computers can be useful as long as they're used as a tool which is under the user's complete control. I would *never* trust AI to do my digging for me, unless it's something really mundane, like what is my city's building code for repairing my porch steps, which just tells me where to look.
highplainsdem
(59,167 posts)or need profiles of doctors' appointments? Although if your doctors are dumb enough to use AI to summarize apppointments, it might be a good idea to have your own recording of the appointment, and to check any notes they have that you're allowed to check.
Just don't expect AI to get it right. There's a reason AI companies make sure you agree to TOS, etc., about being aware AI is unreliable.
rog
(907 posts)I tried to make it clear that I double-check everything. What technology is good for is doing the busy work of organization and getting 'ink on paper', which can then be edited. We are talking about a large volume of technical information here. For example, I'm looking at major cervical fusion surgery. NotebookLM has been really good at helping me organize highly technical information from various academic sources I have found, based on information given by my neurosurgeon, getting all the pros and cons in one place, and putting it into a format that I can study more easily. It has been very good at helping me to 'decode', for lack of a better word, extremely complex medical studies. Again, this particular LLM never strays from a sources (or multiple sources) that I supply.
In another example, I used it to great effect in a recent local election for school board. It is really difficult to get a lot of info about school board candidates, but I found interviews (in a bunch of different places) that individual candidates had done, questionnaires that they had responded to, a couple of facebook posts by a few candidates, newspaper articles here and there in local papers, some audio interviews, a few of them had web pages that offered profiles, etc, etc. I was able to link all those sources and then query that data to compare their positions and points of view regarding various topics I thought were important.
re: "... why do you want or need profiles of doctors' appointments? Although if your doctors are dumb enough to use AI to summarize appointments, it might be a good idea to have your own recording of the appointment, and to check any notes they have that you're allowed to check," I consider that tone EXTREMELY and unnecessarily confrontational, but I'll do my best to respond.
First of all, my doctors are *not* 'dumb'. I want to understand everything that is said in my appointments, and it's not a good idea to rely on one's memory. That's why hospitals recommend taking a friend or relative along, or taking really good notes, or making a recording.
Re: your condescending comment that it 'might be a good idea to have my own recording', if you actually read and understood my post, that is EXACTLY what I do. The voice recognition capability of the LLM allows me to easily make a word-for-word transcription that only requires minor edits. That gives me a hard copy of what was actually said during the appointment, which is MUCH easier to absorb than listening to an audio file. Combining the information from that recoding with the doctor's summary gives me way more detail than either one separately. Using a computer to help me do that is far more efficient than laboriously typing out every word myself, and it puts multiple appointments into a format that I can easily refer to at a later date.
I am 81 years old, and I've been using computers since they became available. I was on the internet before there were browsers, and I remember the first rudimentary, text-only websites. My first browser was LYNX, before that it was telnet, veronica, jughead, archie, etc, etc I believe in using the technology that's available to help me perform tedious tasks.
So, rather than attacking me for whatever reason, how about stating your point of view in a logical and considered manner?
highplainsdem
(59,167 posts)to generative AI. The organizing that goes into writing is critical to thinking. It isn't just getting "ink on paper" that then just needs some editing to make it your own. Too many students using AI that way now are never learning how to organize their thoughts and communicate effectively.
This is important to me because I've been discussing this for years with educators upset by how genAI is dumbing users down. I've read thousands of articles on AI over the last few years, posted hundreds of threads about it here, and also posted a lot about it elsewhere. Besides the dumbing down of users, there's the theft of intellectual property used to train AI tools, which IMO makes them fundamentally unethical to use. Huge issue. Then there's the environmental damage, the waste of money on the AI bubble, etc.
GenAI isn't like earlier tech. It's much more harmful.
I have nothing against tech in general. I first got a PC and first got online - and moderated a forum on politics and technology - in the mid-1980s. Before there was a world wide web, when I had to subscribe to three different online services to keep in touch with people for personal and business reasons. I still appreciated the convenience. Thought word processing software was wonderful. Ditto laser jet printers, though they were expensive then (equivalent to a few thousand today, but computers were also much more expensive then).
Those tools didn't dumb users down and do their thinking for them, and leave them remembering less.
That tech didn't hallucinate while sounding convincing. Didn't have to be checked on every detail because, although it sounded rational, it wasn't.
I'm skeptical of AI transcription being used, wary of doctors using it, because of the errors it can make, and how hard it can be to catch the errors. I've seen too many articles on errors made by AI transcription to consider it trustworthy, with LLMs not just mangling what was said, but adding statements that weren't made, even adding people to meetings when they weren't there.
I'm sorry you're going to need that surgery.
I was my mom's caregiver for years, took her to lots of doctors' appointments. I'll admit I didn't take notes. I do have a very good memory, and I could catch conflicting/wrong advice.
If I were in your situation, I'd probably take notes from a recording, rather than AI transcription, to get the most important details as well as anything important about tone of voice. They have done studies showing notes taken by hand are remembered best.
If you simply can't remember, if you have to have a full transcript, I guess you feel you have to use AI, though it is necessary to check every word.
They had transcription software decades ago, well before LLMs and genAI trained on stolen intellectual property. The Dragon dictation software doctors used goes back a few decades, for instance. It made mistakes, but genAI still does, and at least the older software didn't invent conversations that didn't happen and people that weren't there.
rog
(907 posts)... on almost everything, yet we still seem to be adversaries. I don't know how many times I have to explain what I use this tech for, but I've reached my limit.
I resent your implications that I 'simply can't remember', that I'm incapable of organizing on my own, that my doctors are 'dumb', that I 'let AI do my thinking for me', blablabla, all the other insults and lectures.
I feel that you are purposefully behaving in an argumentative manner, just for the sake of conflict, and I do not appreciate it. This could have been a constructive, supportive conversation; unfortunately that's not the case.
This type of interaction is one of the major reasons why I have a low post count, despite the fact that I've been here since DU's inception. It just is not worth it. It's also the reason why I don't do ANY social media; likewise, it's not worth it.
You have a nice holiday, you hear? Bless your heart.
On the way out, here's a little project for you. Recently I received a notice from Consumer Reports on the hazards of heavy metal contamination in protein supplements. This was of interest, since one of my docs was recommending a protein supplement. The article was great, but I wanted to dig much deeper into the nuances of this topic, even though I'm not a scientist. Take a look at these studies and explain to me how they support or refute each other, what might be the absolute risk vs relative risk, how the study methods compare, how the Consumer Reports analysis of the data compares to the interpretation and analysis of each one of these studies, what the consensus is on whether we even need protein supplements, etc. By the way, I *did* read all 8 of these papers.
https://pmc.ncbi.nlm.nih.gov/articles/PMC7509468/
https://pmc.ncbi.nlm.nih.gov/articles/PMC12286494/
https://pmc.ncbi.nlm.nih.gov/articles/PMC10994440/
https://www.consumerreports.org/lead/protein-powders-and-shakes-contain-high-levels-of-lead-a4206364640/
https://pmc.ncbi.nlm.nih.gov/articles/PMC10815430/
https://pmc.ncbi.nlm.nih.gov/articles/PMC6521232/
https://pmc.ncbi.nlm.nih.gov/articles/PMC10761008/
https://pmc.ncbi.nlm.nih.gov/articles/PMC10458782/
Thanks.
highplainsdem
(59,167 posts)on stolen intellectual property. What I said about the various harms AI does to users has been backed up by studies, a number of which I've posted about here.
I didn't say your doctors were dumb about everything, but that it was dumb to use genAI for transcription, and I explained why.
I'm not interested in playing a game of comparing those studies of protein supplements, but I definitely wouldn't trust a chatbot to do such a comparison, given their error rates. You'd have to do a hell of a lot of checking to ensure its conclusions weren't wrong.
And for that matter, if you need more protein, you're much better off with healthy natural foods than protein powders. As that doctor should have known.
Article explaining why:
https://www.health.harvard.edu/blog/the-scoop-on-protein-powder-2020030918986
Another article on protein:
https://www.health.harvard.edu/nutrition/high-protein-foods-the-best-protein-sources-to-include-in-a-healthy-diet
rog
(907 posts)Now you're giving medical advice to a stranger on the internet ... unsolicited?
re: "You ignored my pointing out ... blablabla ...", I haven't ignored anything you've said before this, but I'm definitely ignoring you now.
Here's some parting advice from me: Don't be a jerk. It's ok to be stupid, though.
highplainsdem
(59,167 posts)foods are considered better than protein powders.You wouldn't have had to look at a bunch of studies on those powders and how much heavy metal they contain, let alone have an error-prone chatbot compare the studies, when that type of supplement itself isn't the best, as that first article I linked to explained.
i'm not being a jerk for pointing that out. And I'm not stupid.
I'm sorry you're fine with using an illegally trained and unreliable type of AI, though. And I hope it never gives you really dangerous advice, as chatbots have given other AI users dangerous advice.
rog
(907 posts)... cook my own food from scratch every day ... only eat whole grains, etc ... never any processed food ... rarely eat out, but when I do, diet is as close to the above as possible. You don't know me, yet you make many very insulting insinuations and assumptions regarding various aspects of my life. When I call you on it, you double down. Good for you.
I'll remind you that I have never disagreed with you or challenged you on any of your points. Rather, I posted how I deal with shortcomings I'm aware of to obtain what I know are good results using technology, with full understanding and attention. That, for some reason, triggers you to the point that you can't just let it drop.
For some reason you have singled me out for your disdain, and it's obvious you need to have the last word, as well as to feel you've come out on top somehow.
I'd like to point out that, while you counsel me on the hazards of using technology to organize information ... something that computers are literally built to do ... I'm supposed to take the recommendations of some crank on the internet at face value regarding everything from my intelligence, to my judgment, to my diet, to my health, to the opinions of my physicians ... on and on. You have lectures on virtually every topic, and I'm supposed to accept *your* slop? Really, it's like I'm dealing with an AI bot, complete with the annoying self-assured arrogance.
I don't think so.
highplainsdem
(59,167 posts)you responded to this one, and I pointed out where the way you're using AI seems problematic in terms of what's known about this very fallible tech.
I haven't said one damn thing about your diet other than pointing out - after you brought up protein supplements and the risk of heavy metal contamination and multiple studies as some sort of weird challenge to me to analyze those studies, which you seem to think requires use of chatbots that hallucinate - that healthy natural foods are better sources of protein. Which is widely known. Natural foods are also a source of protein where you wouldn't have to worry about such contamination or feel you need a chatbot to evaluate studies for you. Anyway, it seemed like an obvious and helpful suggestion to make.
I never expected to be discussing what you eat - YOU brought it up.
You accuse me of disdain, while calling me "some crank on the internet."
As for your attitude...you wrote this silly piece of condescension:
https://pmc.ncbi.nlm.nih.gov/articles/PMC7509468/
https://pmc.ncbi.nlm.nih.gov/articles/PMC12286494/
https://pmc.ncbi.nlm.nih.gov/articles/PMC10994440/
https://www.consumerreports.org/lead/protein-powders-and-shakes-contain-high-levels-of-lead-a4206364640/
https://pmc.ncbi.nlm.nih.gov/articles/PMC10815430/
https://pmc.ncbi.nlm.nih.gov/articles/PMC6521232/
https://pmc.ncbi.nlm.nih.gov/articles/PMC10761008/
https://pmc.ncbi.nlm.nih.gov/articles/PMC10458782/
Thanks.
Honestly, that left me wondering for a second if I should even let you know that protein supplements like those are inferior to real food. But I'll try to help people anyway.
You apparently don't care either about all the problems with using genAI, or the fact that it's an illegally trained type of AI that's fundamentally unethical to use.
I still hope your surgery goes well, and that you won't be harmed by the AI you're using.
erronis
(22,117 posts)Unfortunately some of the scientific journals acknowledge that some of the content is AI generated.
BurnDoubt
(1,285 posts)… in three… two… one………
Or just get off the skid before it kills you.
If we are famous for everything, it’s probably that we rush in with a half-assed idea of what we are doing, and then have to live (or not) with what we have wrought.
Now a bot will make those decisions. Here’s a tip: we need them more than they need us. Will the Future know us as a footnote, or not at all?
paleotn
(21,250 posts)
Sympthsical
(10,787 posts)To formulate assignments, organize notes and presentations, and basically run the core materials through it in order to build lectures and lesson plans.
Some admit to it. Many do not.
They're doing it. Their students are doing it.
The barn door isn't open - it was ripped off its hinges.
And when you think of all the different professions out there, just think that we're releasing subsequent generations of people who will not know how to solve problems when emergencies crop up. Critical thinking will be replaced by asking their technology. And if the technology doesn't know the answer, welp, hope someone's around who does.
highplainsdem
(59,167 posts)There's a lot of resistance to AI, and that resistance isn't going away.
Sympthsical
(10,787 posts)I don't think it's going anywhere. Academia doesn't have the ability or the will to self-police, because they're the chickens, fox, and hen house all in one.
I agree with you. People are not learning core skills as they should - children and adults.
But we're becoming the Wall-E people. Path of least resistance. Working for us is good. Thinking for us is even better.
QueerDuck
(705 posts)Woe are we. Image if we got rid of Google and had them use... gasp!... Libraries with analog books. Ink on paper. Gutenberg printing press and all that! Outlaw typewriters... bring back quills and parchment. Cuneiform and clay tablets rule!
highplainsdem
(59,167 posts)
barbtries
(31,007 posts)but here's how to do a google search without it: type in the search term, space, -AR
rog
(907 posts)That's a tad geeky, but not especially hard. That article also shows how to add a parameter that performs a 'verbatim' search, based on your exact search terms, that is much more targeted. I created two search engines, one with only udm=14, and another one that includes udm=14 and tbs=li:1, for a so-called 'verbatim' search.
Alternatively, you could use websites such as ...
https://udm14.net
CrispyQ
(40,501 posts)The first year of medical students who used ChatGPT are graduating this year. It might be a good time to reevaluate your diet & exercise regimen. ⛹️♀️🏋♂️🚴
BurnDoubt
(1,285 posts)… in three… two… one………
Or just get off the skid before it kills you.
If we are famous for everything, it’s probably that we rush in with a half-assed idea of what we are doing, and then have to live (or not) with what we have wrought.
Now a bot will make those decisions. Here’s a tip: we need them more than they need us. Will the Future know us as a footnote, or not at all?
BurnDoubt
(1,285 posts)Outsmarted Myself.
Upped my Unfrozen Caveman Cred.
OC375
(362 posts)I've noticed that for years. It's easy to spot in IT when you review so much diverse technical information for the first time ever, every single day. The folks leaning on A.I. can't remember, explain, or troubleshoot anything. They just throw A.I. answers at stuff all day, call it quits at 4PM, and tomorrow is like today never happened. Craxy!