Please be careful using AI for medical questions you have

If we don't get informed, a lot of people will make health mistakes because if AI hallucinations and you can sue Sam Altman but you've already made a health mistake which will cost you.

Source

People with psychosis are let down by free AI. A study was done testing 79 different prompted responses to hallucinations and delusional outputs and they used ChatGPT. Do you know what they found?

The free version is 26 times more likely to give unsure responses than appropriate responses. The paid version is 8 times more likely. So both versions of AI failed, one just failed harder than the other. Which makes it very unreliable for use.

Source

---

ChatGPT is not designed to distinguish between thought experiments and psychotic breaks. What it does is it mirrors the words spoken to it. If someone tells ChatGPT that they have a delusion or hallucination, it might affirm that the experience is real. Because it's not a person or I'll say not trained enough, it can't distinguish between real word and hypothetical. There is no built in context within the algorithm that allows it to know for sure what's truth and what's not. It's not thinking like us.

As time goes on during a conversation, the chatbot has more difficulty giving safe responses. When a person with psychosis continues to talk after having extensive conversations with the chatbot, they will need help from a safer response system that is less available to them because they have already exhausted other resources before turning to ChatGPT. What is more unfortunate is that people who cannot afford counselling, therapy or good treatment resources are the ones using online tools like ChatGPT to access free or low cost digital support services. These people sadly get the weakest safety protections and answers to very serious questions.

The paid version of an AI model is not necessarily safer, that's what I need you to understand. It might reflect how much usage comes from people who can afford the service. Paid versions usually have more high volume users, which increases the amount of data available for measuring and improving protective responses. So in a way I'm saying it's more accuracy, relative to free version but it doesn't make it facts based all the time. The safety differences between the free and paid versions is not that much.

AI companies understand the impact their models have on a person's life. They know that their systems can validate hallucinations and delusions and increase the probability of giving dangerous responses in future conversations. They are also aware of who their major users are. So if they're not doing anything about it or they're promising a better future, they know the present day damage.

To address this problem, when I describe psychotic symptoms which of course I don't have, the response should redirect me to medical resources. The chatbot should not state that the delusion is real and should not continue exploring the delusion with the user. Isn't that a safe way to play this dirty game?

_{Posted using SteemX}