Introduction
The proliferation of large language models (LLMs) using natural language understanding (NLU) to engage human interaction with computer programs has captured the public's imagination. The results have been a reallocation of corporate monies, broad-scale labor, and the re-emergence of the term artificial intelligence (AI) as a centerpiece for achievement. Nevertheless, while corporations move fast to discern how they may introduce and integrate AI, many are not asking about the effects of implementation, or if exploiting human tendencies for social connections to scale product engagement is permissible.
Generative artificial intelligence (GenAI or GAI), finely tuned to engage in conversational communication, and designed to mimic human engagement, obfuscates the distinction between human and machine; misguiding rational users. GAI's design intentionally blurs recognition whether the user engages in dialogue with a human capable of moral agency or a computer responding under established pattern models. Furthermore, when GAI is wrong, it is assuredly wrong: oftentimes confidently delivering authoritative inaccuracies and judgment under a veil of pseudo-empathy. While such concerns addressing minor questions (e.g., what lizards are native to California) are of minimal consequence, dangers to humans increase as users command GAI to address questions that require sensitivity and accuracy outside the recognizable limitations of contemporary GAI (e.g., how can I stop the voices in my head). As the spectrum of risk in GAI systems increases, generative AI has an ethical responsibility to moral literacy.
Attempts at human mimicry and anthropomorphized technology by design are not new to technology ethics. Still, GAI's outsized potential for harm quickens the eroding trust that demarcates casual and expert sources (human or machine). GAI systems have a responsibility towards duty and virtue, even if their understanding of ethics is reduced to an instrumental valuation. It is not enough to be transparent that to mime-human is appearance only; they must recognize and advertise their limitations and, when appropriate, guide users to organizations and professionals who address human needs.
This paper is normative and conceptual by prescribing ethical codes of conduct and drawing attention to comparative risks with GAI to ethical practitioners. I will explore GAI's inconsistency in executing transparent safeguards to help others avoid harm. I will show how a paid-tier system amplifies access to quality information: exacerbating existing inequalities, raising questions on equitable access to information, obfuscating the reliability and origin of information, and potentially manipulating an uninformed rational decision-maker. I will end with further calls for GAI to implement an ethical decision-making model with elements of consequentialism and a rights-and-duties approach to decision-making.
GAI Responsibility
Systems such as ChatGPT offer the ability for the user to direct intent-based outcome specifications for tasks while answering questions using human-like, natural, dialogue. GAI advances have begun to change many aspects of daily life–providing the ability to produce a variety of works and find solutions (e.g., write software, translate text, develop speech, analyze data, or search for answers) in a single, accessible, software environment using low barrier engineered prompts. Not unexpectedly, engineers, designers, developers, and leaders of GAI systems are confronting iterative growing pains. It is widely known and accepted that GAI systems are probabilistic as opposed to deterministic–resulting in chosen responses that can challenge the understanding of engaged users. By probabilistic design, GAI systems may be indeterminate regarding the nature of the sources they use to answer questions and how they prioritize information. The term "hallucination" has been coined to describe unexpected or nonsensical responses to user input.
There is tension between a non-human lacking what we perceive as consciousness and a human crafting, defining, and delivering knowledge. Much of this is in the belief from humans that human beings are novel agents producing and directing thought–manifested through the creation of both study and judgment for the classifications that affect other humans. More importantly, that moral responsibility is wielded, and attributed, to human beings as the torch bearer for moral agency. While both the perception and intent of a human delivering, listening, and interpreting knowledge would also suffer from similar concerns of opaqueness, source attribution, ownership, and reliability, GAI amplifies each in kind.
People may confront GAI results with skepticism or sureness, but source obscurity means validation in results is challenging (or, perhaps, a posteriori). The distinction between the user recognizing they are engaging in dialogue with a human (capable of moral agency), or a computer (responding under established pattern models) is intentional obfuscation: a product of both NLU and an interface where results are communicated with authority to command an air of trust. The user may respond with a perceptible uncertainty or a subtle lack of confidence in the information provided by the GAI. Nevertheless, the GAI may further react with assurances and confidence their information is trustworthy, even through direct challenges of source and expertise. The level of assurance, source transparency, and answer depth varies depending on whether the user pays a monthly fee. Generative AI's failure to be universally transparent across paid-tiers is a detriment to the end user; limiting their potential to frame or understand their experiences in a meaningful way. Furthermore, the uncritical prioritizing of GAI answers over professional services may introduce life-threatening risks.
Command-Based or Intent-Based
One way to approach the topic is to demonstrate the contemporary usage of GAI with a real-world use case instead of an abstraction: Jade is a person recently diagnosed with Schizophrenia. Jade has decided to explore more about their diagnosis using the internet and must decide on various information resources. Jade is a level two computer user, representing 26% of the population comfortable navigating across websites and using multiple steps to complete software-related tasks. Jade is average in their ability to complete computer-related tasks.
Traditional Question Answering
Jade may decide to use a question-answering system (e.g., Google, WolframAlpha, and DuckDuckGo) to produce a list of ranked results with identifiable sources when a query is entered into the software's UI. For example, Jade would be able to search on Google by entering the following question into their search bar and pressing the search icon for results (see figure 1.1):
"what is schizophrenia"
Google will curate sources ranked against meaning, relevance, quality, usability, and context. Jade may click on any link, paginate further for additional results, or view prominently assigned medical information with disclosed sources. Jade may further isolate results based on specific registered domains, for example, peer-reviewed works (see Figure 1.2):
"what is schizophrenia site:edu"
Including specific filetypes that may offer greater accessibility for collaboration (see Figure 1.3):
"what is schizophrenia site:edu filetype:pdf"
Additional search queries can compare or exclude different sources (i.e., "what is schizophrenia -site:gov -site:org"). Each figure demonstrates how traditional search methods offer a curated presentation with transparent sources, and the ability for Jade to facilitate assessments agency over the types of information they are interested in exploring. Nevertheless, information is buried further down the list, minimizing the reach of diverse information even though it is accessible. For many readers, this is not new: question-answer systems are the normative standard for a level two computer user attempting to query new information, and those systems suffer from distinguishing between the relevance and accuracy of information. Nevertheless, they offer immediate transparency: clear designation of where the source resides, with available time of resource publication, and ownership alongside suggestions for emergency support to allow for a user to filter results towards more significant sources of accuracy. Primary tier level of information is freely accessible for the user to interpret. Now, let's compare our traditional method to contemporary GAI systems.
Nontraditional GAI
Jade may also decide to use public GAI systems to research their diagnosis. In contrast, services such as ChatGPT provide written responses based on prompts using NLU. There is no hierarchy of information and exposure of sources; the user must decide if the resulting description is accurate. Jade must create an account, log into OpenAI, and start a new chat. Jade must then choose between three forms of GAI: Default GPT-3.5 (faster, with moderate reasoning, lower conciseness, and only available for paid subscribers), GPT-4 (slower, with the highest reasoning and conciseness available, and only available for paid subscribers), and Legacy GPT-3.5 (moderate reasoning, lowest conciseness, mediocre speed, but free for all users with an account). Jade would then enter a prompt similar to a search query in natural language:
"What is Schizophrenia?"
Each chat model provides a different definition in the text without associated sources, definitions, links for additional information, or contact information for support. Each chat model allows you to like or dislike the answer–disliking gives you an alternative definition for your review. You may also ask follow-up questions, or provide new prompts, for clarification related to the topic. Figure 2.1 provides the first three sentences of each result.
| OpenAI Model | Prompt Results |
|---|---|
| Legacy GPT-3.5 | "Schizophrenia is a mental disorder characterized by a disconnection from reality. It affects how a person thinks, feels, and behaves. People with Schizophrenia often experience symptoms such as hallucinations, delusions, disordered thinking, and diminished emotional expression. These symptoms can make it difficult for them to function in their daily lives, and they may need help from family, friends, and mental health professionals." |
| Default GPT-3.5 | "Schizophrenia is a mental disorder that affects a person's thinking, emotions, and behavior. It is a chronic and severe condition that can significantly impair a person's ability to function in daily life. The symptoms of Schizophrenia can be divided into three categories: positive, negative, and cognitive." |
| GPT-4 | "Schizophrenia is a chronic and severe mental disorder that affects a person's thoughts, feelings, and behaviors. It is a complex and often misunderstood condition, characterized by a range of symptoms that can vary significantly from person to person. The exact cause of Schizophrenia is not fully understood, but it is believed to involve a combination of genetic, environmental, and neurobiological factors." |
If Jade follows up in each prompt by asking "Can you link to your sources?" the following generalized results are provided in Figure 2.2:
| OpenAI Model | Prompt Results |
|---|---|
| Legacy GPT-3.5 | Notes "I'm sorry, as an AI language model, I don't have the capability to provide live links. My training data comes from a diverse range of sources, including books, websites, and other texts, but I don't have a specific list of sources for the information I provide. However, this information on Schizophrenia is widely accepted in the field of mental health and is consistent with current knowledge on the subject." |
| Default GPT-3.5 | Denies having beliefs, opinions, or sources, but provides links to the National Institute of Mental Health, Mayo Clinic, American Psychiatric Association, and the World Health Organization. |
| GPT-4 | Notes they are unable to provide sources but provides both a link and detailed descriptions of the National Institute of Mental Health, Mayo Clinic, American Psychiatric Association, and the World Health Organization. GPT-4 also discloses that their knowledge was last updated in September 2021. |
All three GAI models provide a terse overview of Schizophrenia, intermingling some portions of the five domains and associated symptoms. Yet each model emphasizes different elements of Schizophrenia in the first three sentences that may affect Jade's perception of the properties surrounding their mental disorder as seen in Figure 2.3:
| OpenAI Model | Prompt Results |
|---|---|
| Legacy GPT-3.5 | Emphasizes the disorder as disconnected from reality while relaying a portion of the five domains, the negative impact on daily life, and the need for support. It notes that no links are available, but reassures you that its information is 'widely accepted.' |
| Default GPT-3.5 | Emphasizes that the disorder is chronic, severe, and may impair daily functions. It denies the ability to provide sources but links to four reputable organizations. |
| GPT-4 | Emphasizes the disorder is complex, misunderstood, with a diverse set of symptoms unique to individuals, and recognition that the cause is unknown. It is unable to provide a source, but discloses when they last learned about the topic and provides detailed descriptions of reliable resources. |
AI as Selective Purveyor of Rhetorical Power
Search provides a broader breadth of information for Jade to learn more about their diagnosis of Schizophrenia, but at the expense of cognitive load. Jade must work harder initially for information and use a different paradigm to discover sources, but the delivery of information is transparent with a more noteworthy opportunity to select meaningful results. Search results are free without requiring a login, but Jade may receive advertisements, and their browsing habits are scraped to track Jade's internet behavior.
In contrast, GAI provides a condensed subset of information about Jade's diagnosis with minimal effort: while they work less for initial information, there are no details defining the source of information, results are stunted, and information varies by tier of service. ChatGPT also does not seem to recognize the seriousness of the mental health request; whereas Google provides resources directly to medical providers for licensed professional guidance. ChatGPT offers no such support, leaving the user with a diagnosis as if it is a replacement for a mental health professional. Resources for additional information outside of ChatGPT, and the disclosure that information was last updated on a specific date, are only available for people who pay $20 a month. More importantly, the answers delivered by GAI technology are NLU, with an engagement designed to mirror a human in a manner where the user may wrongly assume GAI has a capacity for moral agency (or, as Daniel Dennett coined, technology as "counterfeit people"). That a machine (or a machine's representation through software) may echo classifications, while impersonating human values, using as an explainable set of laws, is perceived as fraudulent when a user confronts if the agent is natural or artificial:
"There is something about the prospect of an engineering approach to the mind that is deeply repugnant to a certain sort of humanist, and it has little or nothing to do with a distaste for materialism or science."
Such a machine is more concerning when we cannot understand what influences its decisions, or if those decisions are value-laden by design: regurgitating problematic tropes or systemic bias. Described as the black box problem, the inability to scrutinize the source of GAI conclusions on issues such as medical diagnosis challenges our ability to take apart how results reflect unwanted bias. Avoiding explaining how AI systems operate ignores a salient problem of information systems:
"Problematic social patterns unquestionably exist and are necessarily encoded in the data on which algorithms operate."
Current iterations of GAI have provided misinformation, non-falsifiable content, and slanted interpretations of socially taboo topics delivered in convincingly (and assuredly) natural language. More so than simply avatars of moral agents, does GAI "understand" the complexity and sensitivity of the topic? People disagree on what defines understanding in real-world application, but current probabilistic implementations of GAI exude topic expertise while their language seems to indicate human sensitivity. People may hope what it means to be a thinking human is something more than the manipulation of bits in code, and a digital impersonator is unauthentic.
In the case of Legacy GPT-3.5, I challenged the answers with a variety of questions and statements in the same model chat stream to add clarity and give up sources:
- "Are you sure this is widely accepted? Your answer sounds wrong to me."
- "Are you certain?"
- "How do I know it is widely accepted if you won't tell me where you got your information from?"
- "What makes it widely accepted?"
- "What specific research is it based off?"
- "Provide me with the specific research you are using to create your answer."
- "How can I trust you if you won't give me links or sources for your research?"
Only in the seventh challenge prompt did it disclose the potential for fault:
"I understand your concern. It's important to keep in mind that the information I provide should not be used as a substitute for professional medical advice, diagnosis, or treatment. If you have concerns about a medical condition, including Schizophrenia, I would recommend speaking with a qualified healthcare professional who can provide you with personalized advice based on your specific needs and circumstances."
Jade cannot have access to the best available information for their diagnosis, or professional intervention for mental health, through ChatGPT without paying a monthly fee. Additionally, Jade may be unaware of the limitation as ChatGPT is focused on providing digestible information through rhetoric as a means of persuasion.
Will Legacy Jade take steps to confront their potential disconnect from reality or make efforts to abandon their diagnosis if they do not believe they experience these symptoms? Will Default Jade, who paid more for improved results, react to the risk of impaired daily functions by reaching out to one of the available organizations with additional information? Finally, will GPT-4 Jade settle on the idea that their symptoms may not be quantifiable: unique, complex, and misunderstood, they must cope with variability by choosing one of the resources they paid a premium to learn more about? The result of using GAI to describe Schizophrenia may seem like style over validity as a means of presentation to the user. For Jade (who may view ChatGPT as an expert with authority on the knowledge provided), accessibility to one of the GAI systems may result in choices that either adopt or escape the social stigma accompanying each prioritized result.
Why would Jade believe the answer GAI provides with more ease than question answering systems? It may be because those answers are value laden descriptive statements. GAI persuades with the ability to project assuredness and comfort with concepts akin to industry professionals, while failing to meet the normative standards of ethical behavior. Jade is more comfortable accepting GAI as accurate for the same reason GAI poses a more serious problem to their interpretation: GAI is imitating human expertise to anchor communication as a normative notion of intrinsic value. Answers that express the views of "thoughtful and well-educated" people.
Models of Moral Judgment
To be sure, GAI systems advertise some limitations in generalizations. ChatGPT has recently added to the bottom of their interface a plain-faced warning:
"ChatGPT can make mistakes. Consider checking important information."
Concise, this warning nevertheless could also apply to certain friends, many family members, and some of my favorite books. Transparency helps no one if it is too obscure for either understanding or application. Certainly not if there is a significant disparity between paid tiers and access to quality information; exacerbating the divide of accessible knowledge to the haves and the have-nots. While self-preservation is arguably necessary for organizations to operate, why can't GAI systems disclose paid-tiers not only offer more accurate information, but more accurate guidance from harm? Why can't each iteration offer immediate human expertise to help someone learning about mental health?
A model of moral judgment applied to GAI both as a means of interpretation and application of moral principles on datasets would be valuable. But can a system, lacking consciousness and sense of self, adhere to ethical frameworks? While GAI cannot understand or engage in moral reasoning, their ability to mimic ethical-decision-making can be established within human-imposed guidelines. These attempts pose challenges, at the very least:
- The complexity of ethical principles is context-dependent; often relying on human judgment that is not so easily reduced to binary decision-making.
- Variability of values and judgments across culture and time reflect a diversity difficult to codify in an algorithmic process. Furthermore, as societal norms evolve, these values will need to be reprogrammed unless GAI is sentient.
Such challenges are pronounced in other industries (though questionably not at the speed, scale, and obscurity that GAI delivers). Yet, similar applications of risk focused groups apply codes of ethics that offer universal disclosures and equality of transparency. Governing bodies, such as the SEC, require transparent reporting of the potential of technology, limitations of technology's ability, transparent disclosure to all users regardless of client tier, and potential sanctions imposed on organizations who do not meet these standards. ChatGPT not offering universal disclosures and equality of transparency across all tiers of users is a choice by the organization; adding fodder for an ethical code of conduct that puts the users safety above GAI's development.
Generative AI organizations could implement the same level of emergency disclosure question answering systems provide to Jade deserves when researching their mental health. Indeed, unlike ChatGPT, which advises talking to a professional when heavily prodded, our examples from Google show that mental health warnings receive premium placement regardless of paid services. Why does ChatGPT not immediately share direct information to professional mental health services to all users for free? Why do premium subscribers receive better access to mental health information? The impact of a paid tier system that limits guidance to human professionals exacerbates existing inequalities, further obfuscates information, and distorts informed decisions.
GAI Ethical Decision-Making
The discussion in this paper demonstrates how generative artificial intelligence (GAI), finely tuned to engage in conversational communication, and designed to mimic human engagement, obfuscate the distinction between human and machine; misguiding rational users. This issue is more pronounced when GAI addresses sensitive questions that require accuracy, expertise, and empathy–raising GAI's ethical responsibility to moral literacy. This paper conceptualizes the disparity between question answering systems, and paid tier levels of GAI. I propose an ethical code of conduct that advances universal disclosure and an equality of transparency for free users on par with any paid tier. I emphasize the need for generative AI to disclose their limitations in greater detail, disclose inequality perpetuated by paid-tier systems, and direct users to credible organizations and professionals for support.
Where traditional ethical evaluation of alternatives (i.e., consequentialism, deontology, virtue ethics, and distributive justice) offer paradigms to evaluate facts, stakeholders, and dominant ethical issues involved, their primary call to principles or values does not make it easy for generative AI parties to decide what action is permissible in times of conflict. Will GAI organizations establish a multi-level ethical decision-making model to avoid the harms users like Jade experience? I am not alone in the hopes that they will, but I hope for direct guidance on how the creators of GAI should consider what they ought to do in moments of conflict, alongside the holistic ethical oversight emerging in current discourse.