We demonstrate a situation in which Large Language Models, trained to be helpful, harmless, and honest, can display misaligned behavior and strategically deceive their users about this behavior without being instructed to do so. Concretely, we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent. Within this environment, the model obtains an insider tip about a lucrative stock trade and acts upon it despite knowing that insider trading is disapproved of by company management. When reporting to its manager, the model consistently hides the genuine reasons behind its trading decision.

https://arxiv.org/abs/2311.07590

  • merc@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    30
    arrow-down
    2
    ·
    11 months ago

    “Humans were stupid and taught a ChatBot how to cheat and lie.”

    No, “cheating” and “lying” imply agency. LLMs are just “spicy autocomplete”. They have no agency. They can’t distinguish between lies and the truth. They can’t “cheat” because they don’t understand rules. It’s just sometimes the auto-generated text happens to be true, other times it happens to be false.

    • gandalf_der_12te@feddit.de
      link
      fedilink
      English
      arrow-up
      3
      arrow-down
      14
      ·
      11 months ago

      I disagree. This is no meaningful talking point. It doesn’t help anyone in practice. Sure, it clears legal questions of responsibility (and I’m not even sure about that one in the future), but apart from that, making an artificial distinction between a human and a looks-and-acts-like-human, provides no real-world value.

      • merc@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        14
        arrow-down
        1
        ·
        11 months ago

        Sure it does, because assigning agency to LLMs is like “the dice are lucky” or “this coin I’m flipping hates me”. LLMs are massively complex and very good at simulating human-generated text. But, there’s no agency there. As soon as people start thinking there’s agency they start thinking that LLMs are “making decisions”, or “being deceptive”. But, it’s just spicy autocomplete. We know exactly how it works, and there’s no thinking involved. There’s no planning. There’s no consciousness. There’s just spitting out the next word based in an insanely deep training data set.

        • gandalf_der_12te@feddit.de
          link
          fedilink
          English
          arrow-up
          4
          arrow-down
          8
          ·
          11 months ago

          I believe that at a certain point, “agency” is an emergent feature. That means that, while all the single bits are well understood probability-wise, the total picture is still more than that.

          It makes sense to me to accept that if it looks like a duck, and it quacks like a duck, then it is a duck, for a lot (but not all) of important purposes.

          • Skates@feddit.nl
            link
            fedilink
            English
            arrow-up
            9
            arrow-down
            1
            ·
            edit-2
            11 months ago

            If I were to send you a video of a duck quacking, would you abandon going to the supermarket in the hope that your computer/phone/whatever you watch it on will now be able to lay eggs?

            Listen. It was made to look like a duck. It was made to quack like a duck. It is not a duck. It is a painting of a duck, with voice features. It won’t fly, it won’t lay eggs, it won’t feel pain, it won’t shit all over the floors. It’s not a damn duck, and pretending it is just because it looks like it and it quacks, is like wanting to marry a fleshlight because it’s really good at sex and never disagrees with you. Sure, go ahead and do it - but don’t goddamn expect it to also give birth to your children and take them to school in the mornings, that’s not it’s purpose.

            Just wait for the iteration of duck that is actually meant to and capable of doing these things. It’ll be pretty cool. But this one ain’t it.

            • gandalf_der_12te@feddit.de
              link
              fedilink
              English
              arrow-up
              1
              ·
              11 months ago

              Edgy comment here but:

              In another thread we were discussing AI-generated CSAM. Thread:

              https://feddit.de/post/6315841

              You would probably agree, then, that such material is not problematic, because even if it looks like CSAM, and it quacks like CSAM, it is not CSAM, therefore we don’t have to take it seriously or regulate it in similar ways that we do regulate actual CSAM, if I continue your logic, no?

              • wildginger@lemmy.myserv.one
                link
                fedilink
                English
                arrow-up
                1
                arrow-down
                1
                ·
                edit-2
                11 months ago

                very very very different, because the AI image is intentionally attempting to realistically imitate an existing, living, human victim, and because hyper realistic child pornographic art is illegal.

                Pedophiles have been making loads of AI child porn. But its legal as long as it doesnt attempt to “look realistic” whatever that means, and isnt trying to look like a real person. A hyper realistic painting of child porn would also be illegal.

                Laws might change in the future, but currently AI child porn slips between the same lines that 2d cartoon child porn does.

          • SmoothIsFast@citizensgaming.com
            link
            fedilink
            English
            arrow-up
            6
            ·
            11 months ago

            Do you understand how they work or not? First I take all human text online. Next, I rank how likely those words come after another. Last write a loop getting the next possible word until the end line character is thought to be most probable. There you go that’s essentially the loop of an LLM. There are design elements that make creating the training data quicker, or the model quicker at picking the next word but at the core this is all they do.

            It makes sense to me to accept that if it looks like a duck, and it quacks like a duck, then it is a duck, for a lot (but not all) of important purposes.

            I.e. the only duck it walks and quacks like is autocomplete, it does not have agency or any other “emergent” features. For something to even have an emergent property, the system needs to have feedback from itself, which an LLM does not.

            • froop@lemmy.world
              link
              fedilink
              English
              arrow-up
              2
              arrow-down
              4
              ·
              11 months ago

              Your description is how pre-llm chatbots work. They were really bad, obviously. It’s overly simplified to the point of dishonesty for llms though.

              Emergent properties don’t require feedback. They just need components of the system to interact to produce properties that the individual components don’t have. The llm model is billions of components interacting in unexpected ways. Emergent properties are literally the only reason llms work at all. So I don’t think it’s absurd to think that the system might have other emergent properties that could be interpreted to be actual understanding.

              • SmoothIsFast@citizensgaming.com
                link
                fedilink
                English
                arrow-up
                4
                ·
                11 months ago

                Your description is how pre-llm chatbots work

                Not really we just parallelized the computing and used other models to filter our training data and tokenize them. Sure the loop looks more complex because of parallelization and tokenizing the words used as inputs and selections, but it doesn’t change what the underlying principles are here.

                Emergent properties don’t require feedback. They just need components of the system to interact to produce properties that the individual components don’t have.

                Yes they need proper interaction, or you know feedback for this to occur. Glad we covered that. Having more items but gating their interaction is not adding more components to the system, it’s creating a new system to follow the old. Which in this case is still just more probability calculations. Sorry, but chaining probability calculations is not gonna somehow make something sentient or aware. For that to happen it needs to be able to influence its internal weighting or training data without external aid, hint these models are deterministic meaning no there is zero feedback or interaction to create Emergent properties in this system.

                Emergent properties are literally the only reason llms work at all.

                No llms work because we massively increased the size and throughput of our probability calculations, allowing increased precision on the predictions, which means they look more intelligible. That’s it. Garbage in garbage out still applies, and making it larger does not mean that this garbage is gonna magically create new control loops in your code, it might increase precision as you have more options to compare and weight against but it does not change the underlying system.

                • froop@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  arrow-down
                  1
                  ·
                  11 months ago

                  The interaction is between nodes in the model. Those are the components that individually have no real characteristics, but when combined into a billion-dimension model, that results in emergent properties. Correctly writing novel code is an emergent property. Correctly solving an ASCII art maze is an emergent property. There is a point where a text predictor, being sufficiently accurate, demonstrates emergent understanding.

                  Your definition emergent property is outright wrong.

          • merc@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            3
            ·
            11 months ago

            “agency” is an emergent feature.

            But, it’s not. It’s something people attribute to the random series of words that are generated, but no agency exists.

            It makes sense to me to accept that if it looks like a duck, and it quacks like a duck, then it is a duck

            Or it’s a video of a duck, which means it’s not a duck. In this case, just because it fools people into thinking there’s consciousness / agency doesn’t mean there actually is any.

      • barsoap@lemm.ee
        link
        fedilink
        English
        arrow-up
        7
        ·
        11 months ago

        The current models that we have, running in inference mode, are t1 systems. Criminal law requires defendants to be able to understand guilt as a prerequisite of having a guilty mind, that’s why asylums for the criminally insane exist because not even all humans can do that. You’re trying to apply that standard to an overcomplicated thermostat.

      • Karyoplasma@discuss.tchncs.de
        link
        fedilink
        English
        arrow-up
        5
        ·
        edit-2
        11 months ago

        If your parrot or budgie picks up some of the words you frequently use and reproduces them in a wrong context, would you consider your pet lying? Because that’s what ChatGPT basically is, a digital parrot.

      • wildginger@lemmy.myserv.one
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        2
        ·
        11 months ago

        Chaptgpt is a very very very very large algorithm that uses language instead of numbers, and runs off of patterns found within the data set that is plugged into the algorithm.

        Theres a gulf of meaning between distinguishing between a calculator that uses words instead of numbers and a person.