-4

I wanted to know the extent to which ChatGPT could generate assembly language, so I gave it the following prompt:

Write an optimized ARM assembly code that bubble sorts an array.

I will not include the full output as it's rather boring and standard, but I will provide the snippets below. Ellipses indicate lines I've cut out. All comments are mine

.data
array: .word 9, 8, 7, 6, 5, 4, 3, 2, 1, 0
arrayLength = 10
...

outer_loop:
    ...

inner_loop:
    add r3, r0, r2                @@r0=0 and r2=0 on first iteration
    ldr r4, [array, r3, lsl #2]
    add r3, r3, #4
    ldr r5, [array, r3, lsl #2]

    cmp r4, r5                    @@N not set, so bgt branches on first iter
    bgt swap
    ...

swap:
    str r5, [array, r3, lsl #2]
    subs r3, r3, #4
    str r4, [array, r3, lsl #2]
    b inner_loop

...

I am trying to get into MCUs currently and have been working with. This is rather easy to understand, but if we just focus on the first iteration, it seems that we're loading 9 into r4 and 8 into r5, then putting these numbers right back into the same place in the array from which we retrieved them. At length, you can sort of see what happens by the following sequence:

  • array + r3 points to the first value in the array.
  • r4 holds the value 9.
  • r3 increments by 4.
  • array + r3 points to the second value in the array.
  • r5 holds the value 8.
  • branch to swap block
  • we store the value in r5, which is 8, in the place array + r3 points, which is the second array slot.
  • r3 decrements by 4.
  • array + r3 points to the first value in the array.
  • we store the value r4, which is 9, in the place array + r3 points, which is the first slot.

So my questions are: am I somehow misunderstanding the opcodes? If so, why does my intuition here fail regarding the sequence I expect to execute? Is the bot just wrong? Also, the subs in the swap block may set a flag if we go negative with our address, but ChatGPT doesn't do anything with this fact. So would a simple sub be better?

I also had a question about the logical left shift. I first assumed this had to be due to endianness but realized that doesn't quite explain it. Why is the array stored at a weird offset like that? I assume I'm missing context where this sort of stuff is normal, but I have yet to encounter this in my experience.

ChatGPT insists this sorts the array, but always with the context of what a bubble sort is. I don't know if it is actually running the code against the actual input data I feed it or else if it is just doing a sort and then telling me it used a bubble sort. For instance, I gave it the following prompt

Imagine you have the array [10,5,4,1] stored in memory and used the above code to sort this array. After running the program, what would the resultant array be?

and received the following explanation

After running the above ARM assembly code on the array [10, 5, 4, 1], the resultant sorted array would be [1, 4, 5, 10].

The bubble sort algorithm works by repeatedly swapping adjacent elements that are in the wrong order until the entire array is sorted in ascending order. In each iteration of the outer loop, the length of the unsorted portion of the array is decremented, and in each iteration of the inner loop, adjacent elements are compared and swapped if necessary. This process continues until the entire array is sorted.

The tense here is important: "the resultant array would be..."

13
  • 3
    Yes, GPT-Chat is an excellent AI liar. It produces confident sounding rationals and gives no idea of it's confidence in the answer. This is why it is banned from providing answers on stackoverflow. It is easy to modify the swap to make the algorithm work. Commented Feb 10, 2023 at 20:52
  • 1
    It's unlikely to even be valid to use array as the base of an addressing mode; ARM only has 12 bit immediates or something, and the array address isn't going to be in the low 4 KiB of virtual address space, if [imm12 + reg<<constant] is even a valid ARM addressing mode. If it was, a normal person would use [array+4 + r3*4] instead of an add instruction to change the register. (x86 addressing mode syntax since ARM can't normally use a symbol address.) Commented Feb 10, 2023 at 20:52
  • 1
    @PeterCordes I assumed that array was a proxy for a register allocated for indexing; this would work. ARM doesn't support absolute addressing as a single opcode (except very limited sets which I have never seen used in practice). ... but the main point is GPT-Chat will NEVER give answers that aren't deserving of some critical thought. I think the idea sketched out can be used, but is not complete. Commented Feb 10, 2023 at 20:57
  • 6
    @artlessnoise: no, questions about ChatGPT output are not banned. Using ChatGPT to create questions, answers or comments is banned; but that's not the same thing. This question was written by a human, asking about the output of ChatGPT. Commented Mar 1, 2023 at 16:31
  • 2
    @blnlb I considered writing a wiki answer on why ChapGPT does not work well with assembler. Note, there is a huge academic and commercial effort to produce 'AI' that can generate assembler. They are called compiler, automata theory and predicate calculus, etc. Perhaps this is not artificial intelligence. Then any assembler questions on using ChatGPT can be closed as a duplicate of this one. Commented Mar 2, 2023 at 14:39

1 Answer 1

0

I know nothing about "ARM assembly code", but I have learned a bit about ChatGPT, much of that from Stack Overflow and Artificial Intelligence SE.

First, Stack Overflow banned posting generated text content, specially answers due to the high rate of misleading answers. See Temporary policy: Generative AI (e.g., ChatGPT) is banned.

Second, while "using" ChatGPT is as easy as typing in a chat application like WhatsApp, the quality of the response depends on several factors being one of them the conversation. On March 1st, 2023 the ChatGPT API was launched, and thanks to this we know that the ChatGPT response depends on a conversation composed by prompts from three roles, the system, the user—in this case you—and assistant, the previous ChatGPT responses in the conversation, but in this case it looks that there wasn't any.

In order to get the most from ChatGPT

  1. Prepare yourself about what you will talk with ChatGPT. In this case, you should learn the most that you can about ARM assembly code.
  2. Prepare yourself about prompt engineering
    • This is key in order to know how to you should conduct your interaction with ChatGPT.
    • Nowadays "we" know that is unrealistic to expect to get reliable code from a simple statement like the one you used.
  3. Prepare yourself to use ChatGPT
  4. Verify the responses received from ChatGPT
    • ChatGPT builds the response using complex algorithms to put together tokens, but it's not able to evaluate it by itself.
    • Be aware that ChatGPT might hallucinate. See Hallucination.
  5. Consider that you might edit your inputs and regenerate the intermediate responses.
    • In case that the response is not satisfactory, you might not have to start the whole conversation from scratch.
  6. Consider that you might ask ChatGPT do do something with the previous prompts, like change a variable name, changing the order of the arguments, among many things.

References

Sign up to request clarification or add additional context in comments.

2 Comments

I wouldn't be surprised if ChatGPT is just not good at assembler in general, perhaps because using the same register name for different things isn't something its language model can do well? That's a total guess based on one thing that's different in asm vs. most other languages. @artlessnoise commented that ChatGPT generally isn't good at asm, so even with expert prompting it might be hard to get correct asm code for anything non-trivial.
@PeterCordes I will not be surprised either. It will be great to have the point of view of somone having expertise at assembler and prompting.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.