STM32 Shell: Why does DMA not callback (Key suspect)

Question

I'm trying to integrate a shell into an STM32 project I'm working on. I used this library and followed a tutorial to get it working.

In the current implementation, the shell_get_char() function arms the HAL_UART_Receive_IT to receive a single byte. The HAL_UART_RxCpltCallback function then calls shell_in() to process the input. This approach works fine, but it has a limitation: it cannot detect arrow keys, which consist of three bytes.

To address this, I modified the code to use a ring buffer. This method works provided that shell_in() is called from the callback function (as it is in my current repo).

[edit]: By works I mean it reads single bytes, I still think it will be too slow to handle arrow keys this way.

Alternatively, I rewrote shell_get_byte() to arm the HAL interrupt and pass the ring buffer data to shell_in(). I also updated HAL_UART_RxCpltCallback() to store the received character in the ring buffer and immediately rearm the HAL interrupt.

void shell_get_byte(char *c){
    //Check that huart1 is ready to accept data
    if (HAL_UART_GetState(&huart1) == HAL_UART_STATE_READY){
        //Enable the UART interrupt
        HAL_UART_Receive_IT(&huart1, (uint8_t *) c, 1);
    }
    if (rgbuffer_empty_c(&rgbuffer) == 0){
        //HAL_UART_AbortReceive_IT(&huart1);
        shell_in(rgbuffer_read_c(&rgbuffer));
    }
}

void HAL_UART_RxCpltCallback(UART_HandleTypeDef *huart){
    //Verify the uart used
    if(huart->Instance == USART1){
        //Write to ring buffer
        rgbuffer_write_c(&rgbuffer, shell_state.c);
        HAL_UART_Receive_IT(&huart1, (uint8_t *) &shell_state.c, 1);
    }
}

With this rewrite, I observe the following behavior:

If I press only the Enter key, I get the expected behavior—a new line followed by the "/STM32" prompt.
If I press any key, it is echoed to the terminal as expected. However, if I press the Enter key, the system freezes.

When I uncomment the following line in the shell_in() function in shell.c:

while (shell_state.busy != 0);  // Don't echo unless the interface is ready (this may be unnecessary here)

I observe different behavior. Now, when I press any key, that key is echoed, and then the system freezes. Pressing just the Enter key still behaves as expected. This leads me to believe that shell_state.busy is not being cleared properly in HAL_UART_TxCpltCallback() After the shell_out() DMA call.

I have limited troubleshooting tools, but I was able to toggle a GPIO pin every time shell_out() is called and another in HAL_UART_TxCpltCallback(). This means the LED states should match if shell_state.busy is being set and reset properly—which they are. I have no idea where shell_state.busy is being set without being reset.

[edit]: Added the Shell_out and HAL_UART_TxCpltCallback

void shell_out(char* buff, int length){
    HAL_UART_Transmit_DMA(&huart1, (uint8_t *) buff, length);
    //HAL_UART_Transmit_IT(&huart1, (uint8_t *) buff , length);
    shell_state.busy = 1;   //DMA Transfer in progress
}

void HAL_UART_TxCpltCallback(UART_HandleTypeDef *huart){
    //Verify which USART is being used
    if(huart->Instance == USART1){
        //Transfer complete
        shell_state.busy = 0;
    }
}

The only real difference is that shell_in() is being called from the shell_get_char() function instead of from the HAL_UART_RxCpltCallback() function, but I can't figure out why that difference would matter.

Instead of copying all my code here, This is a link to the repo. All my source code is in the Core directly and the Shell code is in the STM_SHELL directory

Not an answer, but using the HAL when you are trying to process each character individually in an ISR is just really horrible. Just enable the receive interrupt (by setting UART_IT_RXNE) and write your own ISR, and ditch the HAL. — pmacfarlane
– pmacfarlane, Commented Mar 2 at 22:42
I agree that's why I'm just storing the character with the callback, then processing it during the idle state. — lpaulson
– lpaulson, Commented Mar 2 at 22:44
Posting a link to an external site which requires people to create an account is not very helpful. You should put all the relevant code in your question. Anyway, I suspect your problems might go away (or at least become easier to diagnose) if you removed all reliance on the HAL. Calling a complicated function like HAL_UART_Receive_IT() for every character, and also having to go through the entire HAL callback mechanism for every character, is not fun. You should go look at those functions if you haven't already. — pmacfarlane
– pmacfarlane, Commented Mar 2 at 23:02
The question's title mentions DMA, but none of the body text or code shown uses DMA. — pmacfarlane
– pmacfarlane, Commented Mar 2 at 23:07
I didn't realize gitlab didn't allow public browsing, thank I'll fix that. Yes there is overhead with HAL. The issue I'm facing is not related to trying to process more than one byte at a time. But yes I' also investigating using ISR instead — lpaulson
– lpaulson, Commented Mar 2 at 23:13

korandukar · Accepted Answer · 2025-03-18 05:05:36Z

0

I'm planning on using the same library for another one of my STM32 projects. If you are using the following library from Nefastor: https://nefastor.com/microcontrollers/stm32/libraries/stm-shell/

The author states that they will answer questions, have you tried leaving them a comment?

edited Mar 18 at 5:05

answered Mar 18 at 5:04

korandukar

11 bronze badge

Sign up to request clarification or add additional context in comments.

1 Comment

Gerhard Mar 18 at 7:23

Please do not answer with a comment/question. Understandably, your rep is too low to comment, but that still does not mean answers should be used to make comments as an alternative.

Collectives™ on Stack Overflow

STM32 Shell: Why does DMA not callback (Key suspect)

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related