I have this sample program which when compiled with fstack-protector-all gives a stack smashing.
#include <stdio.h>
#include <stdint.h>
int func(int* value)
{
uint8_t port = 1;
*value = port; //Canary value changes at this point when seen in GDB
return 1;
}
int main()
{
uint16_t index = 0;
int ret = func((int*)&index);
}
I don't understand what is wrong with the line. Is any typecasting required?
It's because the size of int and the size of int16_t are different. The size of int is (usually) 32 bits (four bytes) while int16_t is 16 bits (two bytes).
So when you write an int to a int16_t variable you write two bytes too many, and leads to undefined behavior (and in this case will "smash" the stack).
The problem is more specifically because you call the function with a pointer to index which is a 16-bit variable, but the function expects (and uses its argument) as a 32-bit variable. You should not do the cast there in the call, as that hides the problem but doesn't solve it. It doesn't matter that you only write an 8-bit value to the dereference pointer inside the function, the destination is still a 32-bit variable and the compiler will convert the 8-bit value to a 32-bit value before writing to memory.
int16_t to the function, but the function expects a pointer to int. There's the problem. I assume you added the typecast in the call just because you got warnings about that? Warnings are often a sign of you doing something you should not be doing, hiding it with a typecast just hides the problem but doesn't solve it.
value is an int *, the uint8_t is converted to int before being stored.Since the type of index is uint16_t, only 16 bits are allocated for it. By casting the address of index to int*, you are pretending you have access to more than 16 bits -- 32 bits in most cases.
In
*value = port;
you are trying to set the value in those bits that haven't been allocated. Since unauthorized memory gets used in that line, any thing can happen after that.
gcc -fstack-protector yourcode.c. And it runs fine.