0

We are printing Linux device events in one program than trying to read and interpret the data by piping the input of that program into another.

First program's relevant code:

struct input_event event1;
...
// open correct event file and read its values into the input event struct
... 
printf("%d  %zu.%-6zu  %d  %3d  %9d\n", 1, event1.time.tv_sec, event1.time.tv_usec, event1.type, event1.code, event1.value);`

Output of first program in the terminal, looks correct.

1  1.638837  3   16     -24402
1  1.638837  3   17      23791
1  1.646835  3   16     -25588
1  1.646835  3   17      23753
1  1.654835  3   16     -26378
1  1.654835  3   17      21386
1  1.666836  3   16     -27498
1  1.666836  3   17      20308
1  1.674837  3   16     -29342
1  1.674837  3   17      18248
1  1.682836  3   16          0

Programs piped together in bash: ./prog1 | ./prog2

Second program's relevant code:

int event_num, type, code, value;
double time;
char arr[100];

// this scanf returns 0 on this and following printf is garbage
scanf("%d  %lf  %d  %d  %d", &event_num, &time, &type, &code, &value);

// alternately, this scanf returns 5 as expected and printf works,
//   but my first value is stored in a character array
//   rather than an integer variable as wanted
scanf("%s %lf %d %d %d", arr, &time, &type, &code, &value);

It seems as though I can read the stdin as an array of strings no problem and I can store most of the input as expected, but I cannot seem to get the first value to be stored as an integer.

Thanks in advance, and let me know if you have any further questions or clarification needed.

I tried different format specifiers including %*s and the %[^0-9]%d and getchar() hoping to remove any junk at the beginning of the string that might be messing stuff up. I have tried changing the type of the variable and the format specifier, but I can never read the first value as anything other than a string.

17
  • Can you minimally reproduce this? Commented Feb 4, 2023 at 8:57
  • 1
    Use fgets to read the whole line and the print the string to see what the real input is.. Perhaps print the decimal value of each character to find "non-printable" characters Commented Feb 4, 2023 at 9:00
  • 1
    @Fe2O3 sorry, the code is on a remote server and I tried to copy it as faithfully as I could, I have now double checked and added the newline character and character widths to make it an exact match to the code. Sorry I dont have copy and paste set up on the server. Commented Feb 4, 2023 at 9:03
  • Instead of piping from one black box to another. capture the output of the first program and add that to this question. Otherwise, it's pure guesswork... (re: '\n'... no worries... this happens... :-) Commented Feb 4, 2023 at 9:03
  • 2
    My conclusion is that the problem is in some part of the code that you removed (aka didn't post). I'm voting to close as the question can't be answered with the information provided. Commented Feb 4, 2023 at 9:42

2 Answers 2

2

There is no such thing as relevant code... the problem might lie in lines that you did not post. Yet there are multiple issues with the posted fragments:

  • the return value of scanf() is not tested so one cannot detect conversion errors (actually you document the return values, but I suspect you get them from the debugger).

  • it is very difficult to recover from conversion errors when parsing with scanf directly. One should read the input one line at a time with fgets(), parse it with sscanf() and test the return value to detect errors and report the errors with meaningful messages.

  • prog1 use %zu.%-6zu to output event1.time.tv_sec and event1.time.tv_usec. This poses 2 different problems:

    • the types of tv_sec and tv_usec are not size_t, they are system dependent types time_t and suseconds_t. You should use %lu or %llu and convert the fields appropriately. As coded, the program may have undefined behavior.
    • using %zu.%-6zu to output a decimal value to be parsed with %lf is incorrect because the microseconds are not always output with 6 digits and smaller microsecond numbers will produce fewer decimals that will be parsed with an incorrect value by %lf. You should use %lu.%.6lu or %lu.%06lu instead. As coded, 1 second and 5 microseconds is printed as 1.5 and read back as 1.5 instead of 1.000005.

Here is a conversion loop that should make the parsing errors easier to analyze:

#include <stdio.h>

int main() {
    int event_num, type, code, value;
    double time;
    char buf[100];
    while (fgets(buf, sizeof buf, stdin)) {
        int res = sscanf(buf, "%d %lf %d %d %d",
                         &event_num, &time, &type, &code, &value);
        if (res != 5) {
            fprintf(stderr, "parsing error, returns %d: %s\n", res, buf);
            fprintf(stderr, "  initial bytes: %02hhx %02hhx %02hhx\n",
                    buf[0], buf[1], buf[2]);
        } else {
            /* handle the parsed values */
        }
    }
    return 0;
}

You could also check for potentially invalid trailing data at the end of the line with an extra %c or %n conversion.

Here is a wild guess: you might want to check the output stream for an initial byte sequence EF BB BF using a hexdump utility. This sequence is a UTF-8 encoded Byte Order Mark that may have been inserted at the beginning of the output by prog1 depending on the locale settings.

Sign up to request clarification or add additional context in comments.

10 Comments

Thanks for the response! A couple of notes and clarifications to address your response. I mentioned in response to other comments that I do test the output of scanf and note that when reading the first value as a string I get 5 which correlates correctly to the amount of things I am scanning in, but when I scanf using %d it is zero because it fails on the first value, which is how I know that it is the issue. As far as the relevant code, I only say that as I know this forum is picky about bloated questions and code, and that really is the only part of the code that outputs to the terminal.
The '-' sign in the printf just left aligns the text and does not impact how it evaluates the expression and the width number is also again not seemingly an issue because it just sets a lower limit on how much space the value will take up on the screen, again, works and is not causing any problems with scanf.
@ClaySmith: sorry for coming across as picky and patronizing, but most posts with code fragments tend to miss crucial information. Regarding fgets() and testing the return value of scanf(), my proposed approach lets the program produce useful error output: if a line fails to convert with a %d conversion for the first field, you will get a chance to see its contents.
@ClaySmith: The %zu works because it is always going to be greater than or equal to %lu: You cannot rely on this (indeed it was not the case on 16-bit systems) and it is irrelevant because if size_t is larger than unsigned long (as in the case in Windows 64-bit systems) values might be passed to printf in an incompatible way. Discrepancy between the format spec and the argument type causes undefined behavior. It may seem to work on your system, but this might be a coincidence and another instance might not, whether for the same target or another. Just use %lu and cast the arguments.
@ClaySmith: %lu is a good choice because type unsigned long has at least 32 value bits so it can handle the range of both fields, but you should still add casts to ensure the values are passed as the expected type: printf("%d %lu.%6lu %d %3d %9d\n", 1, (unsigned long)event1.time.tv_sec, (unsigned long)event1.time.tv_usec, event1.type, event1.code, event1.value);
|
1

After a lot of debugging the issue was in part of the code not listed. I was using the system() function as: system("clear") which was effecting my stdout buffer only on the first execution, which is why fgets worked as it essentially throws that garbage out so my scanf can proceed as expected. Other uses of the system() function do not effect stdout such as system("reset"), but apparently that one does.

3 Comments

Your explanation is surprising. calling system("clear:); might indeed have side effects, but unlikely to affect the input stream. Would you mind updating the question and post the source to the program with the system("clear") call that causes the problem?
Sorry I rarely come on here. I did confirm that system("clear") was the issue. This is best explained because what it is essentially doing is this: printf("\033[2J");. I was printing in one program and cleared the screen and piping it into another, well when reading it from the other program (the one reading in stdout from the pipe), the first thing it would read is the "\033[2J" that the system() function call started the program with. I still feel like it should have been made more clear in the docs. lol This was a hard one to figure out.
you could try and send the escape sequence directly to the process group terminal device with this: { FILE *fp = fopen("/dev/tty", "w"); if (fp) { fprintf(fp, "\033[2J"); fclose(fp); }

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.