2

I seem to be observing conflicting evidence regarding how the tty and the shell share responsibility for displaying user input.

In the following interactive session, user input is no longer echoed to the terminal, suggesting that the tty performs the echo.

$ stty -echo
# I typed ls (not displayed) and we see the output of ls only:
Bash.log File1    TTY.log

On the other hand, if we use expect to log exactly what is being emitted by bash vs by the tty, we can see that bash seems to be sending the user's input ls to the tty.

tty-echo-investigate.exp

#!/usr/bin/env expect

spawn bash

log_user 0
set log_tty_fd [open "TTY.log" "w"]
set log_bash_fd [open "Bash.log" "w"]

expect {
  -i $spawn_id "?" {
    send_user -- [set expect_out(0,string)]
    puts -nonewline $log_bash_fd [set expect_out(0,string)]
    exp_continue
  }
  -i $tty_spawn_id "?" {
    send -- [set expect_out(0,string)]
    puts -nonewline $log_tty_fd [set expect_out(0,string)]
    exp_continue
  }
}

Bash.log

[?2004h~/Play4/X ls
[?2004l
Bash.log File1    TTY.log
[?2004h~/Play4/X exit
[?2004l
exit

TTY.log

ls
exit

When user input is echoed by the tty vs bash, and how do they coordinate so we (usually) see only one copy of user input if they both sometimes do it?

Improve this question
9
  • 1
    There's no stty -echo in your Expect script. Commented Nov 13 at 23:26
  • Fair point. I'll try it with that when I get back to laptop. Commented Nov 14 at 3:59
  • @Barmar When I add stty -echo in my expect script, a very interesting phenomenon happens. As I am typing, there is no echo, but when I press enter, the text that would have appeared if echo were on suddenly appears, along with the output. asciinema.org/a/NlWwUOhet5vNYwRY2OZ9un35S Commented Nov 14 at 6:25
  • Curiously, if I do stty -echo inside a bash -i outside of expect, I still never see the characters I typed which is further evidence for bash not echoing in this scenario. Commented Nov 14 at 6:35
  • Also I just noticed without stty -echo, inside the expect script I actually do see ls in my terminal twice -- once from the tty's echo and once from bash's echo. This still leaves open the question of how bash and the tty normally coordinate to not double-echo the user's input. How does bash know it's not supposed to echo user input in the normal case and how does that break down in the expect scenario? Commented Nov 14 at 6:42

4 Answers 4

Reset to default
5

Echoing can be done either by the tty driver or the interactive application (e.g. a shell).

Normal echoing is done in the tty driver. But this provides only very simple editing: deleting the last character and erasing the entire input line. This is what you get when the terminal is in "cooked" mode and you have stty echo enabled, which is the default for login terminals. When you're entering a password, echo is turned off, but cooked mode still allows simple editing, you just can't see what's happening.

But some applications want to provide more advanced editing, usually patterned after the Emacs or vi editors -- you can move around in the line with arrow keys, recall previous inputs, etc. This is usually found in interactive applications that operate using a command prompt, and interactive shells are the most common of these.

These applications put the tty driver in "raw" mode, and process the inputs by themselves character by character. They turn off the tty driver's echoing, and do it themselves. Very often they make use of the popular readline library for consistency. This library allows for user customization of editing and shortcut keys using a .inputrc configuration file.

So either the driver does the echoing or the application does, but they shouldn't both do it. That's why you don't see doubled echoing. Where you sometimes run into this is in Emacs shell buffers. Emacs usually puts the tty used for these buffers into -echo mode, but some applications turn echo back on unconditionally. When this happens, you type a command into the shell buffer, and when you press Enter you see the command repeated.

As far as Expect is concerned, I don't think it should make a difference. It just sees what it receives on the master end of the pty, which is the same as what a human user sees on the terminal as they're typing. It can't distinguish output generated automatically by the tty driver from output written by the process at the other end of the pty.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

This answer gives very helpful context, but it does not address the question of whether the user echo is coming from the tty or from bash in each of my scenarios, and how they negotiate over who should be doing the echoing at any given time.
Also, if you look carefully at my expect script, you'll observe that I've tried pretty hard to separate what the process is outputting from what the user (combined with tty) is outputting by putting their output into separate files. Since I have log_user 0, in theory I should only see what bash is printing, which makes it extra confusing that stty -echo impacts what I see in the output.
I wrote my own answer based on investigation starting from what I learned from this answer. Thank you for this answer!
1

In a regular interactive shell session, it’s the kernel’s TTY line, not the shell, that echoes each key you type. Whether or not your keystrokes appear on the screen depends entirely on the ECHO flag, which you can control with stty echo or stty -echo. When you turn echo off, the kernel stops sending typed characters back to the terminal. Bash still receives the input and runs commands, but nothing is shown as you type.

Bash itself doesn’t echo your keystrokes. Instead, it relies on the TTY for that. From what I understand its the readline library and the print are prompts and full command lines after you press Enter.

To avoid double echoing, programs choose one approach or the other. A typical shell leaves ECHO on and lets the TTY handle character-by-character input display. Programs like full screen editors or password prompts disable ECHO and take full control of how input is shown or hidden. So therefore between Bash and TTY, is why you normally see exactly one copy of your input, and why turning off ECHO hides it completely.

Improve this answer

1 Comment

Interactive bash normally uses the readline library so it can provide advanced input editing. It doesn't rely on tty echoing.
1

Bash never echos stdin unless instructed to do so it's the tty echoing what's typed in the keyboard.

echo "echo 'second echo'" | bash
second echo

echo "echo 'second echo'" | bash -c 'cat /dev/stdin'
echo 'second echo'
  • Copy the string echo 'second echo' into the clipboard
  • Open 2 consoles (konsole app in my case).
  • In the second find the PID of the first ps -alf -C konsole
  • Attach strace to it tracing only stdin read events

strace -e trace=read -e read=0 -s 100 -p 17006 -o tty.log

  • Paste the clipboard into the first console and do not hit enter
  • Terminate strace on second with ctrl + C
  • Kill first one kill 17006
  • Filter tty.log

cat tty.log


read(4, "\1\0\0\0\0\0\0\0", 8)          = 8
<more read(4 ... lines>
read(18, "echo 'second echo'", 18)      = 18
read(4, "\1\0\0\0\0\0\0\0", 8)          = 8
<more read(4 ... lines>
read(4, "\1\0\0\0\0\0\0\0", 8)          = 8

This command will show nothing on console

echo "cat /dev/stdin" | bash

Following @barmar's comment; at least here bash needs -i option to be interactive.

Interactive bash confirms Barmar's comment: interactive mode spits out stdin that is shown before stty -echo gets executed

lmc@lmci7lnx:~> echo "stty -echo; echo 'first echo'" | bash -i
lmc@lmci7lnx:~> stty -echo; echo 'first echo'
stty: 'standard input': Inappropriate ioctl for device
first echo

Non-interactive mode does not

lmc@lmci7lnx:~> echo "stty -echo; echo 'first echo'" | bash
stty: 'standard input': Inappropriate ioctl for device
first echo
Improve this answer

7 Comments

Your example is a non-interactive bash. Interactive bash does normally echo input, using the readline library.
how do you start an interactive bash without a console?
Using Expect, as in the OP.
it should be spawn bash -i, right?
The option isn't needed. Bash defaults to interactive mode when its input is a tty, and Expect uses a pseudo-tty.
Does not work like that here. Added examples to answer.
Your examples use a pipe, not Expect, so bash's stdin is a pipe, not a pty.
1

After further investigation, it appears that bash (via readline) rather than the tty is doing the echo. The reason I do not see the characters I type when I do stty -echo is because readline is reading and respecting the tty setting when deciding whether to echo user input.

  • The main readline function calls rl_prep_term_function before performing the actual read and rl_deprep_term_function afterwards.
  • These function pointers are mapped to rl_prep_terminal and rl_deprep_terminal in rltty.c
  • rl_prep_terminal calls prepare_terminal_settings which runs _rl_echoing_p = (oldtio.sgttyb.sg_flags & ECHO); and also turns off echo in the tty with tiop->c_lflag &= ~(ICANON | ECHO);
  • The documentation for this variable states the following:
/* Non-zero means echo characters as they are read.  Defaults to no echo;
   set to 1 if there is a controlling terminal, we can get its attributes,
   and the attributes include `echo'.  Look at rltty.c:prepare_terminal_settings
   for the code that sets it. */
int _rl_echoing_p = 0;
  • Furthermore, tty echo gets restored after each line is read, which is why it is very tricky to see what is really happening by running commands like stty -a from bash directly.

We can also observe the readline behavior using the following expect script, which will dump the tty state of bash's tty without invoking it via bash itself.

#!/usr/bin/env expect

set timeout -1
spawn bash


log_user 0
set log_tty_fd [open "TTY.log" "w"]
set log_bash_fd [open "Bash.log" "w"]

expect {
  -i $spawn_id "?" {
    send_user -- [set expect_out(0,string)]
    puts -nonewline $log_bash_fd [set expect_out(0,string)]
    exp_continue
  }
  -i $tty_spawn_id "dump" {
    puts [ stty -a < $spawn_out(slave,name) ]
    exp_continue
  }
  -i $tty_spawn_id "?" {
    send -- [set expect_out(0,string)]
    puts -nonewline $log_tty_fd [set expect_out(0,string)]
    exp_continue
  }
}

When invoking this, we can see the true state of the tty when readline is trying to read, which is different from what stty -a returns when invoked on the actual shell:

$ ./tty-echo-investigate.exp
spawn bash
$ stty -a
stty -a
speed 9600 baud; 48 rows; 196 columns;
lflags: icanon isig iexten echo echoe -echok echoke -echonl echoctl
    -echoprt -altwerase -noflsh -tostop -flusho pendin -nokerninfo
    -extproc
iflags: -istrip icrnl -inlcr -igncr ixon -ixoff ixany imaxbel -iutf8
    -ignbrk brkint -inpck -ignpar -parmrk
oflags: opost onlcr -oxtabs -onocr -onlret
cflags: cread cs8 -parenb -parodd hupcl -clocal -cstopb -crtscts -dsrflow
    -dtrflow -mdmbuf
cchars: discard = ^O; dsusp = ^Y; eof = ^D; eol = <undef>;
    eol2 = <undef>; erase = ^?; intr = ^C; kill = ^U; lnext = ^V;
    min = 1; quit = ^\; reprint = ^R; start = ^Q; status = ^T;
    stop = ^S; susp = ^Z; time = 0; werase = ^W;
$ dump
speed 9600 baud; 48 rows; 196 columns;
lflags: -icanon isig iexten -echo echoe -echok echoke -echonl echoctl
    -echoprt -altwerase -noflsh -tostop -flusho -pendin -nokerninfo
    -extproc
iflags: -istrip -icrnl -inlcr -igncr ixon -ixoff ixany imaxbel -iutf8
    -ignbrk brkint -inpck -ignpar -parmrk
oflags: opost onlcr -oxtabs -onocr -onlret
cflags: cread cs8 -parenb -parodd hupcl -clocal -cstopb -crtscts -dsrflow
    -dtrflow -mdmbuf
cchars: discard = <undef>; dsusp = <undef>; eof = ^D; eol = <undef>;
    eol2 = <undef>; erase = ^?; intr = ^C; kill = ^U;
    lnext = <undef>; min = 1; quit = ^\; reprint = ^R; start = ^Q;
    status = ^T; stop = ^S; susp = ^Z; time = 0; werase = ^W;
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.