I am looking for a way to display and interact with long running shell scripts in a separate buffer. While using eshell works, I'd like to automate most shell tasks using elisp.

The best way I came up with is (example for git pull):

(let ((new-buffer (get-buffer-create "*script*")))
     (switch-to-buffer-other-window new-buffer)
     (eshell-command "git pull" t)))

While this displays procedural output in a new buffer, emacs locks until the process finishes and I can't interact with git if it prompts for information.

(start-process "script" new-buffer "git" "pull") runs asynchronously, but does not accept interactive input either.

The git pull command is just an example here.

Update:

I think I got everything I wanted using make-comint-in-buffer. I got correct ansi-colors and can interact with the process if it reads stdin.

Just the gist:

(let* ((script-proc-buffer 
           (apply 'make-comint-in-buffer "script" new-buffer "git" nil '("pull")))
       (script-proc (get-buffer-process script-proc-buffer)))

    (set-process-sentinel script-proc 'special-mode-sentinel))

Where special-mode-sentinel is a defun that switches to special-mode when the process has finished, allowing me to quit the buffer using q.

It's not pretty, but it works...