As part of my musings with irctk (which is not yet usable as of this writing), I have been looking for ways to run commands without bufferisation. This is annoyingly hard, because what should make you save time (one-line IRC bots) is actually a mess of cases where things fail to work because buffering happens somewhere in some program in the pipe.

If the program you're trying to pipe is one that you wrote, there are often (more or less obscure) ways to disable buffering or flush buffers by hand. Think about the "-u" option in Python 2 or the (non-POSIX) "fflush" command in awk. If you didn't write the program, maybe it already has an option to do what you want (GNU grep has "--lines-buffered") or maybe not (I'm not aware of such a trick for cut, for instance).

In the last case, you have to use an external tool to change the behaviour of the program. In the relevant StackOverflow question, the suggested tools are a simple but limited program called unbuffer:

unbuffer myprogram

or a complicated socat call:

socat EXEC:myprogram,pty,ctty,echo=0 STDIO

It turns out that there is a third option: the stdbuf program in (recent versions of) the GNU Coreutils does the job both powerfully and with a pleasant syntax. Here is my current recipe to get rid of buffer woes:

stdbuf -i0 -o0 -e0 myprogram

If you want to really know how all this mess works, here is a nice explanation. If you don't care, just remember that (1) buffering can cause data to get stuck in pipes and that (2) stdbuf is the program you want to use in this case.