Home

Understand your process stdin/stdout: pipes, inter-process communication, GO

Youcef Guichi · · #pipes#linux#go#system-programming

It started from the place where I needed to pass data from a parent process to a child process on my journey of creating my own container runtime. for today's blog we gonna focus mainly on the stdin stdout of a process and how you can use pipes to exchange data.

A familiar scenario#

you have two processes running A and B, and you want the output of A to be read by B.

How do we usually do that? a pipe maybe which in terminal is denoted by this symbol |, considering A is this command ( printf 'hello'; sleep 2; printf ' world\n'; ) and B is cat.

We would do ( printf 'hello'; sleep 2; printf ' world\n'; ) | cat right?, now here is the question, what would be the output?

  1. hello world
  2. hello then after two seconds it print world

If you chose the second option I am assuming you know what's happening under. However if you are team 1, and wondering let's try to unpack it together.

Pipes#

When process A writes to it's stdout, in our case it writes two time hello then world.

you may expected that the process finishes writing all these then somehow they get redirected to cat and it print them all.

But what happened exactly was that the process A stdout out is being poured to the write end of the pipe and the read end of the pipe is being read by the process B. So if anything get poured in the beginning of the pipe it will show at the end of the pipe and the process B can read it. it is a FIFO, first in first out.

So what is a pipe exactly? In linux vocab it is a kernel object. But what is it exactly? It is just a memory buffer, with a very limited capacity, the default is around 1048576 bytes = 1 MiB And we can actually see it.

✗ cat /proc/sys/fs/pipe-max-size
1048576

To conclude, the pipe is a memory buffer, where a process can writes to it and another process can read from it, just like a queue. One thing to be aware of is that the pipe is independent from the process lifecycle. Meaning if a process A write to the the pipe and it dies, the pipe is not bound to it it still exist and another process can still read from it. As the pipe leaves in the kernel space memory and managed by the kernel.

Let's see how we can achieve this scenario in go.

func main() {
    w, r, _ := os.Pipe() // Instruct the kernel to create a new pipe
}

The above snippet will create a pipe and return two file descriptors. w is the write end of the pipe and r is the read end of the pipe.

// process_inout.go
package main

import (
	"bufio"
	"fmt"
	"log"
	"os"
	"os/exec"
)

func main() {
	r, w, err := os.Pipe() // Instruct the kernel to create a new pipe

	if err != nil {
		log.Fatal(err)
	}

	os.Stdin = nil
	os.Stdout = w // redirect the output to the write end of the pipe

	if len(os.Args) > 1 && os.Args[1] == "child" {
		os.Stdin = os.NewFile(uintptr(3), "parent_pid_pipe") // point stdin to the read end of the pipe

		defer r.Close()
		reader := bufio.NewReader(os.Stdin)

		line, _ := reader.ReadString('\n')
		log.Printf("[PID: %v] The main process is saying : %v \n", os.Getpid(), line)
	}

	fmt.Printf("HOLA! [PID: %v] \n", os.Getpid())
	w.Close() // close the write end of the pipe in the parent process

	cmd := exec.Command("/proc/self/exe", "child")
	cmd.Stdin, cmd.Stdout, cmd.Stderr = r, cmd.Stdout, os.Stderr
	cmd.ExtraFiles = []*os.File{r} // share the read end of the pipe with the child process

	cmd.Run()

}

Sample output of the above code, (keep reading the post, and the following output will make sense)

✗ ./process_inout
2025/09/27 13:50:28 [PID: 354560] The main process is saying : HOLA! [PID: 354555]

let's read the code step by step, shall we?

os.Stdin = nil
os.Stdout = w // redirect the output to the write end of the pipe

After creating the pipe I set stdin to nil, since our main process does not need any input, and as you see I pointed the stdout to the read end of the pipe. What does that mean? Generally when writing any program we usually run fmt.Println("something") for example and it does show the output in the terminal, it is because by default both stdin and stdout is pointed to the terminal device for getting inputs and printing outputs.

And what we are doing here is that we are overriding the default to fit our usecase. Coming to the next part of the code which is the part that gonna run first.

fmt.Printf("HOLA! [PID: %v] \n", os.Getpid())
w.Close() // close the write end of the pipe in the parent process

cmd := exec.Command("/proc/self/exe", "child")
cmd.Stdin, cmd.Stdout, cmd.Stderr = cmd.Stdin, cmd.Stdout, os.Stderr
cmd.ExtraFiles = []*os.File{r} // share the read end of the pipe with the child process

cmd.Run()

In a normal senario this print fmt.Printf("HOLA! [PID: %v] \n", os.Getpid()) would show on the terminal, but since we modified the defaults of our process, this line will be written to the memory buffer of the pipe. after finish writing we close the write end of the pipe.

Then the program will re-execute itself, by this special command /proc/self/exe the program will start again with an extra argument called child. as you see for the child process we are setting the defaults stdin, stdout, stderr. in an addition to passing an extra file which is the read end of the pipe. if we don't pass it then the child when re-executed it will not know how to reach to the read end of the pipe buffer.

When re-excuted it will enter the if statement block which the following part

if len(os.Args) > 1 && os.Args[1] == "child" {
		os.Stdin = os.NewFile(uintptr(3), "parent_pid_pipe") // point stdin to the read end of the pipe

		defer r.Close()
		reader := bufio.NewReader(os.Stdin)

		line, _ := reader.ReadString('\n')
		log.Printf("[PID: %v] The main process is saying : %v \n", os.Getpid(), line)
	}

Since file descriptors are just numbers starting from 0. On our child process we passed the following files: stdin, stdout, stderr, and we added an extra file cmd.ExtraFiles = []*os.File{r} that is pointing to the read end of the pipe. so we have the the following fds: 0,1,2,3

Then I modified the stdin of the child to point to the read end of the pipe, and I could get the fd pointing to the read end of the pipe by specifying it's number which is 3. for the stdout I left the default since I want the output to go to the terminal from the child process.

Then I read from stdin, which is again pointing to the read end of the pipe.

Now if you read again this line, you see that the data is received by the child process via the pipe and then it get printed to the terminal.

✗ ./process_inout
2025/09/27 13:50:28 [PID: 354560] The main process is saying : HOLA! [PID: 354555]

We successfully created our own pipe and data flow programmatically!

Also, now you can understand what this guy | is doing and what is happening under this command ( printf 'hello'; sleep 2; printf ' world\n'; ) | cat

and that was it for this blog post!

Onwards!