Tanmay's Blog

OS 101: What is an Operating System?

What Is an Operating System

The problem it exists to solve

Not "it manages resources" — that describes what an OS does, not why it must exist.

You have one machine and want to run several programs written by strangers, some buggy, some hostile. They all need the CPU, RAM, disk, network. What stops program A reading program B's passwords? What stops an infinite loop freezing the box forever?

Nothing — unless something has more power than they do. The OS is that thing, and the whole subject is about where the extra power physically comes from.

Two jobs, permanently in tension

Every OS subsystem is one of these or both:

   ABSTRACTION                        ARBITRATION
   make hardware usable               make sharing safe

   disk sectors & cylinders           who gets the CPU, for how long,
        -> open()/read()/write()      and who decides?
        -> "a file"
                                      A must not read B's memory.
   4 GB of physical RAM               Enforced how?
        -> "your own address space"
                                      A must not hold the CPU
   one CPU                            forever.
        -> "your program runs
            continuously"   <- a lie. A very good one.

Every pleasant abstraction is a lie about shared hardware; the arbitration keeps the lie from collapsing when someone probes it. Your program believes it owns the machine. It owns a slice, and it cannot tell.

The problem software alone cannot solve

The OS is code — instructions executed by the same CPU as everything else. So:

If the OS is just another program, what stops mine from ignoring it?

With software alone, nothing. Protection is not something an OS can implement by being clever. It requires hardware.

Privilege modes — the hardware's answer

 +----------------------------------------------+
 |  KERNEL MODE   (ring 0 on x86, EL1 on ARM)   |
 |  every instruction allowed:                  |
 |   change page tables - disable interrupts    |
 |   talk to devices    - halt the machine      |
 +----------------------------------------------+
 +----------------------------------------------+
 |  USER MODE     (ring 3 on x86, EL0 on ARM)   |
 |  privileged instructions FAULT               |
 |  memory access filtered by the MMU           |
 +----------------------------------------------+

All your code runs in user mode. A privileged instruction there doesn't fail politely — the CPU traps: aborts the instruction and jumps into the kernel. Memory outside your allowed set faults at the MMU; that is a segfault.

This is an Axis-3 hardware feature ([[architecture-taxonomy]]) that exists so an OS can be built. DOS, the Apple II and the Amiga had no such mode — any program could do anything, and one bad pointer took the machine down. Not sloppiness; the hardware offered no alternative.

Kernel mode is the OS's entire power. It has no other source of authority.

System calls — asking, not calling

Your program can't touch the disk, so it must ask. The obvious design is exactly wrong:

 X  call kernel_read_file(...)

    If a user program could CALL into the kernel it could call ANY kernel
    address -- jumping past the permission check straight into the
    "now do the privileged thing" part. The privilege bit would be worthless.

So the mechanism is deliberately not a call:

 1. put a NUMBER in a register        (which service: 0 = read)
 2. put arguments in other registers  (fd, buffer, count)
 3. execute the syscall instruction   (x86-64: `syscall`;  ARM64: `svc #0`)
          |  the CPU, in hardware, atomically:
 4. flips the mode bit to KERNEL
 5. jumps to ONE fixed address the kernel registered at boot
          |
 6. kernel reads the number, looks it up in a table, and VALIDATES every
    argument (is that fd yours? is that buffer in your address space?)
 7. does the work
 8. returns -> mode flips back to USER

Step 5 is the whole security model. You don't choose where you land. One door, placed by the kernel, and everything past it begins with validation. You name a service, never an address.

    mov  $0, %rax        ; syscall number 0 = read
    mov  fd,  %rdi       ; args in registers
    mov  buf, %rsi
    mov  cnt, %rdx
    syscall              ; <- the crossing. one instruction.

open() in C or Python is not the syscall; it's a libc wrapper that arranges registers and executes that instruction.

See it

$ strace -c ls
#   1 execve      <- the shell asked the kernel to run me
#  12 mmap        <- map libc into my address space
#   4 openat
#   3 read
#   1 getdents64  <- "list this directory" -- the actual work
#   2 write       <- text on stdout
#   1 exit_group

Everything ls accomplishes happens on those lines. Between syscalls, a program is just arithmetic in a sandbox. It cannot affect the world except by asking.

The crossing costs real money

   function call        ~1 ns
   system call        ~100 ns - 1 us      <- 100x-1000x worse
                      (worse with Spectre/Meltdown mitigations like KPTI,
                       which swap page tables on every crossing)

This is why buffered I/O exists — purely to amortize the boundary tax:

import time

# Unbuffered: one write() syscall per line -> 100,000 crossings.
t = time.perf_counter()
with open("/tmp/a.txt", "wb", buffering=0) as f:
    for i in range(100_000):
        f.write(b"hello\n")
print("unbuffered:", time.perf_counter() - t)

# Buffered: accumulate in userspace, one write() per ~8 KB -> ~75 crossings.
t = time.perf_counter()
with open("/tmp/b.txt", "wb") as f:
    for i in range(100_000):
        f.write(b"hello\n")
print("buffered  :", time.perf_counter() - t)

# typically ~10-30x apart. Identical bytes on disk.
# The entire difference is how many times you crossed the boundary.

Same output; the gap is pure boundary tax. Also why printf buffers until a newline, why databases batch writes, and why sendfile() and io_uring exist (move data with fewer crossings, or none).

The other door: interrupts

   SYSCALL              INTERRUPT
   voluntary            involuntary
   from below (you)     from outside (hardware)
   "please do X"        "something happened, NOW"

A device raises a line; the CPU stops mid-stream, switches to kernel mode, jumps to a handler. Disk read finished. Packet arrived. Key pressed. And one of them is a clock.

The timer interrupt is where the OS's power actually comes from

How does the OS stop a program stuck in while(1);? The OS is not running — your program has the CPU, never yields, never makes a syscall. Software cannot solve this. So:

   at boot the kernel programs a hardware timer: "interrupt me every ~1-10 ms"

   your program runs ... runs ... runs ...
                   |
          TIMER INTERRUPT  (hardware, unstoppable)
                   |
      CPU -> kernel mode, jumps to the handler
      scheduler: "you've had your slice"
      save your registers, load someone else's
                   |
   a DIFFERENT program runs

Preemptive multitasking rests entirely on a hardware interrupt the running program cannot block (blocking interrupts is itself privileged). Without it you get cooperative multitasking — classic Mac OS, Windows 3.x — where one hung program hangs the machine. → [[cpu-scheduling]]

The OS is not a program that runs. It is code that is dormant almost all the time, entered only by syscall or interrupt, which uses kernel mode to enforce lies about the hardware.

Event-driven, not continuous. When your CPU is at 100%, the OS is barely executing.

Kernel architecture

 MONOLITHIC (Linux, Windows, BSD)      MICROKERNEL (QNX, seL4, MINIX)

 +---------------------------+         [fs][net][drv][..]   <- user mode
 | scheduler memory fs       |          |   |    |    |
 | network stack  drivers    |         +---------------------+
 +---------------------------+         | IPC + scheduling    |  <- kernel
 all in kernel mode                    +---------------------+

 fast (function calls)                 robust (a driver crash
 a driver bug = kernel panic            doesn't kill the system)
                                       slower (message passing)

Linux is monolithic with loadable modules — drivers load at runtime but still land in kernel mode, so a bad driver still panics the box. The substance of the 1992 Tanenbaum–Torvalds debate; monolithic won on performance, microkernels own the domains where a crash is unacceptable (QNX in cars, seL4 in avionics).

What an OS is not

Why it matters

Gotcha

Don't picture the OS as a supervisor process running alongside yours. It isn't scheduled; it has no loop. It is dormant code entered only through two doors — syscall (voluntary, from below) and interrupt (involuntary, from outside). Everything it does happens inside one of those two entries.