Summary of Key Points
This article reveals the rationale behind the design of Windows' HANDLE in comparison to Unix's file descriptors (fd). The concept of HANDLE was not an attempt by Microsoft to show off its technical prowess; rather, it was a solution to address issues related to kernel security, maintainability, and unified management of multiple resources in the complex business environment of the late 1980s. A HANDLE essentially acts as a "permissioned indirect access token." While this approach sacrificed some performance for greater system robustness and unified synchronization capabilities, it also came with challenges such as difficulty in detecting resource leaks and complexity in inter-process communication (IPC). Linux eventually encountered similar problems and had to adopt similar concepts to extend the functionality of its file descriptors. This demonstrates that in the evolution of technology, there is no one-size-fits-all solution; instead, various approaches are chosen based on specific context.
1. Why Didn't Microsoft Follow Unix's "Everything as a File" Approach?
Unix's philosophy of "everything as a file" is elegant: opening a file yields an fd (an index in an array), and reading/writing data is done directly through the fd, just like using a straw to drink from a beverage. However, this elegance assumes that all resources can be abstracted as streams of bytes (such as files or network connections). But Microsoft's goal was not to create a "better Unix"; it had to deal with a chaotic business environment that required support for multiple processors working simultaneously, fine-grained user-level permission control, and the ability to run multiple systems like POSIX, OS/2, and Win32 on the same kernel. Many resources could not be treated as byte streams. For example, threads have execution states and priorities; you can't simply "read" a thread. Attempting to force these resources into the fd framework would result in a mess of difficult-to-maintain control codes (IOCTLs), similar to attaching various complex buttons to a straw, making it much more cumbersome to use.
2. What Exactly Is a HANDLE? Not a Pointer, but a "Permissioned Access Token"
Many people think of a HANDLE as a "wrapped pointer," but in reality, it's more like a permissioned access token:
- The HANDLE you receive is a sequence of meaningless numbers (e.g., 0x0000000C) and does not represent the direct address of a kernel resource (just like a door card number doesn't represent the password to a safe).
- The kernel maintains a "door card registry" (the handle table), where each HANDLE corresponds to an entry in the table, and it is this entry that contains the actual pointer to the kernel resource (e.g., a thread or process).
- Handles also have permissions; for example, a given HANDLE may only allow you to terminate a process, not change its priority. This design ensures security by separating user access from the underlying kernel resources.
Why this approach? Directly providing users with kernel addresses is dangerous because user programs could rely on the specific location of these addresses (e.g., assuming a certain field is located at a particular byte). If the kernel upgraded and changed the structure, all programs would crash. Additionally, users could potentially forge addresses to attack the kernel (similar to using a fake key to open a bank safe). Handles act as a barrier between the user space and the kernel, providing both security and flexibility.
2. Two Major Advantages of Handles
The design of Handles brings two significant benefits:
(1) The System Can Always Be Upgraded Without Re-writing Programs
Users only receive Handles and do not have direct access to the kernel's internal structure. From Win95 to Win11, the kernel's process structure changed significantly, but old programs continued to work seamlessly—just like using a new lock at home without needing to replace all the keys.
(2) All Resources Can Be Controlled with a Single "Remote Control"
All kernel resources in Windows (threads, timers, mutexes, etc.) share a common header structure (DISPATCHER_HEADER), with their status represented as either signaled or unsignalled. This allows you to use a single API (WaitForMultipleObjects) to wait for multiple resources to complete tasks (e.g., waiting for three threads to finish, a timer to expire, or a mutex to be released). This was a groundbreaking design in the early 1990s.
3. Three Pitfalls of Handles
While Handles are convenient, they are not without drawbacks:
(1) Difficulties in Detecting Resource Leaks
Unix's fd leaks are relatively easy to track since they correspond to specific files. In Windows, however, one Handle can represent multiple types of resources (threads, events, pipes). If a program leaks tens of thousands of Handles over several days, it's impossible to determine which ones are related to unused events or forgotten-to-release third-party SDK components—similar to losing a bunch of keys and not knowing which ones belong to which locks.
(2) Complicated Inter-process Communication for Passing Handles
Handles are bound to specific processes, so you cannot directly pass a Handle from one process to another. You must use the kernel's DuplicateHandle API to create a copy in the target process's handle table, which adds complexity to inter-process communication.
(3) Limited Concurrency
WaitForMultipleObjects can only wait for up to 64 Handles simultaneously. This becomes a bottleneck when modern servers handle tens of thousands of concurrent connections. Microsoft later introduced IOCP (I/O Completion Ports) and IORING to overcome this limitation, similar to using a universal remote control that can only control a limited number of devices.
4. A Similar Path for Both Linux and Windows
Linux initially adhered to the "everything as a file" approach but eventually faced similar issues. For example, using process IDs (PIDs) to terminate processes could lead to accidental kills due to PID reuse. Therefore, Linux 5.1 introduced pidfd_open, which uses Handles to represent processes in a similar manner to Windows' Handles: with permissions and indirect access, ensuring no duplication. Later, Linux also encapsulated timers (timerfd) and signals (signalfd) as Handles, adding permission checks.
This shows that in the long run, all technologies face the same physical constraints (security, maintainability, concurrency), leading to similar solutions. Whether starting from a "everything as files" or "everything as objects" approach, engineers make trade-offs based on the specific needs of their time.
In Conclusion
Handles are not a product of Microsoft's desire to show off its technical skills; they represent a clever solution to practical problems. By using indirect access, Windows achieved security and flexibility. Although Handles have their drawbacks, they have proven their value and even influenced the design of other systems. The essence of technology is to find the best balance within given constraints.