by Darius Kazemi, May 30 2019
In 2019 I'm reading one RFC a day in chronological order starting from the very first one. More on this project here. There is a table of contents for all my RFC posts.
Interprocess communication
RFC-150 is titled “The Use of IPC Facilities”. It's by Richard B. Kalin of MIT Lincoln Laboratory and dated May 5th, 1971.
A business note: my ten-month Mozilla Fellowship has supported this project up until now, but that has ended. If you like what I'm doing here, please consider supporting me via my Patreon.
The technical content
Kalin is providing this document as a warning. He says that the current understanding of interprocess communication (IPC) over the ARPANET (communcation between one program on one computer with a program on a different computer) is woefully and perhaps dangerously underdeveloped. In particuar, the current implementations of IPC don't allow for some easily imaginable future uses of IPC.
He says that a general solution to IPC is necessarily going to be “clumsy and inefficient” and that solutions need to be machine-specific.
He then goes into various examples of IPC usage.
First he points out that different processes need different sized chunks of data input in order to actually do a computation on the input. For example, a character converter needs at least 1 character to do its thing. Some programs operate on whole lines of text, and some programs might need an entire file to be sent before they can do their thing.
Every executive system must face the problem of deciding at what times enough input is available for a given routine for it to run efficiently. This decision can not be taken lightly.
(By “executive system” he is broadly referring to the operating system, by the way.)
He talks about “wakeup”, which is the point at which the operating system tells the process: okay, you have received enough data so you can wake up and do some calculations now. Wakeups are computationally expensive, so you want to minimize the number of these orders you issue. While there is no general solution, one common solution is for a process to say to the operating system: “I am expecting N bits of information so wake me up when you have that many to give me.” Or perhaps “wake me up when you get a carriage return character since that's a whole line of input.”
Kalin calls these chunks of actionable data a logical message, and suggests that standardized boundary markers for logical messages might be a good thing to implement.
Sometimes full logical messages are too big to be sent in one go, as buffers on the receiving computer might be smaller than a logical message (for example in the case of sending a full file). So we need to be able to send logical messages in small chunks, indicating “this is part 1 of N, this is part 2 of N, […], this is part N of N.”
There is also the issue of the total amount of buffer available on the receiving computer. There should be some way for a receiving computer to say “I can't receive any more message parts right now, please cease further transmission until I give you the go-ahread.” Otherwise the buffer could overflow simply because a sending computer is sending data at too fast a rate.
Output buffers should be allocated as for similar reasons, though Kalin notes:
This particular user option is sure to raise the wrath of legions of system programmers. From their point of view, the empty buffer space that is being preallocated is necessarily being wasted. For although it contains no messages, it is not available for other uses. The user, on the other hand, does not correlate 'empty' with 'wasteful' nor 'full' with 'efficient'. A process needs empty output buffers as much as it needs full input ones. Both are resources necessary to sustain processing.
Sending processes should be able to know how many of its messages have not yet been processed, so perhaps it could back off on sending further messages. Note that this is with regards to actual processing of the messages, not just the NCP passing it on to a process and sending back an acknowledgment. This is more like getting a job status in the Remote Job Entry system. It could also assist with debugging remote issues.
There should also be an ability for an operating system to have a process operate in “burst mode” (Kalin's phrase). The idea here is that the OS deliberately lets a whole bunch of logical messages pile up in the input buffer, and then wakes the process and lets it process all the messages at once. This reduces the number of wakeup calls to the process, which as explained above are computationally expensive and should probably be minimized.
The remainder of the document describes in detail the various capabilities of an RJE-like remote status system concerning the input data stream on a remote machine and its statuses. Things like: querying basic info about the Nth logical messaged queued for a process, how many logical messages are queued total, measurement of throughput in the logical message queue, manage permissions on the queue, etc.
Kalin closes by requesting feedback.
Even the most informal note or telephone call will be
appreciated.
Analysis
I think it's interesting that the author includes a note near the beginning that he is keeping the document in electronic format, specifically drawing attention to the mutability of such a document:
one should not consider any document of this kind as either authoritative or final. For this reason, this paper is being kept as a computer based textfile, and so will remain subject to editting [sic] and rerelease whenever new insights become understood.
This RFC is asking for a lot and I'm curious to see how it plays out over future RFCs. At a glance, I don't see any direct responses to this RFC, so maybe they just ignore it?
How to follow this blog
You can subscribe to this blog's RSS feed or if you're on a federated ActivityPub social network like Mastodon or Pleroma you can search for the user “@365-rfcs@write.as” and follow it there.
About me
I'm Darius Kazemi. I'm an independent technologist and artist. I do a lot of work on the decentralized web with ActivityPub, including a Node.js reference implementation, an RSS-to-ActivityPub converter, and a fork of Mastodon, called Hometown. You can support my work via my Patreon.