Over at Bill Seymour’s, I made the mistake of getting into the topic of comparative programming languages. Ow. My bad!
A guy named R. Murphy (he claimed that Murphy’s Law was written in his honor but I doubt it) was one of the team I worked with at the ULTRIX Resource Center at Digital. The URC was a pre-sales group – our job was to help positively influence sales by showing prospective customers that Digital had the capabilities to solve their problems. Or something like that. It was a great job for a young, energetic, systems guy who wanted to learn absolutely everything there was to learn about computing, specifically UNIX environment. I got pulled into some really weird and wonderful projects including a few tech demos that went on to become products for the customer, that one time literally made billions off of the code. In those days I was living in the academic mindset my parents raised me in: tell all you know, publish or perish, and if you move harder and faster than everyone else nobody can catch up to you.
The way it worked, my boss and mentor, Fred A. had a meeting each week, with the current project list, and we briefly summarized where we were. If Fred had a new thing come in, he’d have a 1 page writeup in the form of: problem statement, logistics and timeline, contact info, notes on possible resources. He’d go through the stack of writeups (usually only one or two) and hand them off to the team; there were 4 of us. From there, Fred’s role was to bulldog things, offer advice, and make introductions if we needed more resources. Fred knew everyone in the company, so one time I was having trouble with a device driver interrupt for a synchronous I/O capture card and Fred said, “wait” and called Matt T (“the Code Warrior”) who solved the problem for me in 2 minutes of extremely dense code, recited over the phone. Literally, Matt wrote a driver patch for a multiprocessor-safe driver, off the top of his head and it did the crucial part for me and then I added on all the chassis stuff.
Anyhow, one day Fred had a project that was a 2 person gig. We needed to implement a framework demo for a logistical system for a client so they could benchmark it against a similar thing Sun was doing. HP was in the mix, too. So we needed to scope a database, write network APIs for it (there was a time when not all databases were networked) it had to be fault tolerant to a degree, and there needed to be a user interface forms capability for adding and searching and fiddling with the data. Fred gave it to me and Murphy. We repaired to Murphy’s office (which meant: standing outside while he smoked cigarettes) and divvied the work up. I proposed to do the network stuff because at the time I was a good sockets programmer and Murphy nodded and said, “I’ll do all the UI stuff. I’ve done a lot of that.” We agreed to check in on eachother at the end of the next day, and went off to our cubicles.
Actual picture of me, 1987
Murphy is the old schoolest old school DECcie. I remember when he used to hang out on comp.sys.dec and sometimes someone would buy a PDP-8 at a computer fair and try to get it running. They’d post something like, “I powered it up and all it says is ‘4J'” Murphy’d follow up instantly, “that’s a diag code; make sure the I/O processor is correctly seated in the unibus. If it does it again, enter ‘JC3 IPL’ and tell me what it says.” That kind of stuff. Murphy had been a field service guy for the first PDPs, see… He was like that about everything. We all regarded him with an awe close to worship. I, literally, wanted to be like him when I grew up.
So I went back to my cube, hopped onto my hotrodded DECStation 3100, and started putting together a socket-based API atop a database running on a MicroVAX (because at the time VMS databases were much better than the stuff available on UNIX) and having a whale of a good time coding away like a rabid squirrel. Picture the Squirrel in Ice Age – that’s how I coded in those days.
At the end of the next day, I dropped by Murphy’s office and as he lit up I told him where I was and concluded I’d have my part working in another day or 2 and we could integrate around the APIs I had put together. Murphy nodded. I asked him how he was coming along. “Oh, it’s done.” What?!
I honestly don’t know what I expected Murphy to do, because I was not at all worried about his part of the project and was totally head down in mine. I guess I expected he’d get stuck into some kind of curses-based thing, writing a scripting system for placing cells on the screen or whatever. Nah, Murphy had used COBOL. What?! Then he gently explained that there were programming languages written specifically to do forms on screen and, basically, all the stuff he was supposed to be doing, and what he had done yesterday was finish his cigarette then wander back into his office and threw together his side of the system. Total time: 25 minutes. Then he cocked an eyebrow at me and said, “the best part is I don’t have to test anything because the way it works is – if it works at all, it works perfectly. I, of course, had been whacking away in C/UNIX sockets land, where you’ve got to worry intensely about race conditions and buffer space and stuff like that. Murphy had simply picked up the right tool for the job. [Nowadays, of course, it would have been a simple matter of using an SQL engine and some of the many existing APIs to talk to a networked SQL engine, but that stuff literally did not exist at that time – collectively, computing was in the process of figuring out that a common network database capability would be a good thing.]
I worked as a consultant at the database research group at University of Maryland for a while (UMIACS) with some really amazing people. The project was … another networked database. It had been developed as a consulting offering for NASA under contract, and was not working right. It was a sort of interesting problem because (to my knowledge there still aren’t) debugging distributed applications is a tricky problem, especially if there are timing problems and race conditions. The application had been developed using Sun Microsystems’ remote procedure call system (RPC) which was basically an API for exporting APIs between multiple systems over a network. As I dug in, I concluded there was something wrong with how RPC was working. It was ugly. I obtained the source code for the RPC layer and developed my own debugging layer for RPC, in which all the parts of the procedure calls reported synchronously to a central log that I could monitor. Oddly, that solved the problem. Uh-oh. So I had to keep digging and it turned out that RPC communications used un-connected networking, with a timeout and a retransmit. UDP packets. If the database was slow responding, the RPC layer went “aha, must have lost the packet containing our query” and re-sent it. So, the database was crunching away on the query contained in the first packet, when another copy of the query came flying in, and it started crunching away again and… if there were caching effects in the database, the database eventually caught up and the RPC layer threw away all the duplicate replies but sometimes the database would just get buried behind a queue of identical queries. So, I wrote a TCP-based query stack that didn’t need to retransmit, and waited (much more efficient) and suddenly the whole system was more reliable and hundreds of times faster. Job done! So, they let me go. From there I went to Welch Medical Library researching hypertext stuff for the National Library of Medicine and Howard Hughes Medical Institute, and was in on the early stage of the Human Genome Mapping project, and also did some experiments with reverse-engineering phototypesetting systems and implemented “distributed tocs and docs” which was basically a web-like thing, implemented as a shell script, in 1989. OOps. I didn’t make any attempt to do anything with it beyond show it to a few people who said “that’s cool but aren’t you supposed to be working on something else?”
Ah, the good old days, when you could claim shit like that and nobody could check…
https://en.wikipedia.org/wiki/Edward_A._Murphy_Jr.
Gosh, that sounds like a cool job. /s
Meanwhile, the REAL Murphy …
I mean – holy shit. THAT is a cool CV.
I just read my post back and it sounds excessively snarky. It’s not meant to.
I’m puzzled. RPC/XDR supported both UDP and TCP from day one. (See RFC 1831.) So I’m not sure why you needed to write anything new….
@geoffarnold:
I don’t remember. You’re right now that you mention it there was the rpctcp stuff. I think the reason I rolled my own was server-side, the server wanted to block on the database call and had to manage accept() or something like that. The remote function call paradigm didn’t work for 21 year old arrogant Marcus.
Seems unlikely, since the movie that was taken from wasn’t release until 2002
I have been following with interest the progress of Rust. I started going through the tutorial once, but then got distracted by other stuff. But it seems to me that if people are constantly making the same mistakes (e.g. buffer overflows, array bound violations), then engineering them out of the language is a Good Thing.
Obviously @ selkirk #5 they copied an image of Marcus. You don’t live in Los Angeles like I do so it isn’t so apparent to you that writers and animators steal *everything*. Marcus came first.
Folks compare me to House from the TV series. I came first. No promises if I am any less fictional, because I do wonder this about myself sometimes. :)
Reginald Selkirk@#6:
But it seems to me that if people are constantly making the same mistakes (e.g. buffer overflows, array bound violations), then engineering them out of the language is a Good Thing.
Agreed, with provisos.
First – a premise of cloud computing and parallel scaleable systems (e.g.: web server farms with load balancers) is that performance isn’t a consideration unless you are coding a serial process that must do fast transactions – then, maybe. So one of the topics of heated discussion in the 80s was whether programming environments should manage memory, or whether the programmer should do that. The “garbage collection wars” were not helped by some really bad GC implementations in hypercard and smalltalk, where your program might go non-responsive for a second or two, apparently at random. Now, it’s “just throw hardware at it” so inter-piled languages that do memory management and auto-park for the programmer are the status quo.
I do think that language designers make some horrible mistakes and there should be a process for teaching people how not to release bad languages. (I have done a few, myself…) for example, python treats stylistic formatting as syntax, trying to enforce a look. Arrgh. If that is what they wanted, there should be a reformatter in the editor and the editor should just make it right always. Someone over at Bill’s mentioned the difference between K&R C and ANSI – that is another example of loathesome language feature design – the function type declarations are required to … force programmers to do something that a 2-pass linker with error checking ought to do. Of course programmers should declare functions, but making the programmer do the linker’s job is onerous.
I also firmly believe that developing a language is 1/3 of the game. There should be 2 compilers, one of which produces optimally fast code and the other which outputs code that is thoroughly instrumented, so that array boundary errors are checked for, bad typecasts are detected (“hey you can’t put a 16-bit value in a 64-bit word!”) and all memory usage and file descriptors are tracked and checked. I used to pop my own code into the system call library so I could link against my own version for test runs – for example if the number of file descriptors increased monotonically in the same function, it would flag it. Stuff like that is super important, not operator overloading and syntactic gimcracks.
The other piece is a debugging. A great debugger is necessary. The crap that C started out with is horrible. I used to use a thing called Saber-C, which was a C interpreter. It managed every chunk of memory and its scope and it knew if foo was an int or a char and it knew what you put in or took out. You could stop a program anywhere and run code at the command line – i.e: dumpstruct(whatever) if you had a function to examine a struct, or you could just print it and it gave you a manipulatable view. You could pick a data element up in a window and watch it change, or blink when it was referenced. At the end of a run you could get a detailed breakdown of which functions went to the allocator the most, the size of the chunks, what functions freed them, and what leaked. Basically that was all functionality I had in my own version of malloc() but Saber did everything better.
Unfortunately Saber-C cost $5000 a seat and everyone just used gcc and printf() for debugging. Ugh. So the product died because nobody wanted to port it off SPARC.
Anyhow, “kids these days need to get offa my lawn!”
Absolutely! I spend a good chunk of my professional life writing C#, and in the latest version and with a good IDE it will politely cough (without actually being so uncouth as to refuse to compile) if I write something which could potentially attempt to dereference a null. It’s fantastic! But that’s only possible because some other poor bastard has dealt with all the nasty, messy stuff for me. Somebody has to deal with raw pointers and shit, because “you can’t just place a LISP book on top of an x86 chip and hope that the hardware learns about lambda calculus by osmosis” (to quote the inimitable James Mickens). I get to enjoy civilization only because somebody else has wrestled daemons. Somebody has to deal with the bare metal – I just thank the dark gods that it isn’t me.
(Seriously, read this, it’s brilliant: https://scholar.harvard.edu/files/mickens/files/thenightwatch.pdf – “Even as we speak, systems programmers are doing pointer arithmetic so that children and artists can pretend that their x86 chips do not expose an architecture designed by Sauron.” Amen!)
Ch-ch-ch-ch-Changes
@mjr, 8:
I… er… no, I have absolutely no idea what any of that meant.
It’s always slightly disturbing when someone starts to talk properly about what they do, to the point that you lose any comprehension of what the words they’re using even mean in the context they’re using them. On the other hand, it’s comforting to know that someone knows what’s going on…
Entirely offtopic, I thought you might appreciate what someone else has been up to with Midjourney and Pikalabs.
https://youtu.be/eajSxwZnv1M?si=ipULkVRxjaAanVSx
Dunc@#9:
“Even as we speak, systems programmers are doing pointer arithmetic so that children and artists can pretend that their x86 chips do not expose an architecture designed by Sauron.”
Mickens is a lot of fun.
Having watched most of his performances, and enjoyed them greatly, I feel like he and I are making the same broad complaint: software ‘engineering’ is too fad-driven and often the fads focus on language features rather than the surrounding stuff that results in more reliable, faster, easier code.
There is also a problem that the big vendors want to produce proprietary versions of their C, or C++, or whatever, that subtly lock the developer to their platform. That makes code more complicated and full of conditional build directives that make it harder to test.
sonofrojblake@#12:
I thought you might appreciate what someone else has been up to with Midjourney and Pikalabs.
That is so cool! I have been avoiding getting started with the AI video scene because render times can be crazy and it’s potentially a huge time-sink. Eventually… One thing that is nice about the AI scene is that if you drag your feet a bit, things are better by the time you get around to them.
@14
Dunc @ #9: thank you, that was the funniest thing I read this tax year. I haven’t had the pleasure to do proper systems programming on proper systems (multi-user networked computers), but I’ve been systems-programming-adjacent (microcontrollers, working 16-bit PCs around the operating system, drivers, firmware), so it all resonates.
Marcus, you reminded me of two important things: one, that young* me was a bit silly and did not know what he was doing (but thought he did) at least some of the time; and two, that being able to use the right tool for the right job is an amazing skill — we all reinvent the wheel unnecessarily. On the other hand (or on the same hand, depending on how you look at it): https://xkcd.com/1988/
* young = from 14 years old to about two weeks ago.
Maybe this warrants more than a throw-off comment and an XKCD reference. I miss the days when software development was not a mature industry. That is, when an average programmer could have a fair understanding of their system (or several): hardware, bootstrapping, operating system, drivers, compiler, tools, filesystems, applications. Today, the average programmer glues things they don’t understand together. This is partly due to the vast increase of the pool of programmers, and what that increase does to the average. But what-used-to-be-average programmers have also lost all hope of having a thorough understanding of their tools, not because we’re getting old, but because we’re expected to create complex systems quickly and cheaply, building on the vast library of existing software that we don’t have the luxury to study in any depth.
I acknowledge that “library” is the wrong term here, as it implies careful cataloguing and curation. But it flows better than “festering, disorganized pile of the occasional rough diamond and exceedingly rare polished gem, but mostly shit piled on top of more shit, and marketing”.
cvoinescu – if you liked that, there’s more… I particularly like the one about web programming and JavaScript – a language I hate with the burning passion of a thousand supernovas. He does a very good job of hitting most of the things that are wrong with it.
Dunc @ #18: for some reason, WordPress renders your first link (“there’s more”) as an a element without a href attribute.
Probably my fault, here’s the naked URL: https://mickens.seas.harvard.edu/wisdom-james-mickens
(Note that some of the vimeo links on that page are dead, but you can still find the videos on youtube.)