Tuesday 1 January 2019

The Archie search engine: I was there...

On 11th (some sources say 10th) September 1990, a small team of people from McGill University in Montreal, Canada released what is generally recognised as the world's first internet search engine. It was called archie. The team members were Alan Emtage, (J) Peter Deutsch, Bill Heelan and Mike (der Mouse) Parker. As surprising as it may seem, I used to work with these clever people and regarded them as my friends. But I played no part in the development of archie. Or did I?

My story starts in Montreal in 1986 at a small hi-tech company called MUX LAB, where Peter and I were working as part of a small team of programmers developing a product called ACCESS 36. (I still have a copy of the user manual.) It was both a rewarding and a frustrating experience at times, and after a few incomprehensible management decisions, Peter left to work on his Master's degree at McGill full-time. My time at MUX LAB came to an end in early September 1987 -- I should have seen it coming...

About the same time, the post of System Manager at McGill's School of Computer Science had become vacant, and it was offered to Peter -- which he accepted. Not long afterwards, knowing of my lack of employment, Peter asked me whether I would like to work for him as a (Unix) systems administrator. As the saying goes, the rest is history! I was delighted to accept, although I knew it would be a steep learning curve. Peter was already assisted part-time by Luc, whom I also knew from my stint at MUX LAB, and who was also working on his Master's degree. At that time, the School of Computer Science was located on the eighth and ninth floors of Burnside Hall on McGill's downtown campus.

The first few months were challenging but by early 1988 Peter was looking to expand the team. If I remember correctly, Bill came recommended by a friend of mine (Stephen) at Concordia University. One of my earliest memories of working with Bill was helping to set up and configure a new Sun 3/280 server by booting it from a remote 1/2" tape drive. Based on Bill's recommendation to Peter, Wanda joined the team in late spring or early summer of 1988. And by the summer, Alan had swelled our ranks -- although I think it might have been earlier as he shared an office with Luc, separate from the full-time staff.

One probable reason for the relatively sudden increase in staff numbers was that The School of Computer Science (SOCS) was to be relocated to the McConnell Engineering Building, into newly refurbished spaces on the second and third floors. The SOCS system staff were responsible for all the computers, serial cabling and most of the network cabling. So it turned out to be a very busy summer running and terminating cables, getting everything ready, and then moving everything into new rooms and offices.

By 1989, we were settled in to Engineering and the pace of life seemed to slow down to more of a routine. Frankly, and in retrospect, we were over-staffed for the size and complexity of the School at the time. But they were ideal conditions for what was to come.

As has been related elsewhere, Alan had accumulated a number of large files listing the contents of Internet download sites. (I have a feeling that I grumbled when I discovered that some of them had appeared on the server milo that I used to maintain!) In time and quite logically, this transformed into the archie project, and then took off. Peter announced that we were welcome to volunteer to work on the project, but I remember thinking that most of the (interesting) programming was already underway and I didn't think I could contribute anything useful. (Where was the crystal ball when you needed it, eh?) Bill's desk was adjacent to mine so I was aware that he was working on the user interface, and was inspired by the VMS shell that he had used at Concordia. I remember hearing Alan talk about memory mapping as a way to speed up database access. Mike (Mouse) was based in McCRIM a couple of floors above, and was our friendly local Unix guru who would drop by regularly and offer advice.

So I made no direct contribution to archie that I can remember. However, I did once solve a problem that was causing the archie server (a Sun 4/280 running SunOS 4.1 I think?) to crash fairly frequently while under load. I believe it was Luc who came to see me with error messages from the server log files. It looked as though it was related to the serial ports. This rang a bell: the fix was to physically disconnect the DB25 plugs (still cabled-up to the patch panel) from the serial ports on the server. I then heard that this seemed to help with extending the uptime...

This all happened long before the arrival of consumer digital photography. I had taken a few photos of the work environment on colour print film, and a couple of the most relevant appear here. (Apologies for the general low quality: an inexpensive all-manual SLR and matching lens.) The top picture shows an out-of-focus (oops) Bill (L) and Peter (R), and was taken around the spring of 1988 in the old Burnside Hall office. The photo of Wanda (L) and Bill (R), taken from my desk, dates from the summer of 1991 and shows the office where a good proportion of the archie discussion and development took place. Alan would sometimes use the desk in the centre of the photo as his office was next door. (I thought I had a photo of Alan, but nothing has come to light yet. I'm fairly sure that I didn't take any pictures of Mouse.)

My former colleagues, wherever they are now, may remember things a little differently -- some of the events mentioned happened more than 30 years ago, and memory can be unreliable. I felt it would be useful to record a little more detail about the background and context of the creation of archie as I still recall it. (Some of the references I have found about it are misleading or inaccurate!) I am of course proud of the achievement of my former sys. admin. team members: at the time, none of us suspected how it would change the world!