How to Rebuild All of Computer Science Using a Blank Computer and a Floppy Disk

I found this post on Reddit and wanted to save it for posterity.  It reminds me of the old days in both good and bad ways ...

Making Your Own Tools

I remember reading somewhere that you aren't a real carpenter until you've made your own tools.  Not being a carpenter of any sort, I'm not sure if its true or not, but I've started to feel like it might be true for programmers.

More and more I've noticed that IDEs are becoming required to work with code.  And more and more, these IDEs have very similar, but not the same, features.  They all offer customization of one form or another, but what's more custom than something that works exactly the way you want it to, because you wrote it that way?

So I've set out on a quest to take the journey of my programming forefathers and write, to begin with, an editor and a language with its own compiler/interpreter.  Perhaps the two will be related, perhaps not.  Perhaps the language will be generally useful or maybe it'll only be the extension language for the editor ... or perhaps it will be nothing more than an experiment in language design and have no practical purpose whatsoever.  But I expect that once I get to a certain basic level of usefulness with the editor, it will become my sole text editor since I do believe in eating my own dogfood.

Since my platform of choice nowadays is the Mac, I have already started writing my editor as an Objective-C OS X application.  I started out with the code from the article: Building a Text Editor in 15 Minutes.  It has given me a huge appreciation for the underpinnings of OS X and gotten my editor almost all the way to that basic usability level I mentioned before.

As I move forward, I'll be sharing thoughts and discoveries from my journey here.  I hope you find them useful ... or at least entertaining.

Taxes Kill Developer Passion

I read Process Kills Developer Passion earlier today and could not wait to get home and write up my thoughts.  The essay posits that "Over the past few years, the software industry has become increasingly focused on process and metrics as a way to ensure 'quality' code."  I think the software industry as a whole has been tilting at that particular windmill for almost as long as there has been code.  I think the shift in focus is that now it is developers themselves that are trying to build systems to ensure quality code, not just managers.

It lists a bunch of ways that one would be spending time on process rather than not-process, presumably getting real work done.  He also lists a few horror stories like "... the test cases took two- or three-times as much time to code as the actual code ...".  My only question there was, "Were they good test cases?"

I agree that spending more time than is necessary on process can be seen as a tax, resources that are spent on things that do not benefit the end result of the project.  But in the cases of things that are objected to, they are not necessarily a tax.  Perhaps they were on the projects he has worked on.  But I have also seen counterexamples of the things that are listed:

• Doing full TDD, enabling the developers to write less actual code because the tests proved that it wasn't necessary.
• Requiring a high percentage of code coverage, ensuring that the tests that were written don't just test the same code over and over again.
• Having full code-reviews on all checkins, giving newer developers a chance to learn from their more experienced colleagues ... and even their more experienced colleagues being exposed to new techniques.
• Using tools to measure code complexity, so that there isn't a bunch of time wasted in code reviews saying, "I think this is too complex" and "I don't think it is".

So the processes that the article decries can be useful (or at least some of them).  It is only when there is a bunch of wasted effort through mindless devotion to "the process" that something quickly becomes a tax.  And taxes kill developer passion because a developer gets their pleasure out of creating something.  The more taxes that are heaped on top of creating something, the less pleasure they feel.  The less pleasure, the less passion.

But let us not forget the biggest tax of all, bad code.  Bad code forces one to do battle with it in order to make changes.  Bad code drags on a project so that it is moving slower and slower and slower until any change is moving the project backward because fixing one bug leads to the introduction of at least one new bug.  Bad code is a tax because the time spent fixing bad code could be spent moving the project forward.

As a wise man once said, "The only things certain in life are death and taxes."  You are going to pay taxes on your project, whether it is in the form of process to prevent bad code from entering your code base or in the form of fixing the bad code once it was in there.  Does that mean we should just suck it up and accept the drudgery of process?  Hell no!  But there's no reason process should be drudgery.  For example:

• There are many code review tools out there that significantly speed up the code review process as well as make it a searchable system that can be mined for information and learnings.  (Malevich and Mondrian are two that spring to mind.)
• Keeping track of time doesn't have to be a chore.  The Evidence-Based Scheduling in FogBugz uses a very simple system.  When you start working on a work item, you click a little dropdown and say which one you're working on.  When you stop, you set it to Nothing.  It even auto-stops for you at lunch and quitting time if you configure it so.
• Planning Poker and other practices have been developed to make planning easy, accurate and fast.

The article points out that there is a spread of skill levels amongst developers and that it isn't fair to force the best developers submit to process that is designed to prevent the worst developers from making mistakes.  Checklists and following process have shown time and again that they reduce defects even amongst the most talented.  Because all developers make mistakes.

So just as we shouldn't perceive mindless process as a magic bullet, neither should we perceive all process as evil.  Let's just strive for more and more ways to keep the taxes down.

Files Are an Implementation Detail

John C. Dvorak recently wrote that the desktop gets no respect and it should.  This column has been a source of discussion and disagreement between my friends since it made the rounds.  Leaving aside the whole "licensing" issue, what Mr. Dvorak nearly captures is the divide between what most people need to use computers for and what professionals need to use computers for.  He claims that the desktop can do things that appliances can't.  I disagree, one only needs to look at the TiVo to see an example of an appliance that does things that one would once have needed a desktop computer to do or had to do manually.  But let me start by defining my terms ...

An appliance, for the purposes of our discussion, is a device that performs a task or set of related tasks in a way that a user can learn to operate it by intuition or usage and hides most if not all implementation details from the user.  A refrigerator is an appliance because you don't need to know how it works in order to keep your food cold, you just open the door, stick something in and a few hours later it is cold.

A desktop computer is not an appliance because you need to learn aspects of how it operates in order to properly use it.  You need to know about files, folders, networks, programs and a whole lot more.  And the more the computer can do, the more you have to learn.  A desktop computer does not hide its implementation details.  As a matter of fact, its job is more to piece together various things that let you use the implementation details without hiding them.  Yes, you can plug in to a network nowadays without needing to know much about TCP/IP, network protocols or the OSI stack.  But it doesn't keep that stuff from you either.  A refrigerator doesn't give you access to its compressor and let you tweak it.

Here's another point where I disagree with Mr. Dvorak.  The modern automobile is an appliance.  I don't need to know anything about how a car works in order to operate it.  I don't need to know about the transmission, the engine, or the brakes.  I just need to know that putting the car in D and pushing the accelerator makes it go.  I take my car to a mechanic to fix it, just like I have a professional come and fix the refrigerator if its broken.  It is only the American love affair with the automobile that has kept it so the average person still can get access to the engine and other internals.  And while I know more than many about how an automobile works, I have only the most basic grasp of newer automotive technology like anti-lock brakes and traction control.  Those are things that I have decided I don't need to understand, they just work.  That's the magic of an appliance, I don't need to know in order for it to work.

So let's take a look at what I believe computers are and should be evolving into, the modern camera.  You have the point-and-shoot camera for consumers and you have the SLR camera for enthusiasts and professionals.  With the point-and-shoot camera, I don't need to know about white balance, f-stop or any of a bunch of other technical aspects of photography.  I simply point it at what I want to take a picture of and click the button.  On the other end of the spectrum, there are the SLR cameras where I may not need to know those things in order to take a picture ... but it doesn't hide those things from me either.  The point-and-shoot camera is an appliance.  The SLR camera is not.

What does a general computing appliance look like?  Simply enough, it looks like the iPhone or iPad as far as the way it operates.  With these devices, you don't need to know about files or folders or networks.  You turn it on, download an app and perform a task.  Everything else is taken care of for you.  Does this mean you can't "calculate, with accuracy, the trajectory of a rocket shot to the moon, and the amount of fuel it needs to get there"?  Of course not, but you do need an app for that ... which someone else must have made for you.  Does it mean that you can't connect a Wacom tablet and do cool, unspecified things with Adobe Illustrator?  Well, for now, yes ... but the concept of a general computing appliance doesn't preclude this.  A general computing appliance is something for consumers and will hide the implementation details of how the computer works, but that doesn't mean it can't do powerful things or allow one to attach peripherals such as a Wacom tablet or a keyboard.

And here's where we get to files, the big argument amongst my friends was that files are the glue between applications that are never perfect.  A file allows one to take data that was created by one application that does part of the task I want to another application that allows me to finish the task.  The implicit fear is that on a general computing appliance, apps will turn into data islands that do not allow me to perform this kind of assembly-line workflow that the desktop PC has enabled since its inception.  If you look at the iPhone, this is mostly the case.  One app's data is not shared to any other apps, with the exception of some built-in applications like the camera and the photos app.

On Android, it is a different story.  Android has a system whereby one application can serve up its data to the rest of the device via a content provider system.  This allows any application on the device to integrate with the built-in contacts application, or the gallery application.  It allows new applications to serve new kinds of data and control how that data can be accessed or changed.  This system also does not resort to sharing files.  It is an OS-controlled communication protocol that allows an application to have its own data strongly protected but also affords sharing.  This type of system is what will allow for files and folders to truly become an implementation detail on consumer-level general computing appliances.

This is my idea of what the general computing appliance nirvana looks like from the user's perspective:

• A way to download, install and uninstall applications.
• A means of launching an application.
• A system whereby installed applications can notify the user of events.
• A means of switching between running applications and controlling what applications are running.
• A system of configuring the settings of applications.

Additionally, a general computing appliance should afford the following without the user having to be involved:

• Sharing of data between applications.
• Access to the device's peripherals.
• Other integration points between applications (a la Intents in Android).

And that's about it as far as a consumer is concerned.  This is the genius behind the iPhone and why I believe it will be considered revolutionary in the years to come.  It was the first true general computing appliance.

Where does this leave the desktop PC?  Should it get more respect or should it become the computing world's Rodney Dangerfield?  I partially agree with Mr. Dvorak here.  The traditional desktop PC will continue to have its place in the computing future.  But it will belong to enthusiasts and professionals, not to the consumer.  The people who are writing apps for consumer devices will use a desktop PC (or the future's version of it).  The people who are doing more free-form computing tasks will likely use a desktop PC, perhaps engineers or scientists or data miners, people who aren't performing the same task all the time.  I believe though that over time the set of people that use what we now call the desktop PC environment will shrink slowly but inevitably.  As operating systems evolve and developers begin to understand more of what people actually use computers for, we will be able to simplify more tasks to the level of an appliance.  It does not mean the end of the enthusiast or the professional, simply that one does not have to be an enthusiast or professional to get the benefit of the technology.

This is not something to be reviled, but something to be considered natural.  It is the march of progress.

The Perfect IDE

I've been writing a bunch of code lately for an Android project I'm working on.  What this means is that I've been writing Java code in Eclipse.  The Java part isn't bad, though coming from C# and .NET there are a lot of things that I miss.  But that isn't what this is about ... this about IDEs, Integrated Development Environments.  I don't know if Borland's Turbo products, Turbo Pascal being my introduction to the company, were the first ever IDEs, but they were probably pretty close.  Back in those days, one needed an IDE because computers could only run one program at a time ... so running an editor, then leaving it to compile, then running a debugger, etc was all a big pain.  Now, computers are never running just one program ... and even programs are doing multiple things at once.  My phone, right now, has a couple programs running on it with at least seven threads each.  But I digress ...

So we don't need IDEs anymore.  Why do we still have them?  Because opening one program that keeps track of all the minutia of writing code is a good thing.  It knows which files belong to the project so when one needs to build the project, it just takes one click (sometimes not even that) and it's done.  When there is an error in the build, one can simply click the error message and the offending place in the offending file is displayed for easy fixing.  There are all these little things that the IDE helps with that one simply can't get from a set of command-line SDK-style tools and an editor.

My problem with most IDEs is that you can't have just one.  I currently have five installed.  I have Visual Studio 2008 for working with older .NET projects.  I have Visual Studio 2010 for working on newer .NET projects.  I have the built-in PowerShell Integrated Shell Environment for PowerShell scripting.  I have Eclipse for working on Android projects and I have IntelliJ IDEA because a friend recommended I check it out also for Android projects.  Why do I have to have all these different IDEs installed?  Because Visual Studio is best for .NET projects, Eclipse has the tools I need for Android, the PowerShell ISE is the only thing that has syntax highlighting for PowerShell.  Don't get me started on why I need two copies of Visual Studio ... that's just extra stupid.  Could I use Eclipse for .NET projects?  Maybe, but the amount of work it would take to make that happen is just scary.  I'm not even sure that coercing Visual Studio into doing Android development is even possible.  IntelliJ seems to only be an IDE for Java, so using it for anything else is out.

So, essentially, the Perfect IDE means I only have to have one.  With it, I can write code for any language and platform and have all the tools I expect to have at my disposal.  Let's go over those, shall we?

• Editor
• Font selection
• Color syntax highlighting
• Flexible formatting support (brace style, etc) where the editor automatically formats my code as I type
• Intelligent completion (like IntelliSense in Visual Studio)
• Search and replace including regular expressions
• Refactoring tools
• Project support
• Some way of knowing what files, objects, etc are in the project
• Easy navigation to any project item
• Build
• One-click build (or background building if it works reliably)
• Error integration
• Debug
• Breakpoints
• Watch variables
• Expression evaluation
• Window manager
• Support multiple top-level windows
• Allows me to move any view group to any window
• Allows me to dock any view group to one particular area of a window
• Views can include a file in the editor, project explorer, etc.

This is just the bare minimum of what I expect from an IDE because this is what most IDEs get mostly right. Some additional features that most IDEs do not get right or don't offer:

• Source control integration
• Most IDEs have support for one or two source control packages, but they rarely are the one I'm using at the time.
• Build integration
• Most IDEs have their own built-in build tool, but they rarely make it easy to support advanced build tools such as Ant or MSBuild.  IDEs should get out of the build business and use these advanced build tools by generating the required files and letting the build tool do the work.
• Visual Studio gets this kind of right, but there's a bunch of stuff that's done behind the scenes that makes it worthless if you try to use their generated stuff outside the IDE.
• Eclipse supposedly gets this right in that it uses Ant internally, but I didn't see any Ant build files until I generated them myself.
• Support for multiple languages and platforms
• How hard is this?!?  The process of editing, compiling and debugging has been set in stone for decades.  Heck, even the syntax of most languages is becoming more and more similar ... there's C-derived syntax and everything else.
• Make adding new languages drop-dead easy.  I've tried to add just color-syntax highlighting for PowerShell to a couple IDEs and it is a huge pain, if possible at all.
• XML support
• First-class support just like any other language with syntax highlighting, formatting support and intelligent completion.
• Data visualization for well-known formats.
• UI generation tools
• This is something I can understand most IDEs not having support for a bunch of platforms because developing UI is very different even within a platform such as Windows, let alone across platforms.  Though with the advent of XAML and similar technologies, I see convergence happening here too in the coming years.

So that's my laundry list.  How is this achieved?  That's a great question.  I'm glad you asked.

IDEs need to realize their main job is window and plugin management.  The IDE is simply there to mediate the interactions between various plugins that do the heavy lifting and provide windows that let the developer see what the plugins are doing for them.  I think what keeps IDE makers from seeing this is that every IDE includes, by definition, an editor.  So they write an editor first and then start tacking on all this other functionality and then they have an IDE.  Yes, even the editor should just be a plugin.  And I shouldn't have to be stuck with just one editor plugin.  Maybe I want one for XML and a different one for Java and yet another for C#.  I'm a developer, I'm choosy like that!  Multiple installed plugins for all services should be the norm, not the exception.

Maybe I'm asking for too much.  Maybe what we'd end up with if someone actually made what I'm asking for is something similar to Eclipse with eleven different versions to choose from and no help for someone just wanting to get some work done.  Maybe we'd end up with crazy, incomprehensible configuration dialogs (though pretty much every IDE has at least one of those already).  I think it is possible though.  I really believe it can be done right given the right vision.

Someone want to prove me right?