Defiant: A Proposal For Tool Unification.

Defiant is a Node.js module that I am writing to answer the problems that I will address in this post. First of all I must address the question: "Why the name 'Defiant'?" In order to do so, I must first make sure that you understand the mindset of current development (mainly addressed at the Node.js ecosystem, although it is not limited to such).

The Motivation For Tool Development

We should probably begin with the Unix tool philosophy. For the uninitiated, the philosophy states that programs should be small, accomplish a single task, and have a well-defined input/output mechanism. As many command-line warriors will tell you, knowing how the individual tools work saves them time and allows quick, powerful, bug-free productivity without having to reinvent the wheel for every task. It is one of the philosophies that has propelled Linux to be the powerhouse of development that it is today.

Second, we should consider the DRY (Don't Repeat Yourself) software development axiom. It rightly encourages programmers to be smart about their code, and to have good modularization practices (especially in the realm of OOP) in order to reduce interdependence. More colloquially, it is an admonition to developers to not re-implement solutions that others have already written, but to have good structure and make proper use of libraries.

Modern development has embraced these two philosophies, sometimes to the extreme (eg. left-pad anyone?). So is this a problem? Yes and no. It is not a problem in that what is generally produced is small, high-quality, well-used and well-tested libraries. The problem is that the developer must know how to put them together. Assembly of tools is the problem, in that it requires the developer to be an expert in the domain of inputs and outputs that that tool provides. In this sense, writing small modules does not automatically bestow the Unix tool philosophy on a project.

In terms of the Unix tool philosophy, the interface is simple: stdin and stdout. The output of one command is piped to the input of the next. These small tools all share the same interface, and can easily be reasoned about to be assembled quickly and powerfully. The problem is that this abstraction does not scale well in terms of complex systems. To put it simply, you wouldn't build a phone system out of piped-together commands, so why would you think that you can build a secure website using piped-together modules?

Security is complex, and most of the advice on the internet is that you should not roll-your-own, because you will probably get something wrong. This is definitely true in terms of cryptography, but it is also true of security. You are (probably) not a security expert, and even if you are, why would you want to re-invent the security wheel for each project? Are you absolutely positive that you will not make a mistake? Are you up-to-date on the most recent security practices (they are, after all, out of date in two years)? Finally, even if you are, having to write your own security boilerplate for every form you create, every project you work on, is a basic violation of the DRY principle.

Defiant.

My proposal is to create a framework to buck the trend (hence the name "Defiant"). I am not intending to undo the hard work that so many good programmers have done, but rather to begin to integrate the best parts into a cohesive model that can be more easily combined. The form processing complexity is a very good example to give you a hint at the broader picture. When a form is received via POST, it must be processed. There are already libraries to do this (e.g., busboy, formidable). Then the form must be validated. Then any spurious data (for example, the files which were injected in the story told earlier) must be purged. Then the form can be submitted (processed). Then the http response can be assembled. You get the idea: there are a lot of moving parts, and this is just in a single aspect of a website. And yet, for every project, developers have to write the same boilerplate code (or at least their version of it) in order to get the different parts working. Or, more likely, they only write parts of it, unwittingly leaving security holes in their wake.

But perhaps thinking in terms of a website and http traffic is too constricting. Node.js can do many, many things. It can be a socket server (handy for many types of real-time communication), a DNS server, an email server, a server management tool (via ssh and ftp), a Git interface, a humble http server, and many, many other things. Moreover, it is very probable that most projects need to combine several of these tools. For example, a website providing WebRTC communications would probably need shared authentication in both the http(s) and the socket communications. An email server may want to provide a web-based front end for management and email access. A server-monitoring tool may use sockets and ssh and a web-based front end all together. A website may want to offer ftp integration to advanced users.

What am I getting at? I believe that we have many amazingly powerful standalone systems, but that aren't designed to play well together. Defiant is an attempt to provide a standardized engine for these disparate, powerful modules. I'm not trying to make just another CMS (in fact, http services don't necessarily require a CMS), but rather I'm trying to create a framework which can accommodate the needs of developers and end-users alike. Am I inspired by Drupal? Absolutely! But I'm not beholden to it, either. Drupal was written for a specific technology (PHP and database-driven) and purpose (http-based interaction), and I see Defiant as much more flexible than that.

Defiant also seeks to address problems common in the dev-stage-prod workflow faced by many database-driven projects. It seeks to simplify development by offering clear APIs to developers, as well as an ability to override and/or extend core functionality without having to hack core code (à la Drupal's mantra). Defiant should simplify site building for beginners and enforce best practices for developers when its APIs are used. I see it as being scaling-aware so that even small projects can easily grow, but with a simplicity that is transparent to the end-user.

"Is Defiant ready for use?" you may ask. Sadly, no. It's coding has only just begun. If you would like to collaborate on its development, however, please jump in on the GitHub repo: https://github.com/coreyp1/defiant

If anything, I would like to get others to join me on this journey. I see Defiant as the catalyst to make development exciting without the drudgery of writing boilerplate or having to become an expert on everything in order to write secure systems. This is the change that I feel our industry needs, and it is the change that I am passionate about.

Be Defiant!