|
OPEN SOURCE AND THESE UNITED STATES
by
Over the past 40 years a collaborative form of systems development has evolved on the electronic networks of the world. In the wake of the information technology revolution has come a proven method for developing, deploying and maintaining these systems. This method, developed under the auspices of Department of Defense research grants, has resulted in the most successful and reliable software in existence.
The method, based on collaborative intelligence, peer review and functional evolution, has rippled through the world of Information Technology. Its success depends on the uninhibited distribution of the currency of this realm: the source code, documentation and data which are the building blocks of complex information systems. To enhance collaboration and protect its creators, cutting edge source code is commonly released under open source licensing. The associated electronic information is released under open content licensing. Together, I call this duo Open Licensing.
The
Department of Defense can realize significant gains by the formal adoption,
support and use of open licensed systems. We can lower costs and improve
the quality of our systems and the speed at which they are developed. Open
Licensing can improve the morale and retention of Airmen and improve our
ability to defend the nation. These benefits are accessible at any point
in the acquisition cycle and even benefit deployed and operational systems.
Open Licensing can reduce acquisition, development, maintenance and support
costs and increased interoperability among our own systems and those of
our Allies.
Introduction
That the capacity of the female mind for studies is of the highest order cannot be doubted, having been sufficiently illustrated by its works of genius, of erudition, and of science.
Seldom
do those who innovate fully recognize what they have created. Moreover,
it may be many decades before a new idea gains social recognition. Change
is not easy to accept. New methods must first prove themselves among visionaries
before they will be adopted by industry. Time is needed to build the required
infrastructure and expertise; society must protect the equities of older
methods1
The new methods must then be highly successful within leading industries
before they gain general adoption. This paper will discuss a set of ideas,
methods and systems which may be new to some within the Department of Defense.
The ideas, methods and systems resulting from open licensing will be applied
to various aspects of software-intensive systems. The validity of this
application will be examined and discussed. It's hoped that this analysis
will reveal whether opportunities to utilize open licensing exist.
Over the last forty years, open source software has proliferated and come to dominate many important market segments. These segments include the world's most popular world wide web and email servers, the fastest-growing operating system and the underpinnings of the Internet itself. Since being coined by Ms. Christine Peterson of the Foresight Institute, the term Open Source has become a hot topic in the commercial world; it's made headlines in major newspapers, the cover of Forbes and many other national magazines.2 Open Source is a big thing. The term "Open Licensing" embraces both software and electronic documentation. Some believe that the government is on the tail end of emerging technical trends and new ideas.3 In this case, the Department of Defense made the initial investment in the technology but has yet to fully capitalize upon it.
Open licensing of source code and documentation is a legal construct like a copyright. Copyrights grant a temporary monopoly of on works of creativity and expression allowing the holder to reap the rewards of creation and the public the benefit of the work.4 Open licensing has been successful for precisely the same reasons as copyrights on written work- because they both promote the common good. Prior to the invention of the printing press there was no need for copyrights. It was almost as much effort to duplicate a work as it was to write it. The printing press changed this by allowing the ready reproduction and distribution of knowledge. Copyrights were enacted to allow the wide distribution of knowledge during the industrial age. Networking technology offers an opportunity to even more easily distribute the knowledge of the information age. Open licensing allows modern intellectual property to be shared, enhanced and built upon without being stolen or misappropriated.
The industrial age required society to reorganize around the manufacture of sophisticated and abundant products. We are in the midst of another such reorganization. Software systems require as much labor to produce as any industrial age product. Unlike the warehouses needed for industrial age products, an electronic system, which cost millions of dollars to development, can be stored on a single compact disk. Unlike the railroads needed to distribute products, a software system can be sent through email or published on the web in the blink of an eye. The distribution of electronic information can be effortless and essentially free. This ease of distribution serves the public good by capitalizing on a strength of technology and increases efficiency by eliminating the storage and distribution costs which so burdened the products of the industrial age.
Software may be just information, but current licensing restrictions make the distribution of software neither effortless nor free. From their industrial age way of thinking, many people regard software as a product, in the same category as a nail or a pail. Strangely though, unlike these items, the licensing terms of many software applications indicate that we cannot resell the software or even legally loan it to a friend. We can freely dull a nail to prevent it splitting a board. Yet, many commercial licenses prohibit software users from adding notes to its document or features to its implementation even if those changes would improve the value of the system to the customer. A pail can be used to carry water or as a planter, in either case we would not expect it to leak. Yet, i proprietary software is used in a new situation, often the corporation holding ownership neither accepts responsibility nor offers to help get the software working in the new environment. The Y2K problem is only the most recent example of the expensive consequences of these license agreements.
Most people will agree that the cost of software stems from the need for its authors, and the businesses which are built around them, to make a living. This paper will look at an alternative way for all these people to make a living and for the government to get more for its money. The alternative is known as open licensing and it can have a big effect on the way the world does business. The derivatives of Open Licensing may also help alleviate some of the Government's difficulties, including personnel retention and recruitment.
Open licensing is a legal construct, but a construct with important basis in culture and history. Back in the early days of punch cards & magnetic core virtually all software was open source. There was a lot to be written and not many people skilled at producing it. Progress in those days depended on making sure the best examples of code were available so developers could build upon what little already existed. Users of the software and other developers helped the original authors improve their code. These participants became a pool of knowledge available and interested in supporting and continuing the development of the system. If a developer used, enhanced or was inspired by someone else's work, he or she gave them credit by reference to their contribution. In literary terms, this system is very like the notion of footnotes. The system worked very well and is responsible to some extent for the early explosion of automation.5
Some
situations haven't changed much since the 1960s. There's still a lot to
be done and not very many good people available to do it. New developers
still learn from those more experienced by taking up the maintenance of
projects which others began. Experienced developers still move on to new
projects leaving those less experienced to continue maintenance and enhancement
of their creations. In a proprietary environment where only a few people
are allowed access to source code and documentation, this is a major problem.
Companies can be left with few developers who understand their products.
Users can be left with software which is critical to their business but
has been abandoned by it's owners. Within the open source community, this
process of developing systems in a cooperative environment and sharing
the results of a team's work with others, has been coined "the bazaar".
The bazaar is the key process which has allowed users and developers, spread
across the internet, to tackle huge, sophisticated problems like operating
systems. The same process allows these projects to advance far more rapidly
and with greater quality than many traditional methods. The bazaar ensures
that programs are abandoned only when they can no longer serve the needs
of their users. Even in those circumstances, the source code is often for
the basis of future applications.
The
ideas behind open licensing are fundamental to the origination of science
and extend back to the 17th century renaissance; ideas were published,
the knowledge spread and the techniques were enhanced as the technology
matured.9
Scientists have always published their results for others to examine, verify
and use. Just as many early scientists were also artists, architects and
business people in the early days, early software developers and users
were often one and the same. When software pioneers came up with a particularly
efficient implementation of an algorithm or a new idea, they often gave
a copy of it to their friends and associates for them to use and expected
the same courtesy in return. Within this system, productive developer-users
can gain a certain element of notoriety. They find it easier to get jobs
and enjoy a celebrity status among their peers. Their employers can become
rich and famous also. International Business Machines, the Massachusetts
Institute of Technology ,Sun Microsystems, Cisco Systems,Yahoo! Corporation
an many others have all contributed to and benefited from this system.The bazaar is a basic institution of open licensing. The term refers to an organization style based on survival of the fittest and most efficient. The bazaar binds the developers, marketers, writers and users of open source together in an interactive, electronic web. The bazaar works to ensure that the "best" products, as judged by a majority of interested parties, survive. The availability of source ensures that nobody is disenfranchised. If an application or data format is important to an organization or individual they are free to continue using, maintaining and enhancing the product. They are also free to incorporate and integrate any aspect of the technology into future offerings so that the return on their development, training and maintenance investment is maximized through long use and reuse. This process results in systems which harness the collaborative abilities of its user developers to create products of equal or greater quality than any produced by traditional means.14
Within the open source community, there is an inherent tendency for a single implementation and a recognized owner of the official version to prevail. It is true that free access to information and a lack of discipline might allow multiple versions of documentation and systems to develop. Centralized development facilities and a preference for a single support vendor are some of the ways this problem is addressed, but avoiding the problem ultimately relies on organizational discipline. Discipline on the wild-west of the internet comes from a desire by developers to maintain their reputation. Claiming code developed by another to be your own work is a sure fire way to lose that reputation. It's far easier to give credit where credit is due. So rather than create divergent implementations, contributors tend to send their enhancements to the originator or "maintainer" of the application. The maintainer, in turn, incorporates the best of these changes into the core distribution. This arrangement benefits all. The community of users gets an enhanced application. The maintainer gains recognition and additional income through his or her association with the successful application and the organization or individual that contributed the change still has use of the enhanced application. Some measure of recognition and income opportunity may result from individual contributions and those modification may in turn be enhanced or improved by others' effort. Within the military each component or particularly talented branch might contribute that which they know best. Contractors can contribute in the same manner to their own or other systems. This collaboration can function across both time and space bringing together many different interests. This area will be discussed and applied to various software products in the next chapter.
Software
developed under open source licensing combines what have often been thought
to be conflicting goals; innovation, reliability and interoperability.
The internet community values efficiency greatly- that is they do not like
to redo what has been accomplished. As such most open licensed software
incorporates other, existing open licensed code and utilizes only a few
common data formats. As a result of constant reuse and refinement, has
accomplished what proprietary vendors could not. Through constant improvement
and refinement, the open source community has reduced the need for extensive
support of its offspring. The support which is offered often comes from
the original developers and extends not only for an extended period but
as you would expect is of very high quality. The industry and press has
recognized these benefits. In 1998, the readers of Infoworld, a
major information technology weekly, awarded open source operating system
both the "Best Operating System" and "Best Technical Support" awards for
the second year in a row.15
One open source word processor is so reliable the author will pay you $327.68
to report a bug.16
The storage format, templates and other data used by this particular word
processor have remained constant for over a decade, yet the application
has maintained or exceeded the features of its commercial competition.
Open
source has received comparatively little official attention from the organization
that funded most of its early production. As the worlds' largest information
technology customer this organization stands the to gain much from it's
proliferation. The organization is, of course, the Department of Defense.
A popular Government without popular information or the means of acquiring it, is but a Prologue to a Farce or a Tragedy or perhaps both. Knowledge will forever govern ignorance, and a people who mean to be their own Governors, must arm themselves with the power knowledge gives.
Why should the United States government expend the effort to adopt open licensing? The United States government may seem to be well suited to the adoption of open licensing because of its historical traditions and legislation, the many non-monetary exchanges which characterize interagency cooperation and the government's disparate and distributed organizations. Unlike many other nations, the United States has been characterized by a relatively transparent government; meaning a society which freely shares information with it's citizens and believes the exchange of information and goods contributes to the public good.19 Our founding fathers and more recent legislation demand the widespread dissemination of public information and capabilities among citizens and allies. It is a fundamental principle of our culture to encourage the free exchange of information for government and commercial purposes.20 Open licensing of our software and documentation is a natural extension of this characteristic; pragmatists may need a more self-serving justification before they are convinced. I will argue that the use of open licensing should decrease the turnover and increase the job satisfaction of government employees and contractors. The use open source software may also increase the ability of the government's information systems to interoperate and ensure the continued availability of the information which the public has paid the government to produce, collect or maintain. Finally, I will suggest that open source software is less likely to disrupt ongoing government functions while at the same time allowing the government to more quickly adopt new technology.
The use of open licensing can have a role in encouraging cooperation on information intensive systems. Like companies, the various parts of the Department of Defense must cooperate to produce joint systems. Unlike commercial organizations, agencies within the Department of Defense rely on good will, barter and management agreements rather than contracts and monetary exchanges. It's difficult to enforce intergovernmental agreements difficult as there is often no binding contract or clear higher authority to appeal to for a decision. Open licensing can't resolve management disputes, but it can prevent the problems associated with sharing proprietary software or co-development of documentation. The availability of source prevents information hoarding and encourages cooperation. This doesn't obviate the power of money or traditional motivational methods but offers an additional dimension which can be used to motivate and retain personnel.
As
Sun Tzu observed, it is difficult to place a price tag on the value of
leadership or need of followers to be
appreciated by their leadership and peers; loyalty and dedication are seldom
purchased.21
The traditional military concepts of "service before self" and "excellence
in all we do" are both appreciated and practiced widely within the open
source community. Sun Tzu, the ancient Chinese war theorist whose writings
are well known within the military, is also widely read and quoted throughout
the open source community. Open source practitioners work long and hard
on their labors of love or need, it requires true leadership skills to
organize them to do so. Open license projects are often perceived as lacking
a "central authority". Ironically, they are heavily dependent on the existence
of a recognized leader. Leadership ability is critical since there is no
monetary incentive for many open source developers. Participation depends
on the bond between the project leader and other participants. Unlike many
traditional organizations, the leader is not a stagnant position, its occupant
can vary according to talent and desire.
The creation of bazaars to develop open licensed systems may give government and commercial IT specialists an incentive beyond dedication to stay within the Department of Defense. Budgetary restrictions prevent the government from offering the fiscal benefits of many private industries. The starting salaries of Computer Scientists exceed $60, 000 in many areas of the country.22 Government employees do not need to be reminded that it has been over 200 years since the United States conducted its initial public offering, they cannot expect to receive the same salary or benefits of their commercial equivalents. While defense contractors have greater flexibility than government agencies, many of them are also no longer able to offer the compensation found at commercial corporations. In addition, it is not clear that the military is still regarded as a source of cutting-edge, interesting work.
Fortunately, it is clear that remuneration is just one of many aspects affecting developer retention. Many developers and administrators place greater value on the opportunity to do interesting work.23 Open source development projects are regarded as challenging work offering the extra benefit of being able to obtain recognition by sharing the full extent of your talents with others. Many open source projects are able to use this effect to achieve excellent retention rates. The Apache web server team for instance, has maintained five of the original eight core developers for over last four years.24 Cygnus, a subsidiary of Red Hat Corporation which has long produced profitable open licensed projects, experiences turnover rates less than a tenth that of other Silicon Valley firms. Cygnus founders credit much of their success to the attractive power of open licensed projects.25
The adoption of open source licensing may allow the military to leverage some of the current enthusiasm garnered by open license related methods. Government workers are already receptive and knowledgeable about open licensed projects; with over the twice the usage rate of other large organizations.26 Perhaps this is because open licensing offers an alternative for developers who are frustrated. With so many projects canceled before deployment, open licensing opens the potential for reuse of which might not have been released for its intended purpose, but still contains valuable segments which can be reused if licensing permits.
The judicious application of open licensing offers the possibilities of improving both the performance of government systems and the job satisfaction, competence and retainability of military members, civilians and contractors. Open licensing allows developers to share their work to help promote internal projects and gives them a physical portfolio which adds to their external value, productivity and sense of accomplishment. When developers do change jobs, the Government can expect them to bring their portfolio with them. The portfolio serves as a discriminator by providing employers with tangible evidence of the talent or lack thereof of potential employees.
The rationale to adapt open license development methods though are pragmatic as well as social. Just like many organizations developing open licensed systems, the United States Government is deployed world wide. Multinational programs and interoperable systems developed by multiple contractors impose exacting demands on the Government's acquisition system. As functions such as logistics change from personnel-intensive to information-intensive activities, greater levels of system and organizational interoperability will be required. Fewer government & contractor employees will be required to bring together more complex systems in less time. The traditional approach of standards and specifications or purchasing all types of software from a single vendor may be able to produce workable systems, but experience has shown this approach to be a difficult and expensive path.
The lessons learned on the internet indicate that requirements selection and testing27 are more effectively accomplished by the decentralized management, Darwinist selection, reutilized source code and simple protocols encouraged by open license development.28 Since the software is driven by real needs of the user-developers rather than the needs perceived by marketing or sales demands, open licensing avoids nonessential, insecure, unwanted or unreliable functions. Traditional systems are designed to "get it right" out of the box. Open licensed software and documentation evolves as bits of it are made available and debugged. In perfect harmony with the popular "spiral" method of development, open licensed applications start out weak in functionality and allow the marketplace to evolve them. For this reason, open licensing favors systems which produce results even when their eventual functionality is not fully implemented.29
Improved quality and reduced development costs are the major incentives offered by open licensing. Computer science as other with forms of science, benefits from peer reviews and testing. Multiple sets of eyes and minds are the most effective tools for ensuring robust and highly functional applications.30 During major upgrades and development of software intensive systems, peer reviews can significantly lower risk.31 As a specific example of the benefits of open licensing and development within the bazaar, consider how difficult it would become for developers to hide design and implementation faults. Reviews within a bazaar developing open licensed products should not be confused with the more traditional "Independent Verification and Validation" efforts of the past. Reviews within open licensing circles are instead an extension of the process utilized by journals of hard science. Independent verification and validation efforts are not conducted by a nearly unrestricted number of people with a vested interest in the success of the system. Under an open licensing arrangement, code may be reviewed by hundreds or even thousands of independent eyes each of which is focused on those areas of the system where they are a functional expert or interested party. The Apache web server for instance, is developed by a core team of about 20 individuals. Contributions, enhancements and bug fixes however, have been submitted by thousands of users and developers around the globe.32Unlike proprietary products whose features are often determined by marketing surveys and thus reflect past expectations of just a few beta testers, enhancements, bug fixes and contributions to open licensed products reflect the needs of those with a specific need and whose reputation is directly related to the relevancy and quality of their contribution.
The question of whether there should be a limited, selected audience of peers or a thoroughly open process such as used on the Internet is a controversial one. Experiences with many thousand of applications tend to indicate that the more open this process is, the greater the benefits. In general, those individuals taking the time to review and augment the information will be knowledgeable and mature enough to serve as an supplement rather than a detriment to the process. Open licensed products, because they are modified in small chunks and updated quickly to address faults as they appear rather than to the schedule of marketing tend to be evolutionary. There are particular advantages to the release of software products and such artifacts as documentation on a continual rather than periodic basis. Allowing continual comments rather than massive periodic reviews increased the likelihood that controversies will be caught and fixed early when such changes are relatively inexpensive. The increased use of modifiable electronic documentation can allow field comments to be shared and discussed among the end-users of the information. This will require acquisition and headquarters commands to release some control over these documents, trusting them in the hands of Joint Forces and tactical echelons of command.
Many people are concerned with the effect of open licensing on the security of a system. While many non-technical managers believe the release of source code lowers the security of a system, experience shows the opposite. Security "holes" are omissions or weaknesses designed into the code. As more interested parties examine the code, more faults are revealed. Compiling the code into a binary application doesn't fix a security hole or hide it from prying eyes.33 Unfortunately, arguments pro and con tend to be obscured by emotion and the anecdotal examples vastly outnumber the rational studies. Security breaches can result from a failure to follow instructions, inadequate procedures or from unexpected situations. Most vendors do extensive security testing and work hard to protect against known attacks. Unfortunately, security incidents result from unexplored exploitations of an implementation or design. Keeping the source code proprietary and the design secret will temporarily reduce the likelihood that design and implementation flaws will be exploited. In time, the flaws will tend to become publicly known as successful exploitations are publicized. Releasing the source code for widespread review tends to improve the likelihood that design and implementation flaws will be corrected. Unfortunately, the discovery of all security problems is known in computer sciences as an NP-Hard problem- that is one which is believed to be impossible to resolve absolutely.
The best current security practice can hope for is to avoid obvious mistakes, test as extensively as the project's budget can afford and correct problems as soon as they are identified. Open licensing's bazaar is well suited to this process. Most security experts believe that the release of source code improves, rather than diminishes the security of a software system. For instance, following the National Security Agency's release of the Skipjack encryption algorithm, researchers discovered serious flaws in the system and proposed a solution to the problem.34 If the source had not been released, the error might have been discovered and exploited by someone hostile to the U.S. It is unlikely that anyone doing so would have proposed a fix and it would be difficult for the National Security Agency to admit such a flaw existed in a production system they had sanctioned and deployed. Open licensing of the algorithm provided an avenue for both the unbiased review of the algorithm by experts and a quick and effective resolution. As the saying goes, bad news doesn't improve with time. It is far better to go "open kimono" and identify security risks early when they can be fixed with less schedule and cost risk. Further, if components supplied by foreign or unfamiliar subcontractors are incorporated into the system, open licensing makes it far less likely that an accidental or deliberate security problem will be introduced.
When a fix is identified, the government must also work to ensure the fix is incorporated as soon as possible not only into the system within which the problem was identified but anywhere else the implementation may have been used. A bad encryption algorithm for instance must be replaced in all systems which use it be they phones, email systems or radios. Open licensing doesn't alleviate the need to test but it raises the possibility that someone, somewhere will discover and report the problem before the system goes operational when the consequences of a security problem could be life threatening. Open licensing also ensures that an identified fix can be incorporated without being hindered by licensing arrangements or proprietary agreements. If configuration or operation and procedures of the system must be changed to address a vulnerability, open licensing allows the manuals and technical orders to be updated to reflect the change.
It is understandable that the government or any organization might want to restrict distribution of source code. For instance, the government may wish to limit the distribution of important command and control systems to those Allies and Defense Partners to whom we wish to distribute this combat advantage. If the government were able to produce, for example the "perfect" firewall it is unlikely the government would wish its adversaries to possess such a valuable tool. So, the circle of redistribution for government systems may need to be restricted for national security reasons. For those authorized to receive the system though, we should not want to deny their ability to enhance, integrate or debug the system anymore than operational restrictions placed upon a combatant commander improve his or her chances of success. As the size of the bazaar and access to source declines, so do the benefits of open licensing. There is a critical minimum size beyond which the bazaar ceases to be effective; information and communication are required for the organization to succeed. We would not want to restrict the application of information if its use would enhance the combat power of the commander's forces. Further, we would expect the commander to synthesize the new piece of intelligence and his existing knowledge to produce a solution more powerful than either individual piece of information would allow. A bazaar requires the same level of trust and independence as the operational arts.
Open licensing can bring benefits even when there are some level of restrictions. It has been generally established that the most popular open source license, the GNU Public License allows for restricted redistribution within an organization. There are already several successful examples of government sponsored open source projects whose redistribution is restricted. Both the Secure Multipurpose Internet Mail Extensions (S/MIME) funded by National Security Agency's (NSA) X Division and the Public Key Initiative which is funded by the National Institute of Science & Technology (NIST) are classic examples of open licensed projects which promise major advances of the state of the art. The redistribution of the encryption code in both projects is restricted by the United States export controls yet the programs still benefit the government, which receives assistance from commercial and private developers interested in the functionality. The information technology sector in turn gains a functional and efficient security protocol which, since is not controlled by any single corporation or organization may be freely incorporated into their products and will ensure interoperability with other vendors who support the protocols.35
The government places great stock in commonality as a tool to reduce maintenance, administration and development costs. This concern is well placed as studies have shown lack of commonality to be a major cost driver.36 Training is a major part of that cost as are installation and setup charges. It is very difficult to train users, administrators and developers. Once trained, they become extremely valuable even mission essential assets. The government experiences very high turnover rates among military and contracted personnel. On any project, the cost of changing personnel is high and the incidence of personnel changes in today's economy higher still. Turnover rates range up to 70% with averages37 of 35 - 45% per annum.38 This turnover both disrupts and delays ongoing projects. It is possible that the development of open license systems may help alleviate project delays resulting from the Government's retention difficulties.
The government has difficulty keeping information technology positions staffed. Most organizations feel that people are the key factor in determining whether or not a technology project is considered a success. A review of how those people are organized may be time well spent.39 For instance, such a review may reveal an organization or manager which is hoarding code or documentation where others can't take advantage of it. Many projects, both contracted and internal, suffer greatly when a leadership change disrupts the momentum of ongoing work. As previously mentioned, projects developed within a bazaar often voluntarily change leaders and may even have several leaders or none at all. As a result, a bazaar is less susceptible to damage than closely held proprietary "cathedrals". The loose connections between leader and participants allows decentralized open licensed projects to handle leadership and participant changes well. As a result, they are more robust than centralized development teams or projects under the strict control of a single manager. The reasons for this strength are many but central is the ability of all participants to freely access the source code and keep abreast of progress. The code base is the key transport medium for spreading experience and interoperability throughout an organization; it may be seen as the compound interest of information technology. The great number of participants means the contributions of the average participant are less critical than for very small groups. Interestingly, a bazaar may be more durable than a large proprietary development group as well. Since there can be no information hoarding on an open source project, they are less vulnerable to familiar problem of a developer keeping others from reviewing their progress or project leader keeping bad news under wraps.
As organisms are a product of their environment, software products tend to reflect the values of their developers. Open source developers and users of their systems rely on their ability to communicate. Products of a bazaar, in turn, tend to be highly interoperable. Organizations which use open source products can expect improved interoperability, improved long term access to data and an improved ability to more easily incorporate new technology. Interoperability is enhanced by the simple, standardized protocols and data formats generally used by open source applications. Specifications may promote interoperability but experience has shown they do not ensure it.40 Government and industry have tried many approaches to improve interoperability. Extensive testing increases the likelihood that systems will interoperate. However, as more systems and their interpretations of specifications are integrated, the difficulty of testing all combinations grows exponentially. The number of variables in this scenario may be reduced by incorporating the same code within all systems. Open licensing allows you to do precisely this. Once a workable implementation of a specification exists, open licensing permits an organization to utilize that very same code, assumptions and interpretations into every component of the overall system.
Consider the government's need to maintain long-term access to archived information. In addition to the changes in physical media the protocols and formats used by applications can change dramatically over the life of a system. Data formats of non-open licensed systems seldom remain the same. For instance, the data formats of Microsoft's Word 6.0/95, 97 and 98 are all incompatible and indistinguishable from each other to humans, you must attempt to open a document to discover that it is corrupted or unreadable. By contrast, during the same period the Hypertext Markup Language used on the World Wide Web has evolved from version 2.0 to version 4.0 yet all versions are interoperable and can be read with even the simplest text editor.41 The Hypertext Markup Language format is also many times more bandwidth efficient than an equivalent Microsoft Word format document. The point here is not to argue which format is superior, but to note the emphasis open licensed projects and specifications place on true interoperability, error recovery and efficiency. There is temptation to just use the latest office application storage format and convert the archived data into a readable format when it needs to be retrieved. How can archived information be retrieved when contemporary programs are no longer able to import it?
Now consider how open licensing might affect data access. Today, an organization will often have to abandon years of work to use a new system or application. Under open licensing, an organization has full access to its own systems. Under open licensing, potential support vendors have access to the same information as the original supplier. This access means it's possible to contract out maintenance and enhancement to any vendor willing to become familiar with the systems. Differentiation under these circumstances is obtained by superior knowledge of the system. When the time to migrate from existing systems comes, the availability of information and the number of knowledgeable vendors will allow old systems to be migrated and new systems adopted more easily. In the best case, a significant portion of the existing data, source code and documentation can be moved directly to the new environment.
It is critical that the government work quickly to ensure continued access to public information. Unfortunately, there are already many cases where there is no working code able to interpret archived data and no information on how the data itself is formatted. We can anticipate greater problems if the government continues to use the unintelligible and easily corrupted binary formats favored by many modern office applications. Once damaged or unreadable by contemporary applications, there is no partial recovery, this data is lost forever. These office applications represent the lower end of complexity; things become more difficult when you consider information stored in databases or other complex systems. If source code, standardized data formats and protocols are used, gaining access to archived information becomes a matter of compiling or porting the access routines to contemporary hardware. Only when the source code and information on the format the application used is available can we truly insure long-term access to the information.
The Government has traditionally used information systems much longer than their commercial suppliers wish to offer upgrades and support. Current political trends make it important that the government work quickly to ensure long-term access to public information. In an effort to reduce expenditures, the government is moving to contract more business to private concerns. Vendors derive revenue from frequent releases and will they will often arbitrarily modify data storage formats. The vendor update schedules can overwhelm any large organization's ability to integrate and distribute the updates. The asynchronous nature of commercial updates also disrupts an organization's ability to plan for the change and incorporate new systems according to its own needs. In the end, the contractor's revenue is diminished because they cannot sell the government as many updates as they are able to produce and the government is dissatisfied because they feel they are unable to keep up with the latest product release and must incur large costs for retraining and equipment upgrades when they do incur a major update. Open licensed projects also tend to be updated often but it is rare that major data storage formats or other disruptive changes are made. Formats and configuration files which have remained stable for many years are common. Since the information format remains stable, upgrades are less likely to disrupt the flow of work.
Open licensing can lower the cost and risk of change- allowing the government to more easily adopt new technology. Open licensed project updates tend to be evolutionary and hence less disruptive to ongoing operations. In addition, since there is little economic incentive towards releases for the sake of revenue, updates to open licensed projects generally occur for a specific reason- either to fix errors, ease maintenance or incorporate features which have been specifically requested by users. Open licensing dramatically lowers the disincentives to evaluating new functionality. The scheme allows the government to try out a system without up front acquisition costs or contract actions. Vendors, in turn, gain an entré to the governmental market and valuable feedback on the quality and suitability of their products. If generally satisfied with the system, the government can fund or the contractor provide small modifications or additions as required to fully meet requirements. The vendors are then free to use these changes in the systems they offer to other customers. Modifications to open licensed systems may also come at a lower than traditional cost due to reduced "fog and friction". Without the usual restrictions on documentation or source distribution, many more developers will be familiar with the inner workings of the application so bugs are more easily unearthed and enhancements can be more rapidly incorporated.
With an annual IT budget exceeding $57 billion, the government is still a large monolithic culture with a significant market presence.42 The wishes of the government are still a powerful influence on industry. Obviously, vendors will have to determine for themselves if such techniques would be profitable. However for those organizations "on the edge" of moving in this direction, the active or tacit approval of the government may provide the additional market incentive which convinces them to release their systems under an open license. Adopting open licensing seems to fit well within the government culture and bring with it some important benefits. How though can we actually begin to use it?
Applying
Open Licensing to Government Software Acquisition and Maintenance
Anything
purchased by the military must go through the Department of Defense acquisition
system and is subject to the legal agreements imposed by that process.
Acquisition is where the rubber meets the road, licensing-wise, and the
adoption of open licensing must be integrated into the process. The current
acquisition process recognizes several distinct categories of software
systems, each of which will be examined in detail to determine and where
open licensing might be appropriate. A view of the applicability of open
licensing using the DoD Acquisition Phases as a framework is included as
Appendix
A.
One consequence of vendor considerations taking precendence over the needs of the user is a loss of data format constancy. The loss of access to data will become a major issue as the government examines the long term viability of COTS. Traditionally, software applications have consisted of a large amount of logic for producing and manipulating a relatively small amount of data. Many modern systems, such as web servers, however use very small amounts of code to distribute or manipulate truly huge amounts of data.47 Data access has always been a problem and will become one of critical importance as medical records and other information which require long-term access become entombed in obsolete or non-functional, undocumented, proprietary formats. Neither the government nor any other individual customer can afford the continual maintenance effort required to convert data to the latest format every few years.48 Without open licensing, the question of how the government will maintain access to information and correct problems in products abandoned by the commercial market will become a critical one.
Examining this potential problem under the auspices of open licensing however, changes the situation substantially. Under open licensing, the government is no longer tied to a single vendor if immediate, mission essential fixes are required. Upgrades and problems can be implemented with in-house development, outsourcing to the original vendor or by using an alternative support vendor. Existing manuals can be easily augmented with the changes or organization and mission specific information. Data formats can be designed or modified to accommodate the government's existing infrastructure or emerging requirements. How reasonable is itto expect to obtain functional open source products or convince existing vendors to adopt these provisions? There is evidence that the answer to this question lies in the evolving nature of the software dependent industry and revolves around the question of whether software is a product or service industry.
Industry is moving away from emphasizing the software "product". The new service-oriented members of the information technology industry emphasize the effect of software functionality rather than a shrink-wrapped product.. Economics is driving this decision. The software corporation Intuit, for instance, now offers a traditional COTS income tax product for $49 or a web-based, but functionally equivalent, alternative service for $19.49 Within the commercial sector, the software industry is beginning to be dominated by service oriented corporations.50 IBM and Digital Equipment Corporation (DEC), for instance, receive a majority of their revenues from support. This service orientation is especially strong among start-ups on the Internet. Deja.com, America On-line and Yahoo! are all information technology corporations which market a service. It's interesting to note the extent to which these corporations both depend on and develop open source software. Yahoo! for instance uses open source extensively and attribute to it much of their success.51
The government focus on commercial off the shelf software has inhibited it's adoption of this new paradigm. The military views COTS as a means to maintain parity of functionality, improve interoperability and reduce fixed expenditures. This perspective however, assumes software is a product not an information generation, distribution, manipulation and archival service. Under the product model, a software product may be purchased and once used remains static until the next purchase. In the new world, software as a service becomes an living organism evolving along with the organization to meet emerging needs and to eliminate obsolete functionality and data. Evolving, open licensed systems can provide increased flexibility and improved supportability. If the source of a manual is available, its content can be tailored to suit individual departments.
Many
organizations are finding shrink-wrapped, open licensed systems can provide
both the monetary and schedule benefits of COTS and the flexibility of
customized applications.52
Commercial vendors, such as CISCO Systems, are already utilizing open licensed
systems because other vendors, such as Cygnus, are willing and able to
conduct prompt, customized development specifically for their needs. Open
licensing may even result in a marketing opportunity for those involved.
Sun Microsystems found, for instance, that the release of Java source code
resulted in many bug fixes and new operating systems being supported. These
contributions increased Sun's goodwill and the value of their Java product
line. The improvements could be leveraged by others to create new products
which then further increase the value of the original corporation. The
porting of StarOffice to Sun's Java and subsequent purchase of Star
Division Corporation by Sun greatly increased Sun's chance of success
in the business environment. Without the open licensing of Java and the
Open Source proponents who popularized Star Office, Sun might be still
confined to its legacy position of backend applications.
Ensuring access to the products it pays to produce can bring substantial benefit to the government. It reduces the overhead of separately negotiating licensing agreements with vendors and provides a vehicle for moving previous work into new applications. If the Federal Government, for instance insisted that the product of grants given for software soluations be released as open source, the solutions developed by States to implement welfare reform tracking and other common functions could be shared and not duplicated fifty times over. Much work is repeated by many contractors because of the uncertainty over who might own various functions. Without open licensing, similar functions are continually reimplemented because there is no easy mechanism for using just a portion of a previously developed function or system produced by another military service or government agency.
Interoperability improves our ability to work with coalition partners and allies. More generally, the open licensing of products will ensure that tax dollars contribute to the public good by ensuring the results of publically funded research is available to all those who's tax dollars have contributed to its creation. As former enemies work to equip themselves with our software intensive products, greater interoperability is required to effectively work with the many nations who may be put in ad hoc cooperation. The ability to easily share functionality is becoming critical, even long time allies struggle to integrate their electronic systems with ours. A vehicle such as open licensing could alleviate many of these problems. The open licensing of code and documentation will allow allies and other organizations to build on our information technology foundation and easily tailor these products to their environment while retaining key functions and interoperability. This ability may eventually mature into a distributed development and maintenance organization with allies working jointly to implement a project. We should strive to take advantage of international interoperability and the being able to leverage the talent of other nations to improve our cumulative capabilities.
Military unique development accounts for an increasingly small share of the overall software used within the military but is, perhaps, the category most important to national security. These systems include major weapons systems with embedded software such as jam resistant communications gear and systems for military unique functions such as nuclear blast simulations.
Many senior leaders assume military unique software will be far buggier and less feature-full than many of its commercial counterparts. For practitioners of the bazaar, this is an expected result; military software is often available to only a small group of persons and not seen or reviewed by outside experts. There's a lot of evidence to suggest that one of the best means of achieving higher quality is the open license distribution of these military systems. As Eric Raymond's paper, "The Cathedral and the Bazaar" so eloquently discussed, ".. given enough eyeballs, all bugs are shallow.." The more talent which examines a system, the greater improvementin functionality and quality. Unless a strong technical case can be made that unlimited distribution of source code will harm national security, it is in the government's interest to ensure all it's systems are released under an open licensing arrangement.
This discussion has shown open licensing is appropriate to all of the software systems types in popular use within the government. Moreover, the adoption of open licensing brings with it substantial benefits to the government, in the form of reduced costs and risks, more tailored and efficient solutions and the potential for more reliable and interoperable systems. If more military systems were available under open license it would enable one of the most powerful mechanisms for advancing technology transfer and interoperability. The government would lower the risk of archived data being lost. Since source code and documentation are freely available, the lifetime of open licensed systems can be extended indefinitely. Vendors can diagnose difficulties even if they did not originally develop the system. Research institutions can contribute useful ideas and expand on existing functionality to provide something totally new. These benefits stem from unrestricted redistribution and modifications rights and the development bazaar generally used by open licensed systems. Commercial firms are more than willing to recognize and take up the maintenance or development of anything for which they see a market. Many past government projects have made the transition to open licensing with excellent results. A few of these instances will be examined in Chapter 5, Success Stories.
If you claim to be an expert on modern information systems and you aren't rich then most people will infer that you are stupid.
One other application which has received a large amount of press is the most popular web server. This web server is the first server to hold over 50% of the world-wide market share.56 It's the Apache web server, widely recognized as the most featureful, reliable and innovative of the web server offerings which includes Microsoft's Internet Server, Netscape's Commerce Server and Oracle's Web Server. In fact, the Apache web server is so dominant, Apple and IBM have both decided to provide commercial support for it and make it the default web server on some of their commercial platforms.
Open licensed systems dominate the networking and infrastructure segments of the software industry. The use of open-licensed source code has become ubiquitous within even commercial systems. The Request for Comment (RFC) system used to develop and document the requirements for internet systems has become the model, developing interoperability among all information systems. Developers of open licensed systems were the first to recognize and exploit computers as an information sharing tool.
This
chapter has examined only three of the open source systems which without
advertising budgets, commercial support or huge paid development staffs
have come to dominate their markets. There are many others such as the
Domain Name Service (DNS) which allows us to easily address the millions
of computers on the internet. In fact, every aspect of the internet relies
on open licensing. These systems succeeded, not because they hid their
implementation behind closed doors, but resulted when the developers opened
them wide and invited the world to use, examine and critique their work.
This open, widespread analysis led to greatly improved functionality, quality
and security.
Open licensing can help organizations to avail themselves of previous efforts. Open licensing removes the barriers to widespread adoption of new or improved ideas and enhances the value of existing systems by allowing organizations to incorporate segments past efforts into new offerings. Open licensing allows you take what you or anyone else has done and tailor it in whole or part to fit a new situation or fix a new problem. Traditional software licenses force the reimplementation of many equivalent systems from scratch. Open licensing allowed groups of cooperating developers to compete with and win against products from the world's largest and most respected firms. The wider adoption of open licensing may be held back from greater contributions mainly by misconceptions.
Humanity needs practical men, who get the most out of their work, and, without forgetting the general good, safeguard their own interests.
There's
a term in a the telecommunications industry called, "the last half-mile"
which refers to all the details
required to get even simple things to work. A communications network stretching
halfway around the world can be stopped in its tracks by a neighborhood
council preventing the company from digging a trench across a road leading
to the building you want to connect. How easy might it be for the Department
of Defense to move towards open licensing? If the Department of Defense
wishes to implement open licensing, are there "neighborhood councils" which
might stymie the effort?
There are several areas where the wider adoption of open licensing promises clear benefits. Research organizations, such as government laboratories, should be actively encouraged to adopt open licensing and utilize open license development methods. Systems released in this fashion can easily be adopted by commercial industry for commercial development. Another low hanging fruit software category is security enforcement and detection. The National Security Agency's FORTEZZA algorithms used by the Defense Messaging System and the Defense Advanced Research Project Agency's firewall toolkit are two extremely successful examples of how open licensing can dramatically increase the acceptance and quality of systems in this area.
There are no major legal or policy obstacles to a decision to formally decide that software and data wholly funded by the government should be open licensed. The framework of open licensing of unclassified government documents is already in place. The Freedom of Information Act lays out the circumstances under which the Department of Defense and other government agencies must divulge information. The redistribution of the documentation is not generally restricted. Information systems and the data they generate and store are partially covered by the Freedom of Information Act, however tradition holds the source code of those applications must be addressed separately.
Misunderstanding the nature of software source code and the value of a review of that code by others are the major impediments to a more widespread adoption. The growth of vendors such as Cygnus and the many distributors of the Linux and FreeBSD operating systems offering shrink-wrapped, open license products show this business model offers significant opportunities. The widespread adoption of open licensed systems by commercial institutions, large and small, indicates they feel such systems may reduce costs or provide greater productivity than other commercial offerings. While there are no documented examples of corporations whose business has been damaged by the release of source code, corporations will naturally be hesitant to modify their current policies. Software system source code is often regarded as different and more powerful than other forms of documentation. Despite the real-world success of open licensing, it is still difficult to convince government or corporate management that there is no real difference between source code and other forms of documentation.
Open licens