John K. Ousterhout
People often ask me how I got the idea for Tcl and why it became so popular. In this page I've attempted to reconstruct the thought processes that led to Tcl and Tk and trace the evolution of Tcl and Tk over the last decade along with my path from Berkeley to Sun to Scriptics. As you will see, Tcl's beginnings were very modest; the success of the system was quite a surprise to me, and it's only in retrospect that I've begun to understand the reasons for Tcl's popularity.
The birth of Tcl
The Tcl scripting language grew out of my work on design tools for integrated circuits at the University of California at Berkeley in the early 1980's. My students and I had written several interactive tools for IC design, such as Magic and Crystal. Each tool needed to have a command language (in those days people tended to invoke tools by typing commands; graphical user interfaces weren't yet in widespread use). However, our primary interest was in the tools, not their command languages. Thus we didn't invest much effort in the command languages and the languages ended up being weak and quirky. Furthermore, the language for one tool couldn't be carried over to the next, so each tool ended up with a different bad command language. After a while this became rather embarrassing.
In the fall of 1987, while on sabbatical at DEC's Western Research Laboratory, I got the idea of building an embeddable command language. The idea was to spend extra effort to create a good interpreted language, and furthermore to build it as a library package that could be reused in many different applications. The language interpreter would provide a set of relatively generic facilities, such as variables, control structures, and procedures. Each application that used the language would add its own features into the language as extensions, so that the language could be used to control the application. The name Tcl (Tool Command Language) derived from this intended usage.
The notion of embeddability is one of the most unique aspects of Tcl, and it led me to the following three overall goals for the language:
- The language must be extensible: it must be very easy for each application to add its own features to the basic features of the language, and the application-specific features should appear natural, as if they had been designed into the language from the start.
- The language must be very simple and generic, so that it can work easily with many different applications and so that it doesn't restrict the features that applications can provide.
- Since most of the interesting functionality will come from the application, the primary purpose of the language is to integrate or "glue together" the extensions. Thus the language must have good facilities for integration.
I started work on Tcl when I returned from my sabbatical early in 1988, and I began using the first version of Tcl in a graphical text editor in the spring of 1988. By this time I had stopped working on IC design tools in my research, so the experiments with Tcl were mostly academic: I didn't think that anyone besides myself would be interested in an embeddable command language.
The birth of Tk
One of my other interests at that time was graphical user interfaces. As GUIs became more and more popular in the 1980s I had noticed that the complexity of interactive software was rising rapidly. The most interesting new developments seemed to require large projects with enormous investments. As a professor with modest resources, this worried me: would it become impossible for small teams like my research groups to build innovative interactive systems?
I concluded that the only hope for us was to reduce the resource requirements by building large systems out of reusable components. If most of the complexity of a system was in the components, and if we could carry the components forward from system to system, perhaps we could build large powerful systems with small teams. It would take quite a bit of work to develop the components, but this could be done gradually over time, perhaps by several smaller groups working together.
I also reasoned that component-based design would not work unless there was a powerful and flexible mechanism for integrating the components. These thoughts occurred shortly after I had begun thinking about Tcl, and it occurred to me that an embeddable command language such as Tcl might also be useful as a scripting language for integrating components. I decided to test this theory by creating a set of GUI components as a Tcl extension and using Tcl to assemble the components into graphical user interfaces. This extension became Tk.
I began work on Tk in late 1988, but it was a part-time project, so it took about two years before there was enough functionality for Tk to be useful.
Early distributions
I no longer remember exactly when I began giving Tcl away to other people. By 1989 I was using Tcl in a couple of projects, and I believe that I gave a talk on Tcl during the Berkeley Industrial Liaison Conference, where several hundred people from industry came to Berkeley to hear about research in our department. I believe that this led to invitations to give talks on Tcl at a few interested companies, and that some of the people at those talks asked if they could have copies of Tcl. In any case, I gave away a few copies of Tcl to early adopters in 1989.
In January of 1990 I presented a paper on Tcl at the USENIX Conference. The talk was attended by several hundred people and generated enough interest that many people began asking me for copies of Tcl. Around this time I decided to make the Tcl source code freely available on our FTP site at Berkeley. Once that happened, Tcl usage began to spread by word of mouth over the Internet.
One of the people at the USENIX presentation was Don Libes of the National Institute of Standards and Technology. Don had been wanting for some time to build a program to automate interactive Unix applications, but the project had stalled because it needed a good command language and Don didn't have the time to build one. After hearing my talk, Don rushed home, downloaded Tcl, built a wonderful application called Expect, and wrote a paper on it in time for the submission deadline for the next USENIX Conference. This whole process took just three weeks. Don made Expect freely available after presenting his paper in the summer of 1990, and it became an instant hit among system administrators. Expect was not only the first widely distributed Tcl application, but for many years it was the most popular. Many of the early Tcl users learned about it by using Expect.
The basic features of Tk became usable late in 1990, and I presented a paper on Tk at both the USENIX Conference and the X Conference in January 1991. Within a few weeks of these conferences I made the first Internet release of Tk. The first version of Tk lacked several important widgets (e.g., it had no multi-line text widget), but by the summer of 1992 I had added both a text widget and a powerful canvas widget. At this point usage of Tk began to increase dramatically.
Growing popularity
Usage of Tcl and Tk grew roughly an order of magnitude each year in the early 1990s, from a few users in 1989 to several tens of thousands in 1993. There were two primary reasons for this growth.
The first reason for the growth in Tcl/Tk usage was that it provided the easiest way to create graphical user interfaces under Unix (in those days Tcl only ran on Unix systems). Tk's GUI facilities were both very simple and very powerful; interesting applications could be written entirely in Tcl, without writing any C code at all. C-based alternatives such as the Motif toolkit were much more complicated, yet provided less functionality. It quickly became clear that GUIs could be created with 5-10x less effort in Tcl than Motif. It also turned out that many people in industry were just as concerned as I was about the high cost of developing interactive applications.
The second reason for the popularity of Tcl was its embeddable nature. When I started on Tcl I assumed that no-one besides myself would care about an embeddable command language, but I was wrong. Many people, including Don Libes and others, wanted to build applications that were scriptable and extensible but didn't have the time to build a language of their own. Because it was embeddable, Tcl provided a perfect solution for these people. As a result, Tcl ended up being used in applications ranging from stock trading to scientific visualization to manufacturing automation.
Out of this dichotomy in usage grew a discussion that reoccurs every year or two in the Tcl user community. Many Tcl users assert that Tk is the only reason they use Tcl. Some of them don't even like the Tcl language and wish that Tk could be mated with a different scripting language (in fact, this has happened several times, including TkPerl and Python). A different group argues that Tcl is all they care about; they use it as an embedded scripting language, often without Tk, and like the language's properties for this purpose. In a sense both groups are right.
The Tcl community
As Tcl usage grew in the early 1990s, a community formed around the language, contributing many powerful extensions and providing support for new users. Two of the earliest pioneers were Mark Diekhans and Karl Lehenbauer. They wrote TclX, which was one of the first freely available Tcl extensions. TclX provided file access, time and date manipulation, and many other useful facilities. Over time the features of TclX have proven so important that many of them have been incorporated into the base Tcl releases.
Many other Tcl pioneers wrote high-quality extensions and made them freely available. For example, Michael McLennan wrote [incr Tcl], an extension that provides object-oriented programming features in Tcl. George Howlett wrote BLT, which added several important widgets to Tk. Tom Poindexter wrote Sybtcl and Oratcl, which provided access to popular databases. Brian Smith and Larry Rowe wrote TclDP, which provided socket access and other distributed programming features. There are others as well; apologies to the many worthy contributors whom I have failed to recognize.
Other forms of community developed in this same period. I initially created a mailing list for communication among Tcl users, but within a few years it became so large as to be unwieldy. With help from the Tcl community the comp.lang.tcl newsgroup was created as a replacement for the mailing list; since then it has provided a wonderful forum for exchanging information and discussing new Tcl features.
In 1993 Larry Rowe recognized the need for people in the community to meet in person, so he organized the first Tcl Workshop at Berkeley that summer. About 60 people attended, and the workshop was so successful that it became an annual event. In 1995 the USENIX Association offered to help run the workshop; since then it has evolved into a full-fledged conference with several hundred attendees and a rich program of papers, tutorials, and other events. The Tcl Conference is also one of the world's premier events for upper body wear: each year there are 5-10 different T-shirt designs given away to recognize various extensions, events, etc.
During this period I made new releases of Tcl and Tk roughly every 6-12 months. Most of the new features were suggested by people in the Tcl community; in some cases users even supplied me with prototype code for the new features, which I then integrated into the core software. Typically each new release cycle would start off with a discussion period where I proposed a candidate list of new features and everyone in the community could vote for the ones they thought were most important. I tried to give priority to the features with the most votes.
Ouster-votes
Over time the Tcl release cycle tended to align with the Tcl Conference. A new release would typically come out around the time of the conference, and I would use one session of the conference to discuss new features for the next release and get feedback. I would often call for an impromptu show of hands to see how the audience felt about various issues, and then attempt to make a rough count of how many hands were raised ("ten, twenty, thirty, ... looks like about 35 hands to me"). These became known as "Ouster-votes" and their accuracy has often been a topic of discussion. Some wags have insinuated that my vote counting is less than totally objective....
The Sun years
In 1994 I decided to leave Berkeley for industry. I'd always wanted to spend part of my career in industry, and after 14 years in academia the time seemed right. After considering a variety of possibilities, I decided to accept an offer from Sun Microsystems. Eric Schmidt, Sun's Chief Technology Officer, and Bert Sutherland, Director of Sun Microsystems Laboratories, offered me the chance to build a team at Sun Labs and evolve Tcl into a universal scripting language for the Internet. I had always felt that Tcl would eventually need to have profitable businesses built around it in order to survive over the long term, and the Sun offer was a great opportunity to take Tcl to the next level.
I joined Sun in May of 1994 and began building a team of Tcl developers. Over the next three years the Sun Tcl team grew to a dozen members. Up until this point nearly every line of code in Tcl and Tk had been written by me personally, but over time more and more of the responsibility for Tcl code shifted to other members of the Sun Tcl team.
The additional resources provided by Sun allowed us to make major improvements to Tcl and Tk. Scott Stanton and Ray Johnson ported Tcl and Tk to Windows and the Macintosh, so that Tcl became an outstanding cross-platform development environment; today, more than two-thirds of Tcl downloads are for Windows. Jacob Levy and Scott Stanton overhauled the I/O system and added socket support, so that Tcl could easily be used for a variety of network applications. Brian Lewis built a bytecode compiler for Tcl scripts, which provided speedups of as much as a factor of 10x. Jacob Levy implemented Safe-Tcl, a powerful security model that allows untrusted scripts to be evaluated safely. Jacob Levy and Laurent Demailly built a Tcl plugin, so that Tcl scripts can be evaluated in a Web browser, and we created Jacl and TclBlend, which allow Tcl and Java to work closely together. We added many other smaller improvements, such as dynamic loading, namespaces, time and date support, binary I/O, additional file manipulation commands, and an improved font mechanism.
When I moved to Sun many people were worried that Tcl would become a proprietary language, and there were several "spirited discussions" on the comp.lang.tcl newsgroup. Fortunately, as part of my offer Sun had agreed that the core Tcl and Tk libraries would continue to be distributed freely in source form, and Sun honored this agreement faithfully. As new releases appeared with more and more improvements, it became clear that Sun's support for Tcl was a wonderful thing that allowed the language to evolve much more rapidly than it could have otherwise. The concerns were gradually replaced with enthusiasm for all the new features.
During these years Tcl usage continued to explode. By the end of 1997 there were hundreds of thousands of Tcl developers; the rate of Tcl downloads from the Sun FTP site increased from 2000 per week at the end of 1995 to more than 10000 per week by early 1998. It was clear to me that there were huge business opportunities around Tcl. It was also clear that the Tcl community had many needs that weren't being met by the open-source Tcl development. For example, there were almost no development tools for Tcl, and professional services such as training and support were only sporadically available. Many important extensions, such as ActiveX and CORBA support, still needed to be written.
Ajuba
In late 1997 I decided that the best thing for Tcl was to spin out of Sun and form a startup that could focus entirely on Tcl. In January 1998 I founded Scriptics (now Tcl) with Sarah Daniels, who became VP of Marketing and Sales. Within a month about half of the Sun Tcl team had joined us and begun development of TclPro, a set of development tools that would comprise our first product. TclPro 1.0 shipped in September of 1998. Its success in the marketplace is now fueling a major growth spurt at Ajuba. by the end of 1999 we expect to have nearly 50 employees.
Tcl has also taken over from Sun the role of developing the Tcl core facilities, and it will continue to release them freely just like Sun did. In 1998 Tcl made several patch releases for Tcl 8.0 to fix bugs and add small new features, such as a better support for the [incr Tcl] extension. In April 1999, Tcl made its first major open source release, Tcl/Tk 8.1. This release added Unicode support (for internationalization), thread safety (for multi-threaded server applications), and an all-new regular expression package by Henry Spencer that includes many new features as well as Unicode support. With the 8.1 release, Tcl became the first scripting language to have all the facilities needed for mission-critical enterprise applications.
In the future, Tcl will continue to develop both open source and commercial products. With this arrangement we hope to achieve the best of all worlds, with the open source core continuing to improve rapidly and Tcl products filling the gaps and providing the revenue to support both the free and product developments. The Tcl user community will continue to play an important role in the evolution of Tcl, as it always has.
ActiveState
This section added by Jeff Hobbs
Ajuba became in part the victim of its own success in creating enterprise class XML server software based on Tcl, and was acquired by Interwoven in October, 2000. Interwoven was not interested in open source software howevr, just the XML and server stuff. Jeff Hobbs, formerly the Tcl Ambassador at Scriptics/Tcl, moved to ActiveState (already known for their Perl products and development) to head their Tcl technologies group.
ActiveState now provides the full range of products and services related to Tcl development, in addition to being a very active participant in continuing the further development of the open source core.
Tcl: award-winning software
In the spring of 1998 I learned the wonderful news that Tcl had won two major awards. The first is the ACM Software System Award, awarded each year for "a software system that has had a lasting influence". Past winners of the award include such seminal systems as the TCP/IP protocols, the first spreadsheets, the first relational databases, the World-Wide Web, Unix, PostScript, and Smalltalk. The second award was the USENIX Software Tools User Group (STUG) Award, given each year in recognition of an outstanding software tool.
Vision for the future
Integration applications will continue to grow in importance as more and more businesses discover that their competitiveness depends on being able to make all of their computerized systems work together effectively. Tcl provides a uniquely great platform for building these integration applications. In the future, when CIOs are planning the future of their information infrastructure, I believe that the integration platform will have the same level of importance that the operating system platform and database platform have today. I hope that Tcl will be the premier integration platform, and Tcl the premier vendor of these platforms.