A partial solution to the software code usability problem via encapsulation and low level documentation guidelines


Commercial software code tends to reflect the goals of commercial software development. Get the result as fast as possible, as cheaply as possible, and as efficiently as possible. At first glance, it appears this goal can be met by sacrificing noncritical concerns such as good encapsulation, good structure, documentation, low level comments, and general readability. Were code to always be written perfectly the first time I would agree wholly noncritical factors are entirely that. However, that is never the case. Commercial code is typically enhanced, modified, and sometimes just completed over a period of several years after the initial product shipment. Not surprisingly, commercial code which was confusing to begin with rapidly degenerates into patchwork. Dangling ends, apparently meaningless objects, cryptic naming, structure shortcuts, and obvious bugs become self-begetting. This is only worse when programmers other than the original author(s) have to work on it. Since virtually all consumer software requires more than one programmer to create, it is just as important that software code be as easy to understand and analyze as it to run correctly. Here I present three partial solutions to the problem of software code usability: complexity hiding, stand-alone comments and documentation, and the use of an intuitive naming scheme. While these solutions MAY increase the amount of time needed to produce code in the short run, they will benefit code maintenance for the life of the product and ultimately save money

The so-called Software Crisis originated from the difficulty of writing increasingly complex programs, especially the cost of maintaining them. It lead to the development of C++, the primary object orientated development language used commercially today. In general, object orientated programming methods can reduce the complexity of the material a programmer has to be concerned with. Encapsulation, a goal of object orientated programming, is especially important as it defines the scope of the material the programmer has to learn. The potential benefit of encapsulation is that a new programmer can learn less but accomplish the same tasks.

It's important to note that the benefit of encapsulation is potential and nothing more. Any object orientated programming language leaves the level of encapsulation up to the programmer. Simply using encapsulation does not guarantee a program will be less complex. One could define scope, but that scope may still be the entire program. Code segments could be encapsulated, but those segments may still be individual functions. As before, complexity was defined by the programmer with no agreed standard level of complexity. In general it takes more work to encapsulate code segments at a higher level so it tends to be left at a low level in commercial programming.

Documentation is another significant way the complexity of code can be reduced. Documentation falls in three categories: high level documentation, code comments, and descriptive variable names. High level documentation techniques are well defined including that from the International Organization for Standardization under subcategory Information technology / Software Engineering / System software documentation. Low level code commenting is left to the programmer and is not standardized except perhaps by individual companies or programmers. Comments may range from clear and descriptive to spare to nonexistent. Object names compose an element of documentation and are likewise not standardized.

More comments tend to increase code clarity while better comments by definition do. However, it takes a substantial amount of time to write comments and it typically takes more thought, typing, and hence time to write better comments. Similarly, shorter object names require less time to type but are less descriptive unless the reader already knows what the name represents. Object names that are shorter than the full word or poorly chosen names present a risk of misinterpretation, decreasing readability. However, longer names take longer to write, i.e. it takes more work to write longer variable names.

Software is functional as soon as all code is written and debugged. Proper encapsulation requires additional initial time. Producing good low level comments, function block documentation, and choosing good object names require additional time both during and after coding. Therefore, it is against the programmer's immediate interest to create clear code while in his or her immediate interest to write code which is harder to read, learn, debug, modify, and trace.

According to Dianna Mullet


Why is it that the team produces fewer than 10 lines of code per day over the average lifetime of the project? And why are sixty errors found per every thousand lines of code? Why is one of every three large projects scrapped before ever being completed? And why is only 1 in 8 finished software projects considered "successful?"
...
The cost of owning and maintaining software in the 1980ís was twice as expensive as developing the software.
During the 1990ís, the cost of ownership and maintenance increased by 30% over the 1980ís.
In 1995, statistics showed that half of surveyed development projects were operational, but were not considered successful.
The average software project overshoots its schedule by half.
Three quarters of all large software products delivered to the customer are failures that are either not used at all, or do not meet the customerís requirements.

Given that a great deal of time and expense goes towards software maintenance, it is imperative that software be as easy to maintain as possible. It is generally accepted that the time savings gained by producing poorly documented and encapsulated code are lost many times over by the extra time required to perform maintenance. This point is illustrated later in this paper where I have a series of programmers analyze a published shell sort algorithm.

My solution to the problem of poor software code readability consists of three parts:
  1. Using multiple levels of encapsulation.
  2. Adhering to a clear comment principle.
  3. Using descriptive variable names.

Multiple levels of encapsulation

The concept of information hiding is to allow the user to use functions without being aware of their underlying implementation. The concept of encapsulation is to group related methods within a scope such that the encapsulated object will always work and be independent of other conditions as long as its preconditions are met. Encapsulation typically has the additional requirement of applying information hiding in order to present a scoped interface of methods whose use does not require knowledge of the underlying implementation. Multiple levels of encapsulation, which I propose here, is the application of information hiding with regard to dependent functions. If function B has a direct or indirect precondition that function A has been previously executed then this condition is hidden within a function C, where function C executes both functions A and B in the appropriate order and manages the interface between the two. Any necessary parameters are passed to C. Certain assumptions may need to be met in C, these should be satisfied with a default condition and this should be clearly stated in the documentation. Function A and B remain callable by the user who then assumes responsibility that the preconditions are met. In this way, every top-level function of the encapsulated object can be called independently of every other object. The goal is to provide the user with the minimal possible interface that still encompasses all member functions, while at the same time allowing use of the full feature set of the object when needed.

Structure of an object using multiple levels of encapsulation:

/*
High level documentation - i.e. header
Common usage example(s)
*/
[Initialization Routine]
[Independent Member function | Independent Member function set 1]
[Independent Member function | Independent Member function set 2]
...
[Independent Member function | Independent Member function set n]
[Termination Routine]

Independent Member function set is composed of:
Independent Member function | Independent Member function set 1
Dependent Member function | Dependent Member function set 2
[Dependent Member function | Dependent Member function set 3]
[...]
[Dependent Member function | Dependent Member function set m]

Independent functions are those that have no preconditions which can only be satisfied by a member function of the object. Dependent functions are functions that have preconditions which require the prior call of a member function of the set. In both cases this is excluding the Initialization Routine.

Common usage example(s) is worth special note. Complete, compilable, stand alone code example(s) should always be provided to illustrate the most common usage of the object set. Code fragments should be avoided because they assume the user understands the context in which the object is used. That may not necessarily be true. If the example code is long enough to decrease readability it can be off-loaded, for example put in a separate file. The example documentation should also state any assumed global preconditions, for example compiler settings or system requirements.

Stand alone comments and a clear comment principle

Comments written by the programmer tend to reflect the program writer's knowledge of the program at the particular time the comments are written. It is usually not true that the comment reader will have a knowledge of the program equal to or greater than the knowledge of the programmer at that time. Therefore, there exists a risk of 'false assumptions', where the programmer will write a comment that cannot be readily understood without certain prerequisite knowledge of the program.

The standard user can be assumed to have complete knowledge of the core language. The core language is the standard set of keywords, operators, and methods that define the language. No other assumptions should be made.

Code comments should be stand-alone, complete, and clear in both function and context. The assumption can be made that the reader is the standard user plus has a function specific knowledge set stated in the function header. No other assumptions should be made unless stated in the high level documentation to apply to that function or all functions of that type. This should be mentioned in the function header.

Example:

High Level Documentation
The user is assumed to have a knowledge of the BinaryTree object member functions in the class AVLBinaryTree

Class Definition
...

Member function Definition

/*
This function assumes the user has a knowledge of the BinaryTree object member functions
...
OR
Refer to high level documentation section x.y.z for comment prerequisites
...

Additional comment prerequisites here
...
*/

...

Commenting should minimally be included at three locations in the code: within the function, before the function definition, and before the object declaration. Comments within the function should serve to clarify confusing statements or difficult to follow sections of code, such as nested loops. Comments before the function definition should present a summary of what the function does, what preconditions are necessary for it to execute correctly, and return values or postconditions. Comments before the object declaration should minimally give a high level overview of the purpose of the object, usage examples, and user notes. Other information, such as preconditions and postconditions, should be included as needed.

Intuitive naming scheme

It is a practice of programmers to use acronyms to represent names in order to reduce typing. This suffers from the problem of 'false assumptions' also found with code comments. An acronym always has a chance of being misinterpreted. Therefore, with the exception where the acronym for an object is synonymous with its name, the full word should be spelled out. The first letter of every word, except the first word, should be capitalized. If a prefix is used, such as with Hungarian notation, the first letter of the first word should be opposite case the prior letter or lower case if the prior character is not a letter. In cases of naming ambiguity, it is up to the programmer to use whatever scheme would be more clear to the standard user. If there still exists ambiguity, this should be clarified with comments.

For example, the G. M. Adel'son-Velskii and E. M. Landis tree, mentioned at http://www.cs.oberlin.edu/classes/dragn/labs/avl/avl5.html, is virtually always called an AVL Tree so spelling the full name out would reduce readability. However, ptr may represent pointer, parameter, or partner should it should be spelled out fully.

One argument against fully spelled out object names is that they take extra time to write. The implicit argument is that this decreases the rate of code production. While true for straightforward code, the time savings are minimal since the majority of a programmer's time is spent in analysis and debugging rather than typing. Whatever time might have been saved can be lost many times over if the code has to be learned or relearned in the future. For example, when the code has to be modified by a programmer other than the creator, or modified when the original creator has forgotten enough of the code for it to be become obscure. This is also the case when the code needs to be analyzed for usage, for example if it is undocumented, not sufficiently documented, or the documentation is poorly written. Fully spelled out object names serve as implicit documentation. They reduces the amount of time required to relearn the code and reduce the chance of creating bugs during modification. For example, the undocumented / terse version of the shell sort algorithm (see below) took participants from 22 to 47 minutes to analyze with the best common conclusion being that it simply is a sort. That's a trivial conclusion since 'sort' is the name of the function. It took me roughly twice as long to retype the algorithm with fully spelled out variable names AND write the comments. Every participant took less time to analyze the second code block, in one case the participant correctly identified and analyzed the code because of the documentation.

Hungarian Notation, created by Charles Simonyi, has the benefit of specifying type within the variable name itself. It has one benefit: the programmer can tell what type a name is without having to look for it. It has three problems: it is ambiguous, it is not necessarily intuitive, it is not universally accepted (i.e. non-portable). It should be used when it will enhance code clarity from the perspective of the standard user and not otherwise. In almost all cases proper commenting will eliminate the need for Hungarian notation.

From the website "a partial example of an actual symbol table routine".

18      for (; *pbsy!=0; pbsy = &psy->bsyNext)
19         {
20         char *szSy;
21         szSy= (psy=(struct SY*)&rgwDic[*pbsy])->sz;
22         pch=sz;
23         while (*pch==*szSy++)
24            {
25            if (*pch++==0)
26               return (psy);
27            }
28         }

See appendix for full code

In my opinion the naming scheme is nearly incomprehensible.

Enumerations of pure types should be avoided in favor of a proper variable naming scheme. The standard user can be assumed to understand the pure types. The use of enumerations increases the amount of material to remember, thus decreasing readability, but does not contribute to code functionality. For example, it is easier to remember unsigned char than it is to remember UCHAR, commonly used by Microsoft. UCHAR is redundant at best and subject to misinterpretation as are all enumerations of pure types.

Enumerations or compiler replacement commands (such as #define in C) should be used to replace Magic numbers, defined by the Jargon Dictionary, in all cases. They should follow the naming scheme proposed in this thesis except where a language specific standard already exists (such as all capital letters for #define in C).

There is no absolute measure of name clarity and it would be impractical to perform statistical surveys. As with all documentation, it is the programmer's responsibility to use what is most clear given the context.

To test my proposed naming and commenting scheme, I gave a shell sort algorithm, published in Data Structures & Algorithm Analysis in C (58) with minor modifications to C students and programmers via the Internet. Given half an hour, they were to analyze the code and write a conclusion on what the code did as well as comments about the writing style. For part two, they were to analyze the same code but commented and named according to my proposed guidelines.

I renamed the function so readers would not automatically know it is a shell sort and removed an enumeration to improve clarity. The function is otherwise the same.


void sort(int *A, int n)
{
    int i,j, increment, tmp;
    
    for (increment=n/2; increment>0; increment/=2)
        for (i=increment; i < n; i++)
        {
            tmp=A[i];
            for (j=i; j>=increment; j-=increment)
                if (tmp < A[j-increment])
                    A[j] = A[j-increment];
                else
                    break;
            A[j]=tmp;
        }
}

This is the code for part two. It is functionally the same but uses my proposed naming and documentation technique. It does not have the function header as that would defeat the purpose of the experiment.


void sort(int *Array, int ArrayLength)
{
    int OuterLoopCounter, InnerLoopCounter, Increment, Temporary;
    
    /* This loop determines the size of the increment and reduces it by half each iteration */
    for (Increment=ArrayLength/2; Increment>0; Increment/=2)
     /* OuterLoopCounter points to index Increment in Array.  It is increased by one, i.e. moves right, until the end of the array */
        for (OuterLoopCounter=Increment; OuterLoopCounter < ArrayLength; OuterLoopCounter++)
        {
            /* Save the first element at index Increment (i.e. OuterLoopCounter) of Array in the variable Termporary */
            Temporary=Array[OuterLoopCounter];
            
             /* Moving LEFT, perform the inner loop on every element in the Array Increment elements apart.
                Note for the first iteration OuterLoopCounter starts at index Increment and the value of the array at this index is stored in Temporary */
            for (InnerLoopCounter=OuterLoopCounter; InnerLoopCounter>=Increment; InnerLoopCounter-=Increment)
            {
                 /* Compare Temporary with its neighbor to the left */
                if (Temporary < Array[InnerLoopCounter-Increment])
                 /* If the element to the left is greater, overwrite the current element with that.  Note the loop then points to the element to the left of that, in effect shifting all elements right */
                    Array[InnerLoopCounter] = Array[InnerLoopCounter-Increment];
                else
                 /* If the element to the left is greater, then stop */
                    break;
            }
            
            /* Put the Temporary back in the array.
              The result is that the array is shifted to the right as long as Temporary (the initial rightmost element) < each of the elements to the left of it, and temporary is reinserted (moved) to the left edge of the block shifted. */
            Array[InnerLoopCounter]=Temporary;
        }
}

Comments from the participants:

Matt Harris
3rd year software engineering student at Cogswell

22 minutes to present conclusion for part 1

A binary sort algorithm that sorts an integer array pointed to by *A and defined in size by n.

A = [1 4 3 6 3 3 0 3 5]
n = 9
pass 1: 
inc = 4
i = 4 to 8
inner pass 1:
	tmp = A[4] (3)
	The inner for loop (with j) is to cover values of i that aren't equal to inc.
	a[j(4)] = tmp (3)
	
	tmp = A[5] (3)
	for (j=5; j>=4; j-=4)
		if (3 < 4)
			A[5] = A[1];

This code is obfuscated. I give.

8 minutes to present conclusion for part 2

This function performs a shell sort on array (*Array) of length (ArrayLength). The sort is performed on ArrayLength/2 decreasing increments. Each nth element is sorted on its own, then the increment is halved. rinse and repeat until you sort the whole damn thing.

He was correctly able to give more specific details when requested:

0,5,10,15 as one group, 1,6,11 as another, 2,7,12 as a third, etc. then 0,2,4,6,8,10,12,14 as one, 1,3,5,7,9,11,13,15 as another, then the whole group.

Beautiful. Clear. Concise. Kernel-worthy.

Robert Litscher
5th year CS student at Ohio State University

30 minutes to present conclusion for part 1

The result seems to be a sorted array with the first element of the
array being the lowest value and the last element being the highest
value. It will begin by comparing the middle element to the first. Then
the (middle + 1) element will be compared to the second and first
element. Next, the (middle + 2) element will be compared to the third,
second and first elements.

In any of these cases, if the comparison is true, the (middle + i)
element is swapped with the value it is being compared to.

As for comments on coding style, nested for loops never appealed to me.
They seem more difficult to follow than while loops.


24 minutes to present conclusion for part 2

No additional comments on functionality.

Simply putting curly brackets around the code for the "for
(InnerLoopCounter=OuterLoopCounter; InnerLoopCounter>=Increment;
InnerLoopCounter-=Increment)" block makes it much more readable. There
is absolutely no ambiguity as to what you are breaking out of when
"break;" is encountered.

The use of meaningful variable names also helps. More so for the
incoming variables Array and ArrayLength. The other modified variable
names simply makes variables easier to keep track of, especially if you
are going to put the code aside for a long period of time.

The comments in the code help, much in the same way that meaningful
variable names help. If you are looking at the code for the first time,
they help reinforce what you believe the code is doing.


Mike Carpenter
Seattle, WA
Firewall Engineer

47 minutes to present conclusion for part 1

It's a bubble sort function with very little clarity. C allows us to use longer names without consuming memory (unlike older languages).

Instead of void sort (int *A, int n) we can use void sortIt (int *sortStr, int length).

It is readable and if I had run into the snippet in the middle of some other code (and knew what was calling it), it may have been easier to read it.

Some people suggest commenting frequently, but in this case, a short comment at the top may not have added any more value than just changing the variable names. I refrain from using vars like 'i' and 'j' unless it's a very simple loop.

It makes it easier to use concise var names when programming because when I have to come back and fix a bug or add functionality, 6 months, 12 months, or 2 years later, I don't have to 'read' very far before 'remembering' what the variables, structures, pointers, functions are etc.

30 minutes to present conclusion for part 2

The updated function sort() is much more understandable, however, does add quite a bit of bulk in the comments area. A brief comment at the top using /* comments */ would probably suffice rather than comments at each loop or stage of the code, imho.

If it were a college text.. I might use the detailed information, but just coding for personal use or work, I'd just use a brief statement at the top, and assume whomever is working on the code will be able to figure it out.

Conclusions:

  1. Encapsulation does not necessarily hide information. Encapsulation does not necessarily make a object or set of objects easier to use or modify.
  2. Comments sometimes make false assumptions about the reader's knowledge. When that is the case comments have a greater chance of misinterpretation.
  3. Name acronyms can be ambiguous and hard to read. Full names are less ambiguous and easier to read.
  4. Hungarian notation suffers from the same problem as name acronyms and is not portable. It should be used when it will increase code clarity and not otherwise. It should not replace good commenting.
  5. A great deal of time and money is spent on software maintenance. Poorly written code is increasingly more expensive to maintain and produces buggier results.

Summary of Contributions

  1. My proposed system of multiple levels of encapsulation ensures information hiding is enforced and makes objects simpler to use and modify.
  2. My proposed system of commenting decreases the possibility of comment misinterpretation.
  3. My proposed system of object naming improves code readability and helps avoid ambiguity.
  4. Taken together, my proposed solutions decrease the chance of bugs, decrease learning time, and decrease the costs related with software maintenance.

Appendix

From Charles Simonyi, copied from http://msdn.microsoft.com/library/techart/hunganotat.htm
"a partial example of an actual symbol table routine"

Used to illustrate the ambiguity of Hungarian notation.

1   #include ďsy.hĒ
2   extern int *rgwDic;
3   extern int bsyMac;
4   struct SY *PsySz(char sz[])
6      {
7      char *pch;
8      int cch;
9      struct SY *psy, *PsyCreate();
10      int *pbsy;
11      int cwSz;
12      unsigned wHash=0;
13      pch=sz;
14      while (*pch!=0
15         wHash=(wHash<>11+*pch++;
16      cch=pch-sz;
17      pbsy=&rgbsyHash[(wHash&077777)%cwHash];
18      for (; *pbsy!=0; pbsy = &psy->bsyNext)
19         {
20         char *szSy;
21         szSy= (psy=(struct SY*)&rgwDic[*pbsy])->sz;
22         pch=sz;
23         while (*pch==*szSy++)
24            {
25            if (*pch++==0)
26               return (psy);
27            }
28         }
29      cwSz=0;
30      if (cch>=2)
31         cwSz=(cch-2/sizeof(int)+1;
32      *pbsy=(int *)(psy=PsyCreate(cwSY+cwSz))-rgwDic;
33      Zero((int *)psy,cwSY);
34      bltbyte(sz, psy->sz, cch+1);
35      return(psy);
36      }

Works Cited

AVL Balanced Trees. Oberlin College Computer Science Department. 11 January 2001 < http://www.cs.oberlin.edu/classes/dragn/labs/avl/avl5.html >

Charles Simonyi. Hungarian Notation. Microsoft Developer Network. 11 January 2001 < http://msdn.microsoft.com/library/techart/hunganotat.htm >

Dianna Mullet. The Software Crisis. University of North Texas. 11 January 2001 < http://www.unt.edu/benchmarks/archives/1999/july99/crisis.htm >

ISO Homepage. The International Organization for Standardization. 11 January 2001 < http://www.iso.ch/ >

magic number [The Jargon Dictionary]. The Jargon Dictionary. 11 January 2001 < http://info.astrian.net/jargon/terms/m/magic_number.html >

Mark Allen, Weiss. Data Structues & Algorithm Analysis in C Atlanta, GA: Addison Wesley, 1996

Matt Harris. "Code Analysis Experiement." E-mail to the author. 12 January 2001.

Mike Carpenter. "Code Analysis Experiement." E-mail to the author. 11 January 2001.

Robert Litscher. "Code Analysis Experiement." E-mail to the author. 12 January 2001.