Mark Grosberg www.conman.org/projects/essays/memmgr.html
A common question I get asked is what to do about storagemanagement, in particular reclaiming memory from used objects. Thereare so many solutions, each with their own quirks, that its hard todecide what to do in a large and complex project. Some of thetechniques are:
- Manual storage management
- Lifetime pools
- Reference counting
- “Mark and sweep” garbage collection
- “Stop and copy” garbage collection
The most common technique, manual storage management, is also themost limited and difficult to implement. The only advantage of thistechnique is for small software there is no code space wasted forany kind of management code. This is only true until the complexityof all the error paths results in more code in calls to free thestorage than an automatic method would have wasted. Most programmersalso feel that this approach is the most efficient (it isn't).
For any kind of complex software storage management in the faceof errors and complex structures really becomes too difficult andcan cause major maintenance headaches. Some types of software(especially request-oriented systems) can divide allocations into“pools” of storage. In fact, the Apache webserver usesthis approach. In fact, pools can be allocated inside of other poolsforming a hierarchy of storage.
This results in a very efficient way of maintaining storage. Asingle operation can free an many allocations in a single operation.The downside to this approach is that it requires great programmerdiscipline to make sure pools get destroyed when appropriate andthat objects are allocated from pools with the correct lifetime.
Reference counting is a common approach in C++ programs becauseoperator overloading allows it to be implemented easily (but notefficiently). Each object holds a counter that indicates how manyother objects depend on this ones existence. When the counter isdecremented to zero, the object can be freed. Programs that make useof a generic data structure libraries can implement the majority ofthe reference counting logic in common code.
Reference counting and pooled allocation are not necessarilymutually exclusive. Pools can be used for allocations where thereference counting model doesn't fit or the cost is too high. Inthese cases the pool object can be reference counted. When no moreobjects are using the pool, its entire contents can beeliminated.
Another approach to combining reference counting and pools is touse pools as a mechanism to correct object counts in the face oferrors. In this case the pool doesn't handle the low level aspectsof storage allocation but instead functions more like a containerfor objects. When an error happens the pool will de-reference all ofthe temporary objects. Unless they are referenced by longer-liveddata structures they will all be reclaimed.
This approach is quite powerful but has a flaw. Referencecounting only works for non-circular data structures. As soon asdata structures have a cycle (two nodes depending on each other) itbecomes nearly impossible to release the storage of the nodes,efficiently. It is possible to mark the state of an object and tracepointers to determine cycles during a reference decrement, but it isvery inefficient to do on every reference.
Garbage collection uses the idea of tracing pointers to determinelive objects from the above reference counting problem. However, itdoes not do this every time an object is no longer used by another.Rather, the system runs until it is too low on free memory tocontinue. When the system needs to have storage reclaimed the maintask of the software is stopped and all of the active objects aretraversed. Any unreachable objects are discarded.
There are two main strategies for garbage collection. The first,mark and sweep, works the obvious way. Each object posses a one bitflag. During garbage collection each object that is reachable ismarked. Any objects that are not marked after all the reachableobjects are traversed are unreachable and therefore garbage. Thesweep phase frees each node.
Mark and sweep has a problem in that it requires two passes overall the active objects in the system. Furthermore it allows the heapto become very fragmented. Another approach to garbage collection,stop and copy, does not have some of these problems (but if suffersfrom others).
In stop and copy garbage collection the free space is dividedexactly in half. A global flag indicates which half of the freespace is used for allocations. When the current free space isexhausted the system must collect garbage. Each object is traveresedand copied into the second half of the free space. Any unreachableobject is not copied during the traversal. The end result is thatthe reachable objects are transferred to the other half of the heap.After each cycle the meaning of the halves is swapped.
The neat thing about stop and copy garbage collection is that asa side effect of collecting garbage the heap is also compacted(eliminating fragmentation). This means that allocations become veryefficient because there is no free list to be searched. The downside is that only half of available memory can be used at any onetime.
Many programmers are quick to believe that garbage collection isa panacea that solves all their allocation difficulties. Thiscouldn't be further from the truth. Garbage collection has severaldisadvantages. The biggest disadvantage is that garbage collectiondisturbs “locality of reference.” Techniques like cachesand virtual memory depend on the fact that not all data is accessedwith the same frequency. During a garbage collection cycle themajority of memory is not only examined but changed.
The next major problem with garbage collection is that it makesthe response time of a software system very unpredictable. Forreal-time systems or user interfaces this is not acceptable. Infact, in the case of real-time systems it can result in the failureof the system to meet a deadline.
A commonly stated problem of garbage collection is the so called“finalization problem.” When an object is to be destroyedit is often necessary to perform some action. For example, an objectmay contain an operating system resource or represent some hardware. This means that cleanup must be accompanied by a cleanup routine(called destructor in C++). This is very difficult because thecleanup routine can change the pointers while the garbage collectoris in the midst of traversing them.
One of the most obvious, but often ignored, problems is the needto be able to trace all the pointers in the system. This isn't aseasy as it first appears because garbage collection can happen atany time (in the case of multitasking). This means that it is notsufficient to trace just the pointers in global variables (which canbe easily identified). Rather, pointers on the stack(s) must also beidentified. What if, for example, an object is pointed to only by alocal variable as it is being moved (no globally reachable pointerspoint to the object)? The object could disappear while it is stillbeing used.
Another problem with garbage collection is that it makesprogrammers to complacent about memory allocation. Garbagecollection makes programmers believe that it is impossible for theircode to have memory leaks. Of course if code forgets to release areference to an object, that object will leak. I have seen thisvery bug in a Java application that a co-worker was developing. Theprogrammer kept adding an object that pointed to a huge treestructure (around a megabyte) to a list. The object in question wasa small widget that kept getting added to a focus list. Of course,that meant the structure was never freed.
The picture is not as bleak as I make it out to be. Newer garbagecollection algorithms can reduce the locality of reference problem.These newer algorithms (called “generational garbagecollection”) even allow garbage collection to be used in some(soft) real-time systems. Most actual implementation of garbagecollection are “conservative.” This means that they make abest guess, erring on the side of caution, about what values arepointers. Meaning that some garbage can end up never beingcollected. In systems demanding high uptimes this is clearlyunacceptable.
So what can be done about circular data structures? Thankfullycircular data structures are very rare. In fact, a lot of cases, allof the nodes that may contain circular pointers can be grouped intoa single data structure where the nodes are managed internally usinggarbage collection. Compartmental garbage collection like this ismore efficient than garbage collection of the entire system. It isalso more easily implemented.
The most important point to all this is that there is no onememory management solution that can meet all the requirements of acomplex software system. What is evident is that software shouldn'trely on a completely manual approach, it is tedious and easy to makemistakes. Fully automatic systems often have performance problemsand can't handle complex object cleanups.
A complex system has many data structures, some very specific tothe application and others being generic (linked lists, trees, hashtables, etc). Different techniques are applicable to different datastructures. Memory management should be built into the foundationof the system as much as possible. Codifying the memory managementlogic into generic libraries of techniques as much as possible.
