-------------------------------------------------------------------------------

How to Install the Library
==========================

   To configure, compile, and install the library, follow these steps
carefully.

  1. Make sure you have the latest version of the library available
     from the home page `http://dmalloc.com/'.

  2. You may want to edit or at least review the settings in
     `settings.dist' to tune specific features of the library.  The
     `configure' script will copy this file to `settings.h' which is
     where you should be adding per-architecture settings.

  3. Type `sh ./configure' to configure the library.  You may want to
     first examine the `config.help' file for some information about
     configure.  You may want to use the `--disable-cxx' option if you
     do not want the Makefile to build the C++ version of dmalloc.  You
     may want to use the `--enable-threads' option to build the
     threaded version of dmalloc.  You may want to use the
     `--enable-shlib' option to build the shared versions of the
     dmalloc libraries.  `sh ./configure --help' lists the available
     options to configure.  Configure should generate the `Makefile'
     and configuration files automatically.

  4. You may want to examine the `Makefile' and `conf.h' files created
     by configure to make sure it did its job correctly.

  5. You might want to tune the settings in `settings.h' file to tune
     the library to the local architecture.  This file contains relevant
     settings if you are using pthreads or another thread library.
     *Note Using With Threads::.  The `configure' script created this
     file from the `settings.dist' file.  Any permanent changes to these
     settings should made to the `settings.dist' file.  You then can run
     `config.status' to re-create the `settings.h' file.

  6. The `DMALLOC_SIZE' variable gets auto-configured in `dmalloc.h.2'
     but it may not generate correct settings for all systems.  You may
     have to alter the definitions in this file to get things to stop
     complaining when you go to compile about the size arguments to
     malloc routines.  Comments on this please.

  7. Typing `make' should be enough to build `libdmalloc.a', and
     `dmalloc' program.  If it does not work, please see if there are
     any notes in the contrib directory about your system-type.  If not
     and you figure your problem out, please send me some notes so
     future users can profit from your experiences.

     _NOTE_: You may experience some errors compiling some of the
     return.h assembly macros which attempt to determine the callers
     address for logging purposes.  You may want to first try disabling
     any compiler optimization flags.  If this doesn't work then you
     may need to disable the `USE_RETURN_MACROS' variable in the
     `settings.h' file.

     _NOTE_: The code is dependent on an ANSI-C compiler.  If the
     configure script gives the `WARNING' that you do not have an ANSI-C
     compiler, you may still be able to add some sort of option to your
     compiler to make it ANSI.  If there such is an option, please send
     it to the author so it can be added to the configure script.

  8. If you use threads and did not add the `--enable-threads' argument
     to configure, typing `make threads' should be enough to build
     `libdmallocth.a' which is the threaded version of the library.
     This may or may not work depending on the configuration scripts
     ability to detect your local thread functionality.  Feel free to
     send me mail with improvements.

     See the "Using With Threads" section for more information about the
     operation of the library with your threaded program.  *Note Using
     With Threads::.

  9. If you have a C++ compiler installed, the library should have
     automatically built `libdmallocxx.a' which is the C++ version of
     the library.  If it was not done automatically, you can build it by
     typing `make cxx'.  You should link this library into your C++
     programs instead of `libdmalloc.a'.  See the `dmallocc.cc' C++
     file which contains basic code to overload the `new', `new[]',
     `delete', and `delete[]' C++ operators.  My apologies on the
     minimal C++ support.  I am still living in a mostly C world.  Any
     help improving this interface without sacrificing portability
     would be appreciated.

 10. Typing `make light' should build and run the `dmalloc_t' test
     program through a set of light trials.  By default this will
     execute `dmalloc_t' 5 times - each time will execute 10,000 malloc
     operations in a very random manner.  Anal folks can type `make
     heavy' to up the ante.  Use `dmalloc_t --usage' for the list of all
     `dmalloc_t' options.

 11. Typing `make install' should install the `libdmalloc.a' library in
     `/usr/local/lib', the `dmalloc.h' include file in
     `/usr/local/include', and the `dmalloc' utility in
     `/usr/local/bin'.  You may also want to type `make installth' to
     install the thread library into place and/or `make installcc' to
     install the C++ library into place.

     You may have specified a `--prefix=PATH' option to configure in
     which case `/usr/local' will have been replaced with `PATH'.


   See the "Getting Started" section to get up and running with the
library.  *Note Getting Started::.


-------------------------------------------------------------------------------

Getting Started with the Library
================================

   This section should give you a quick idea on how to get going.
Basically, you need to do the following things to make use of the
library:

  1. Make sure you have the latest version of the library available
     from the home page `http://dmalloc.com/'.

  2. Follow the installation instructions on how to configure, make,
     and install the library (i.e. type: `make install').  *Note
     Installation::.

  3. You need to make sure that the dmalloc building process above was
     able to locate one of the the `on_exit' or `atexit' functions.  If
     so, then the dmalloc library should be able to automatically call
     `dmalloc_shutdown' when `exit' is called.  This causes the memory
     statistics and unfreed information to be dumped to the log file.
     However, if your system has neither, you will need to call
     `dmalloc_shutdown' yourself before your program exits.

  4. Add an alias for dmalloc to your shell's rc file if supported.
     Bash, ksh, and zsh users should add the following to their
     `.bashrc', `.profile', or `.zshrc' file respectively (notice the
     `-b' option for bourne shell output):

          function dmalloc { eval `command dmalloc -b $*`; }

     If your shell does not support the `command' function then try:

          function dmalloc { eval `\dmalloc -b $*`; }
     or
          function dmalloc { eval `/usr/local/bin/dmalloc -b $*`; }

     By the way, if you are looking for a shell, I heartily recommend
     trying out zsh.  It is a bourne shell written from scratch with
     much the same features as tcsh without the csh crap.  If you are
     _still_ using csh or tcsh, you should add the following to your
     `.cshrc' file (notice the `-C' option for c-shell output):

          alias dmalloc 'eval `\dmalloc -C \!*`'

  5. Although not necessary, you may want to include `dmalloc.h' in
     your C files and recompile.  This will allow the library to report
     the file/line numbers of calls that generate problems.  *Note
     Allocation Macros::.  It should be inserted at the _bottom_ of
     your include files as to not conflict with other includes.  You
     may want to ifdef it as well and compile with `cc -DDMALLOC ...':

          /* other includes above ^^^ */
          
          #ifdef DMALLOC
          #include "dmalloc.h"
          #endif

  6. Link the dmalloc library into your program.  The dmalloc library
     should probably be placed at or near the end of the library list.

  7. Enable the debugging features by typing `dmalloc -l logfile -i 100
     low' (for example).  This will:

        * set the malloc log path to `logfile' (`-l logfile')

        * have the library check itself every 100 iterations (`-i 100')

        * enable a number of debug features (`low').  You can also try
          `runtime' for minimal checking or `medium' or `high' for more
          extensive heap verification.


     `dmalloc --usage' will provide verbose usage info for the dmalloc
     program.  *Note Dmalloc Program::.

     You may also want to install the `dmallocrc' file in your home
     directory as `.dmallocrc'.  This allows you to add your own
     combination of debug tokens.  *Note RC File::.

  8. Run your program, examine the logfile that should have been
     created by dmalloc_shutdown, and use its information to help debug
     your program.  See the next section for help with this.  *Note
     Troubleshooting::.



File: dmalloc.info,  Node: Allocation Basics,  Next: Features,  Prev: Getting Started,  Up: Overview

Basic Description of Terms and Functions
========================================

* Menu:

* Basic Definitions::           General memory terms and concepts.
* Malloc Functions::            Functionality supported by all malloc libs.


File: dmalloc.info,  Node: Basic Definitions,  Next: Malloc Functions,  Prev: Allocation Basics,  Up: Allocation Basics

General Memory Terms and Concepts
---------------------------------

   Any program can be divided into 2 logical parts: text and data.
Text is the actual program code in machine-readable format and data is
the information that the text operates on when it is executing.  The
data, in turn, can be divided into 3 logical parts according to where
it is stored: "static", "stack", and "heap".

   Static data is the information whose storage space is compiled into
the program.

     /* global variables are allocated as static data */
     int numbers[10];
     
     main()
     {
             ...
     }

   Stack data is data allocated at runtime to hold information used
inside of functions.  This data is managed by the system in the space
called stack space.

     void foo()
     {
             /* this local variable is stored on the stack */
             float total;
             ...
     }
     
     main()
     {
             foo();
     }

   Heap data is also allocated at runtime and provides a programmer with
dynamic memory capabilities.

     main()
     {
             /* the address is stored on the stack */
             char * string;
             ...
     
             /*
              * Allocate a string of 10 bytes on the heap.  Store the
              * address in string which is on the stack.
              */
             string = (char *)malloc(10);
             ...
     
             /* de-allocate the heap memory now that we're done with it */
             (void)free(string);
             ...
     }

   It is the heap data that is managed by this library.

   Although the above is an example of how to use the malloc and free
commands, it is not a good example of why using the heap for runtime
storage is useful.

   Consider this: You write a program that reads a file into memory,
processes it, and displays results.  You would like to handle files with
arbitrary size (from 10 bytes to 1.2 megabytes and more).  One problem,
however, is that the entire file must be in memory at one time to do the
calculations.  You don't want to have to allocate 1.2 megabytes when you
might only be reading in a 10 byte file because it is wasteful of system
resources.  Also, you are worried that your program might have to handle
files of more than 1.2 megabytes.

   A solution: first checkout the file's size and then, using the
heap-allocation routines, get enough storage to read the entire file
into memory.  The program will only be using the system resources
necessary for the job and you will be guaranteed that your program can
handle any sized file.


File: dmalloc.info,  Node: Malloc Functions,  Prev: Basic Definitions,  Up: Allocation Basics

Functionality Supported by All Malloc Libraries
-----------------------------------------------

   All malloc libraries support 4 basic memory allocation commands.
These include "malloc", "calloc", "realloc", and "free".  For more
information about their capabilities, check your system's manual pages
- in unix, do a `man 3 malloc'.

 - Function:
     void *malloc ( unsigned int SIZE )

     Usage: `pnt = (type *)malloc(size)'

     The malloc routine is the basic memory allocation routine.  It
     allocates an area of `size' bytes.  It will return a pointer to
     the space requested.


 - Function:
     void *calloc ( unsigned int NUMBER, unsigned int SIZE )

     Usage: `pnt = (type *)calloc(number, size)'

     The calloc routine allocates a certain `number' of items, each of
     `size' bytes, and returns a pointer to the space.  It is
     appropriate to pass in a `sizeof(type)' value as the size argument.

     Also, calloc nulls the space that it returns, assuring that the
     memory is all zeros.


 - Function:
     void *realloc ( void *OLD_PNT, unsigned int NEW_SIZE )

     Usage: `new_pnt = (type *)realloc(old_pnt, new_size)'

     The realloc function expands or shrinks the memory allocation in
     `old_pnt' to `new_size' number of bytes.  Realloc copies as much
     of the information from `old_pnt' as it can into the `new_pnt'
     space it returns, up to `new_size' bytes.  If there is a problem
     allocating this memory, 0L will be returned.

     If the `old_pnt' is 0L then realloc will do the equivalent of a
     `malloc(new_size)'.  If `new_size' is 0 and `old_pnt' is not 0L,
     then it will do the equivalent of `free(old_pnt)' and will return
     0L.


 - Function:
     void free ( void *PNT )

     Usage: `free(pnt)'

     The free routine releases allocation in `pnt' which was returned by
     malloc, calloc, or realloc back to the heap.  This allows other
     parts of the program to re-use memory that is not needed anymore.
     It guarantees that the process does not grow too big and swallow a
     large portion of the system resources.


   _NOTE_: the returned address from the memory allocation/reallocation
functions should be cast to the appropriate pointer type for the
variable being assigned.  Also this may no longer be necessary on your
system, it can still provide some documentation value to the code.

   _WARNING_: there is a quite common myth that all of the space that
is returned by malloc libraries has already been cleared.  _Only_ the
`calloc' routine will zero the memory space it returns.


File: dmalloc.info,  Node: Features,  Next: How It Works,  Prev: Allocation Basics,  Up: Overview

General Features of the Library
===============================

   The debugging features that are available in this debug malloc
library can be divided into a couple basic classifications:

file and line number information
     One of the nice things about a good debugger is its ability to
     provide the file and line number of an offending piece of code.
     This library attempts to give this functionality with the help of
     "cpp", the C preprocessor.  *Note Allocation Macros::.

return-address information
     To debug calls to the library from external sources (i.e. those
     files that could not use the allocation macros), some facilities
     have been provided to supply the caller's address.  This address,
     with the help of a debugger, can help you locate the source of a
     problem.  *Note Return Address::.

fence-post (i.e. bounds) checking
     "Fence-post" memory is the area immediately above or below memory
     allocations.  It is all too easy to write code that accesses above
     or below an allocation - especially when dealing with arrays or
     strings.  The library can write special values in the areas around
     every allocation so it will notice when these areas have been
     overwritten.  *Note Fence-Post Overruns::.

     _NOTE_: The library cannot notice when the program _reads_ from
     these areas, only when it writes values.  Also, fence-post checking
     will increase the amount of memory the program allocates.

heap-constancy verification
     The administration of the library is reasonably complex.  If any
     of the heap-maintenance information is corrupted, the program will
     either crash or give unpredictable results.

     By enabling heap-consistency checking, the library will run
     through its administrative structures to make sure all is in
     order.  This will mean that problems will be caught faster and
     diagnosed better.

     The drawback of this is, of course, that the library often takes
     quite a long time to do this.  It is suitable to enable this only
     during development and debugging sessions.

     _NOTE_: the heap checking routines cannot guarantee that the tests
     will not cause a segmentation-fault if the heap administration
     structures are properly (or improperly if you will) overwritten.
     In other words, the tests will verify that everything is okay but
     may not inform the user of problems in a graceful manner.

logging statistics
     One of the reasons why the debug malloc library was initially
     developed was to track programs' memory usage - specifically to
     locate "memory leaks" which are places where allocated memory is
     never getting freed.  *Note Memory Leaks::.

     The library has a number of logging capabilities that can track
     un-freed memory pointers as well as runtime memory usage, memory
     transactions, administrative actions, and final statistics.

examining unfreed memory
     Another common problem happens when a program frees a memory
     pointer but goes on to use it again by mistake.  This can lead to
     mysterious crashes and unexplained problems.

     To combat this, the library can write special values into a block
     of memory after it has been freed.  This serves two purposes: it
     will make sure that the program will get garbage data if it trying
     to access the area again, and it will allow the library to verify
     the area later for signs of overwriting.

   If any of the above debugging features detect an error, the library
will try to recover.  If logging is enabled then an error will be
logged with as much information as possible.

   The error messages that the library displays are designed to give the
most information for developers.  If the error message is not
understood, then it is most likely just trying to indicate that a part
of the heap has been corrupted.

   The library can be configured to quit immediately when an error is
detected and to dump a core file or memory-image.  This can be examined
with a debugger to determine the source of the problem.  The library can
either stop after dumping core or continue running.

   _NOTE_: do not be surprised if the library catches problems with
your system's routines.  It took me hours to finally come to the
conclusion that the localtime call, included in SunOS release 4.1,
overwrites one of its fence-post markers.


File: dmalloc.info,  Node: How It Works,  Next: Troubleshooting,  Prev: Features,  Up: Overview

How the Library Checks Your Program
===================================

   This is one of the newer sections of the library implying that it is
incomplete.  If you have any questions or issues that you'd like to see
handled here, please let me know.

   The dmalloc library replaces the heap library calls normally found in
your system libraries with its own versions.  When you make a call to
malloc (for example), you are calling dmalloc's version of the memory
allocation function.  When you allocate memory with these functions, the
dmalloc library keeps track of a number of pieces of debugging
information about your pointer including: where it was allocated,
exactly how much memory was requested, when the call was made, etc..
This information can then be verified when the pointer is freed or
reallocated and the details can be logged on any errors.

   Whenever you reallocate or free a memory address, the dmalloc library
always performs a number of checks on the pointer to make sure that it
is valid and has not been corrupted.  You can configure the library to
perform additional checks such as detected fence-post writing.  The
library can also be configured to overwrite memory with non-zeros (only
if calloc is not called) when it is allocated and erase the memory when
the pointers are freed.

   In addition to per-pointer checks, you can configure the library to
perform complete heap checks.  These complete checks verify all internal
heap structures and include walking all of the known allocated pointers
to verify each one in turn.  You need this level of checking to find
random pointers in your program which got corrupted but that won't be
freed for a while.  To turn on these checks, you will need to enable the
'check-heap' debug token.  *Note Debug Tokens::.  By default this will
cause the heap to be fully checked each and every time dmalloc is called
whether it is a malloc, free, realloc, or another dmalloc overloaded
function.

   Performing a full heap check can take a good bit of CPU and it may be
that you will want to run it sporadically.  This can be accomplished in
a couple different ways including the '-i' interval argument to the
dmalloc utility.  *Note Dmalloc Program::.  This will cause the check to
be run every N-th time.  For instance, 'dmalloc -i 3' will cause the
heap to be checked before every 3rd call to a memory function.  Values
of 100 or even 1000 for high memory usage programs are more useful than
smaller ones.

   You can also cause the program to start doing detailed heap checking
after a certain point.  For instance, with 'dmalloc -s 1000' option, you
can tell the dmalloc library to enable the heap checks after the 1000th
memory call.  Examine the dmalloc log file produced and use the
iteration count if you have `LOG_ITERATION_COUNT' enabled in your
`settings.h' file.

   The start option can also have the format `file:line'.  For
instance, if it is set to `dmalloc_t.c:126', dmalloc will start
checking the heap after it sees a dmalloc call from the `dmalloc_t.c'
file, line number 126.  If you use `dmalloc_t.c:0', with a 0 line
number, then dmalloc will start checking the heap after it sees a call
from anywhere in the `dmalloc_t.c' file.


-------------------------------------------------------------------------------

Some Solutions to Common Problems
=================================

   This section provides some answers to some common problems and
questions.  Please send me mail with any additions to this list -
either problems you are still having or tips that you would like to
pass on.

   When diagnosing a problem, if possible, always make sure you are
running the most up to date version of Dmalloc available from the home
page `http://dmalloc.com/'.  Problems are often fixed and a new release
is published before people encounter them.

`Why does my program run so slow?'
     This library has never been (and maybe never will be) optimized for
     space nor speed.  Some of its features make it unable to use some
     of the organizational methods of other more efficient heap
     libraries.  If you have the `check-heap' token enabled, see the
     `-i' option to the dmalloc utility.  *Note Dmalloc Program::.

`Why was a log-file not produced after I ran my program?'
     This could be caused by a number of different problems.

       1. Are you sure you followed all of the items in the "Getting
          Started" section?  Please review them if there is any doubt.
          *Note Getting Started::.

       2. Use the `env' or `printenv' commands to make sure that the
          `DMALLOC_OPTIONS' variable is set in your exported
          environment.  *Note Environment Variable::.

       3. Make sure that your program has been compiled correctly with
          the dmalloc library.  The `ident' program should show chunk.c
          and other dmalloc files compiled into your program.  You can
          also do `strings -a your-program | grep chunk.c' and look for
          something like `$Id: chunk.c,v 1.152 1999/08/25 12:37:01 gray
          Exp $' with different versions and date information.  If this
          doesn't show up then chances are dmalloc was not linked into
          your program.

       4. If your program changes its working directory, it may write
          the dmalloc log-file somewhere else in the filesystem.  You
          will need to check both where the program was started and to
          where it might change directory.

       5. The logfile is only producted when `dmalloc_shutdown()' is
          called.  By default it will be called when `exit()' gets
          called.  If you are running your program and press
          `Control-C' under Unix the program will stop immediately and
          `dmalloc_shutdown()' will not get called.  You can either
          setup a signal handler for `SIGINTR' and call exit yourself,
          or you can enable the `catch-signals' token.  *Note Debug
          Tokens::.

       6. If your program is segfaulting or otherwise crashing when it
          exits, the `exit()' routine may not being called.  You will
          have to resolve these issues so the dmalloc library can
          gracefully exit and write it's log file.

       7. You may want to call `dmalloc_log_stats()' and
          `dmalloc_log_unfreed()' (or `dmalloc_log_changed()') directly
          to have the library write its log file.  Some system modules
          may not have shutdown if you call this before `exit()' so
          extra unfreed memory may be reported.


`I don't see any information about my non-freed (leaked) memory?'
     The library will not (by default) report on "unknown" non-freed
     memory.  Unknown means memory that does not have associated file
     and line information.

     This will be necessary if you are _not_ including `dmalloc.h' in
     all of your C files or if you are interested in tracking leaks in
     system functions.

`Dmalloc is returning the error "malloc library has gone recursive"'
     This most likely indicates that you are using the Dmalloc library
     within a threaded application and two threads are trying to use
     the dmalloc library at once.  Please see the section of the manual
     about threads for more information about properly configuring the
     library.  *Note Using With Threads::.

     If you are not using threads, then your program could have caught a
     signal while within Dmalloc, which then in turn called a memory
     allocation routine.  It is unwise to allocate memory on the heap
     in most signal handlers.  Lastly, some functions called by the
     library may call memory routines that it does not anticipate.  If
     you think this the case, please report the problem and include a
     stack trace, operating system version/type, and the version of
     Dmalloc you are using.



-------------------------------------------------------------------------------
