Stdlib Memory Management


Memory Management Routines
10.1 - Interface
10.2 - Memory Management Overview
10.3 - MemInit
10.3.1 - Calling Conventions and Assertions
10.3.2 - Syntax & Examples
10.4 - MemInit2
10.4.1 - Calling Conventions and Assertions
10.4.2 - Syntax & Examples
10.5 - Malloc
10.5.1 - Calling Conventions and Assertions
10.5.2 - Syntax & Examples
10.6 - Free
10.6.1 - Calling Conventions and Assertions
10.6.2 - Syntax & Examples
10.7 - Realloc
10.7.1 - Calling Conventions and Assertions
10.7.2 - Syntax & Examples
10.8 - DupPtr
10.8.1 - Calling Conventions and Assertions
10.8.2 - Syntax & Examples
10.9 - IsInHeap
10.9.1 - Calling Conventions and Assertions
10.9.2 - Syntax & Examples
10.10 - IsPtr
10.10.1 - Calling Conventions and Assertions
10.10.2 - Syntax & Examples
10.11 - HeapStart
10.11.1 - Calling Conventions and Assertions
10.11.2 - Syntax & Examples
10.12 - BlockSize
10.12.1 - Calling Conventions and Assertions
10.12.2 - Syntax & Examples
10.13 - MemAvail
10.13.1 - Calling Conventions and Assertions
10.13.2 - Syntax & Examples
10.14 - MemFree
10.14.1 - Calling Conventions and Assertions
10.14.2 - Syntax & Examples
10.15 - BasePtr
10.15.1 - Calling Conventions and Assertions
10.15.2 - Syntax & Examples


Memory Management Routines

The UCR Standard Library (stdlib) contains a large number of routines that convert data from one format to another. These routines typically convert between some binary format and ASCII. For example, the ATOI routine converts an ASCII string to an integer. Conversely, the ITOA routine converts an integer value to an ASCII string. There are other conversions as well. For example, the TOUPPER routine converts lower case characters to upper case.


10.1 Interface

To access the routines in the conversions package, your assembly language module must include the file "memory.a" during assembly. You can accomplish this with either of the following include statements in your assembly code:




	include	memory.a
or
	include	ucrlib.a

The memory.a include file exports several symbols. The UCR Standard Library prefaces all "private" names with a dollar sign ("$"). You should not call any routine in this package that begins with this symbol. To avoid name conflicts, you should not define any symbols in your programs that begin with a dollar sign ("$"). Note that future versions of the stdlib (that remain compatible with this release) may change "private" names. To remain compatible with future releases, you must not refer to these "private" names within your programs unless otherwise directed.

Source code appearing in this chapter is current as of Version Two, Release 40. There may be minor changes between this source code and the current release.


10.2 Memory Management Overview

The memory management package provides routines that let you dynamically allocate and free memory on the heap during program execution. The stdlib defines the heap as all memory from the end of your program to the end of conventional memory (at address 9FFF:FFFF). The memory management routines accept requests for blocks of memory (up to 64K minus eight bytes in size) and, if sufficient memory is available, returns a pointer to an available block of that size. When you are done using a given block of memory, you can return it to the heap for future use.

A typical application program (that uses dynamic memory allocation) calls three different subroutines: MemInit, Malloc, and Free. The MemInit routine initializes the memory management system. You must call this routine (or the MemInit2 routine) before using any memory management routines or other routines that call memory management routines. Since a large percentage of programs utilize the memory management facilities in the standard library, the SHELL.ASM skeleton file already contains a call to MemInit for you. After the memory management system is initialized, you can begin making calls to malloc and free to allocate and deallocate blocks of memory.

The stdlib memory management routines maintain a reference count on every allocated block. By using the DupPtr routine, you can tell the stdlib that you have two (or more) pointers pointing at the same block in the heap. You must call Free once for each pointer you've duplicated in your system (i.e., you must call Free "reference count" times). Free decrements the reference count by one and only returns the memory to the free list if the reference count hits zero. By making a call to DupPtr for every copy of a pointer you make, you can avoid the "dangling pointer" problem in your code.

The stdlib contains many other utility routines that let you see how much memory is currently available, check the size of the largest available block, and check the status of some pointer you're currently using. See the following descriptions for more details.


10.3 MemInit

MemInit initializes the memory management subsystem. It grabs all the available memory from the end of your program to the end of conventional memory for use by the heap. To work properly, your program must contain the following segment as the last segment in your program:




zzzzzzseg	segment	para public 'zzzzzz'
LastBytes	byte	16 dup (?)
zzzzzzseg	ends

Typically, you would only call MemInit once in a program. If you make more than a single call to MemInit, all variables allocated on the heap prior to the second call are effectively deallocated.

If you want finer control over the position and size of the heap in memory, see the MemInit2 routine. That routine lets you specify these values as parameters.


10.3.1 Calling Conventions and Assertions


10.3.2 Syntax & Examples

Generally, you only call MemInit once, at the beginning of your program. The call requires no parameters and takes the following form:




	MemInit

10.4 MemInit2

MemInit2 also initializes the heap (effectively deallocating any previously allocated variables). MemInit2, unlike MemInit, lets you specify the starting address and size of the heap. This allows you, for example, to continue using the memory management package when writing TSRs that need to return memory to the system or use the DOS EXEC function that requires free memory after your program.


10.4.1 Calling Conventions and Assertions


10.4.2 Syntax & Examples

The following code fragment creates a heap in the data segment that is 32768 bytes long.




dseg            segment para public 'DATA'

        <normal variable declarations>
        
                align   16
Heap            byte    32768 dup (?)
dseg            ends
                 .
                 .
                 .
                mov     ax, offset Heap
                shr     ax, 4           ;Convert to a paragraph
                add     ax, seg dseg    ; address.
                mov     es, ax
                mov     cx, 32768/16    ;# of paragraphs in the heap.
                MemInit2


10.5 Malloc

Malloc (memory allocation) lets you request a block of memory from the heap. You specify the number of bytes you want to allocated in the CX register. If malloc can find a block at least this large on the heap, it will carve off a block that is at least this size and return a pointer to this block in th ES:DI register pair. Malloc will mark that block so that future Malloc requests will not reuse this memory until you explicitly free it with a call to the Free routine.

If the memory allocation request fails because there isn't a large enough block on the heap, the Malloc routine will check to see if the exception handling package is enabled. If this is the case, Malloc will raise the $InsuffMem exception. If the exceptions package is not active, Malloc returns with the carry set (conversely, Malloc always returns with the carry flag clear if it was able to satisfy the memory allocation request). To prevent Malloc from raising an exception, you can always (temporarily) disable the exceptions with a call to DisableExcept.

Because the current memory manager requires a certain amount of overhead data to keep track of blocks it allocates, and because it supports only paragraph granularity, Malloc will actually allocate more data than you request. As of version two, release 40, each block you malloc requires eight bytes of overhead and the total allocation (overhead plus requested size) must be an even multiple of 16 bytes. If you request only one byte, malloc will actually allocate 16 bytes on the heap. Likewise, if you request nine bytes, Malloc will actually allocate 32 bytes (eight bytes of overhead plus nine bytes of data is 17 bytes, Malloc always rounds up to the next even multiple of 16 bytes). This loss of available memory is known as internal fragmentation.

Malloc also suffers from another problem that may waste memory known as external fragmentation.. Whenever you call Malloc, it chooses an appropriate block of memory from a list of free blocks. It may turn out that there is sufficient free memory on the heap (if you add up the sizes of all the free blocks) but there is no single block large enough to fill the request. Since Malloc cannot rearrange other blocks in memory, it will fail to satisfy the request even though, technically, suffient free memory exists. To avoid external fragmentation you should avoid sequences of Malloc operations with random Free calls intermixed. If you Malloc blocks that are likely to be Free'd together, you will reduce the effects of external fragmentation.


10.5.1 Calling Conventions and Assertions


10.5.2 Syntax & Examples

The following example carves off a block of 256 bytes from the heap:




MemPtr          dword   ?
                 .
                 .
                 .
                mov     cx, 256
                malloc
                jc      MemAllocErr
                mov     word ptr MemPtr, DI
                mov     word ptr MemPtr+2, ES
                 .
                 .
                 .

10.6 Free

The Free routine lets you return memory to the memory management system when you are finished using it. You pass the address of the block you want to return to the system (i.e., the pointer returned in ES:DI by Malloc) and Free does the rest.

Calling Free may not immediately make that block of memory available for future Malloc requests. Free decrements the reference counter associated with the current free block. If this counter is zero, then Free returns the block to the free memory list. If the count is not zero, then Free assumes some other pointer still references the block, so it does not deallocate the block.

There are several errors that can occur when you call Free. If Free successfully frees the memory block (or decrements the reference counter without incident), it returns with the carry flag clear. If it encounters an error, it will return with the carry flag set or raise a $BadPointer exception if exceptions are initialized and active.

There are two basic problems Free encounters. First, you attempt to free a block that Malloc did not allocate. This happens when you call Free and you don't pass it a pointer passed to you by Malloc. Another problem Free encounters is when you attempt to Free a block of memory that you've previously released to the system via a call to Free. Corruption of the heap will also cause problems with Free, but that usually falls into the second category above.


10.6.1 Calling Conventions and Assertions


10.6.2 Syntax & Examples

The following example carves off a block of 256 bytes from the heap:




MemPtr          dword   ?
                 .
                 .
                 .
                mov     cx, 256
                malloc
                jc      MemAllocErr
                mov     word ptr MemPtr, DI
                mov     word ptr MemPtr+2, ES
                 .
                 .
                 .
        <use the memory pointed at by MemPtr>
                 .
                 .
                 .
                les     di, MemPtr
                Free
                jc      FreeError

10.7 Realloc

The Realloc routine lets you adjust the size of an existing allocation block on the heap. It will let you shrink or expand a block. If you are making the block smaller, Realloc leaves the current block in place and simply frees up any necessary bytes at the end of the data structure. If you are making the block larger, Realloc will allocate a new block of the requested size, copy the old block to the new block, and then call Free to release the memory held by the old block.

Like Malloc and Free, Realloc will return with the carry flag clear if the reallocation process is successful. If a failure occurs, Realloc will either raise an exception (if exceptions are active) or return with the carry flag set (if exceptions are not active). Failure can occur if there is insufficient memory to make the block larger or if you pass a bad pointer to the Realloc routine.


10.7.1 Calling Conventions and Assertions


10.7.2 Syntax & Examples

The following example carves off a block of 256 bytes from the heap and then expands this block size to 512 bytes sometime later:




MemPtr          dword   ?
                 .
                 .
                 .
                mov     cx, 256
                malloc
                jc      MemAllocErr
                mov     word ptr MemPtr, DI
                mov     word ptr MemPtr+2, ES
                 .
                 .
                 .
        <use the memory pointed at by MemPtr>
                 .
                 .
                 .
; Now we need another 256 bytes:

                les     di, MemPtr
                mov     cx, 512
                Realloc
                jc      MemAllocErr
                 .
                 .
                 .
        <Use the 512 bytes available>
                 .
                 .
                 .
                les     di, MemPtr
                Free
                jc      FreeError

10.8 DupPtr

DupPtr increments the reference count of an existing block on the heap. You pass it the address of the block (i.e., a pointer that Malloc or Remalloc returns) in the ES:DI register pair and it increments the associated reference counter. Note that a call to Free will decrement this counter. See Free for more details.

DupPtr returns the carry flag clear if the pointer you pass has the same format that Malloc and Realloc returns. DupPtr does not actually check to see if this is legal, allocated, block of memory because of performance concerns. If you want to verify that the pointer is completely legal, call the IsPtr routine (see the example later). If DupPtr encounters an illegal pointer, it will return the carry flag set if exceptions are not active, it will raise a $BadPointer exception if they are active.


10.8.1 Calling Conventions and Assertions


10.8.2 Syntax & Examples

DupPtr does not check to see if its pointer is valid because almost all calls to DupPtr occur immediately after (or very soon after) the corresponding call to Malloc or Realloc:




                mov     cx, BufferSize
                malloc
                mov     word ptr BufPtr, di
                mov     word ptr BufPtr+2, es
                
                DupPtr
                mov     word ptr BufPtrToo, di
                mov     word ptr BufPtrToo+2, es
                 .
                 .
                 .
On occasion, you might actually want to check the validity of the block as well as the pointer during a DupPtr operation. The following procedure shows how to accomplish this:




DupPtrChk       proc
                IsPtr
                jc      BadPointer
                DupPtr
BadPointer:     ret
DupPtrChk       endp

DupPtrChk       proc
                IsPtr
                jc      BadPointer
                DupPtr
                ret

BadPointer:     mov     ax, $BadPointer
                Raise
DupPtrChk       endp

10.9 IsInHeap

IsInHeap checks the pointer in ES:DI to see if it possibly points to some object within the heap. IsInHeap simply checks the address in ES:DI to see if it is in the range StartHeap..EndHeap. This routine does not verify that ES:DI is pointing at or into an actual data object. If ES:DI is within the specified range, IsInHeap returns the carry flag clear. Otherwise it sets the carry flag.


10.9.1 Calling Conventions and Assertions


10.9.2 Syntax & Examples

Typically, you would use IsInHeap within a subroutine to determine if a pointer points at a block being managed by the memory management system or simply points at a data object in one of your program's segments. The following CopyPtr routine uses IsInHeap to increment the reference count of an object that is already on the heap, but not bother if the object is not on the heap.




                les     di, MemPtr
                IsInHeap
                jc      SkipOperation
                DupPtr
SkipOperation:

10.10 IsPtr

IsPtr checks the address in ES:DI against the set of allocated blocks to see if it actually points at the start of an allocated block on the heap. It returns the carry flag clear if the address in ES:DI is a valid heap pointer. It returns the carry flag set if this is not the case. Note that IsPtr does not raise an exception if the pointer is invalid.

IsPtr may run somewhat slowly because it has to compare the value in ES:DI against the starting address of possibly every allocated block on the heap. To do this, it has to do a linear search of the allocated blocks. Once it finds the block, it also has to check its reference count to make sure this value is greater than zero.


10.10.1 Calling Conventions and Assertions


10.10.2 Syntax & Examples

If you want IsPtr to raise an exception, you could put a "wrapper" around it as follows:




ChkIsPtr        proc
                IsPtr
                jc      RaiseExcept
                ret

RaiseExcept:    mov     ax, $BadPointer
                raise
ChkIsPtr        endp


10.11 HeapStart

This function returns the segment portion of the starting address of the heap in the AX register (the offset portion of the starting address is always zero). Typcially, you would use this function to deallocate all the heap memory from DOS' memory allocation table just before going resident or EXECing some other program. Note that this call will destroy all data currently on the heap.


10.11.1 Calling Conventions and Assertions


10.11.2 Syntax & Examples

The following call to DOS deallocates all storage associated with the heap:




                mov     ah, 62h         ;Get the current
                int     21h             ; program's PSP address.
                mov     es, bx
                HeapStart
                sub     ax, bx          ;Compute size of program
                add     ax, 2           ;Just to be save, add 32 bytes.
                mov     bx, ax
                mov     ah, 4ah         ;DOS memory resize call.
                int     21h

; Now the heap is free memory as far as DOS is concerned.


10.12 BlockSize

BlockSize returns the currently allocation size for a block on the heap. ES:DI contains the address of the block. On return, CX contains the actual size of that block and a clear carry if ES:DI contains a reasonable looking pointer. If ES:DI does not point at the start of a block in the heap, BlockSize returns the carry flag set and zero in CX.

Note that the value that BlockSize returns may be slightly larger than the original allocation size because of internal fragmentation. In theory, you can safely use all the bytes that CX says are available. Keep in mind, however, that other sections of code may respect the original allocation size and never reference any bytes beyond the specified allocation size.


10.12.1 Calling Conventions and Assertions


10.12.2 Syntax & Examples

The following example duplicates a block on the heap:




                les     di, MemPtr
                BlockSize
                push    cx
                malloc
                pop     cx
                mov     word ptr MemPtr2, di
                mov     word ptr MemPtr2+2, es
                lds     si, MemPtr
                cld
        rep     movsb

10.13 MemAvail

MemAvail returns the size of the largest available block in the CX register. The value in CX is the number of paragraphs available in that block. The next memory allocation request can be up to this size (or 64K, whichever is smaller).


10.13.1 Calling Conventions and Assertions


10.13.2 Syntax & Examples

A robust program will periodically check the amount of remaining memory and warn the user if it is about to begin a series of memory allocations that could possibly use up all available memory. Such a code sequence would allow the program to warn the user that the amount of available free memory is running low. The following program demonstrates this.




                MemAvail
                cmp     cx, 100h        ;Largest block is not
                jae     AtLeast16K      ; at least 16K long?
                print   "Warning, you are getting low on memory.",nl
AtLeast16K:

10.14 MemFree

MemAvail returns the cumulative size of all free blocks in the CX register. The value in CX is the total number of paragraphs available. Not all of this memory may be available due to external fragmentation effects. None the less, this is a useful procedure to call since it provides a good estimate of total memory use so you can warn users when memory is getting low.


10.14.1 Calling Conventions and Assertions


10.14.2 Syntax & Examples

A robust program will periodically check the amount of remaining memory and warn the user if it is about to begin a series of memory allocations that could possibly use up all available memory. Such a code sequence would allow the program to warn the user that the amount of available free memory is running low. The following program demonstrates this.




                MemFree
                cmp     cx, 1000h       ;Is there at least
                jae     AtLeast64K      ; 64K available?

                print   "Warning, you are getting low on memory.",nl

AtLeast64K:

10.15 BasePtr

BasePtr checks the pointer in ES:DI to see if it points into an allocated block the heap. If it does, BasePtr returns the carry flag clear and ES:DI pointing at the beginning of the block. If ES:DI does not point into an allocated block, BasePtr returns the carry flag set and does not change the pointer in ES:DI


10.15.1 Calling Conventions and Assertions


10.15.2 Syntax & Examples

The following code reads a sequence of characters from the user (calling GETC) and stores the characters into a block on the heap. It uses BasePtr to restore the pointer at the end of the input operation. This isn't a great example, the program really should save the pointer in memory rather than calling BasePtr to restore its value, but it demonstrates the general idea.




                mov     cx, 128
                malloc
                mov     cx, 128
RdLp:           getc
                mov     es:[di], al
                inc     di
                cmp     al, cr
                loopne  RdLp
                
                BasePtr         ;Restore ptr to beginning of str.
                Puts            ;Print the string.