      DATA STORAGE TECHNIQUES - FIXED OR VARIABLE LENGTH RECORDS

                            Matthew Probert
                           Servile  Software




Variable length records make the optimal use of a storage medium. Each 
record only occupying as much space it needs. In order to do this, 
each field must be delimited and each record also delimited. When an 
application's data is often of varying lengths, such as an address 
file, the usefulness of variable length records is apparent. For 
example. 

Mr Smith*15 New Road*Basingstoke*Hampshire#
Mrs Brown*17 Orchard Rise*Luton*Bedfordshire#
Mr T Jones*53 Everton Drive*Woking*Surrey#
Dr Johnson*The Surgery*St Pauls Road*Cobham*Surrey#

Each field is delimited by an asterisk '*', and each record by a hash 
'#'. 

Reading variable length records involves reading a single byte from 
the file, checking it against a delimiter, and storing it if not. This 
is an awkward and time consuming task. But is still much simpler than 
the writing of variable length records. 

Appending a new variable length record to a file involves writing each 
field followed by a delimiter byte and finally a record delimiter 
byte. Problems occur when you try to amend an existing record. The 
existing record must be deleted from the file by moving all subsequent 
data down, and then the amended record can be appended. 

Seeking other records in a variable record length file is a nightmare. 
Dummy reads must be carried out in order to locate the end of each 
record, until the desired record is reached. 

Fixed length records occupy more storage space than variable length 
records, but are far quicker and simpler to access. An entire record 
is read and written in one operation. Moving the file pointer to other 
records is done by simply incrementing or decemneting the file 
pointer. 


If the entire file is to be read and written, then variable length 
records make sense. The problems of changing record lengths is no 
longer a problem. However, if random access is desired, then fixed 
length records are the only practical consideration. 

As a case study, consider two different, and yet useful applications. 
The first is a "note book". This is a database system with just two 
fields: Item and Details. The item field is, say, thirty bytes long, 
and the details field is one thousand bytes long. This provides scope 
to record sensible notes on subjects. If this system is developed 
using a fixed length record structure, then each record will occupy 
1030 bytes. If the file records five thousand items, then the total 
storage required will be 5000 x 1030 equals 5 150 000 bytes. If the 
average length of an entry is one hundred bytes, then the application 
is wasting 4 500 000 bytes. Obviously a better solution would be to 
use a variable length record structure. This would involve reading the 
entire file into RAM, and so the main limitation is the available free 
memory. In practice this is about 480 000 bytes. If the average entry 
is one hundred bytes, then the application will be limited to about 
3600 entries. In practice a limit of 5000 entries can be made quite 
successfuly, since very few entries will actually have a title of the 
full thirty bytes, and still fewer will have over one hundred bytes of 
details. 

The second consideration is of an address recording system. In this 
case there are eight short fields: Name, company, address, area, town, 
county, post code and telephone number. No field is longer than thirty 
bytes, and so storage optimisation is much higher than the note pad. 
This application lends itself to fixed length records. 

