=====================================
======== Accessing Datafiles ========
=====================================
In order to access the contents of a datafile, you will need to know where
each object is located. The easiest way to do this is by integer index,
using an automatically generated header file. With the grabber, type a name
into the "Header:" field, and the object indexes will be written to this
file whenever the datafile is saved. With the dat utility, use the '-h'
option, eg. "dat filename.dat -h filename.h". The header will define C
preprocessor symbols for each object in the datafile, for example:
#define SOME_DATA 0 /* DATA */
#define SOME_MORE_DATA 1 /* DATA */
To prevent name conflicts, you can specify a prefix string for these
definitions by typing it into the "Prefix:" field in the grabber or using
the '-p' option to dat.
To load a datafile into memory, call the function:
DATAFILE *load_datafile(char *filename);
This will load the entire file, returning a pointer to it, or NULL on error.
When the data is no longer required, the entire thing can be destroyed by
calling:
void unload_datafile(DATAFILE *dat);
When you load a datafile, you will obtain a pointer to an array of DATAFILE
structures:
typedef struct DATAFILE
{
void *dat; - pointer to the actual data
int type; - object type ID
long size; - size of the data, in bytes
DATAFILE_PROPERTY *prop; - list of object properties
} DATAFILE;
The only really important piece of information here is the dat field, which
points to the contents of the object. What type of data this is will depend
on the type of object: for bitmaps it will be an Allegro BITMAP structure,
for RLE sprites an RLE_SPRITE, for fonts a FONT structure, etc. If you are
programming in C you can pass this pointer directly to the relevant Allegro
library functions, but if you are using C++ you will need to cast it to the
appropriate type to prevent the compiler giving a warning.
For example, if you have a datafile called myfile.dat, which contains a
bitmap called COOL_PICTURE, and you have used it to produce a header called
myfile.h, you could display the bitmap with the code:
#include "myfile.h"
void show_the_bitmap()
{
DATAFILE *dat;
BITMAP *bmp;
dat = load_datafile("myfile.dat");
if (!dat) {
/* report an error! */
return;
}
bmp = (BITMAP *)dat[COOL_PICTURE].dat;
blit(bmp, screen, 0, 0, 0, 0, bmp->w, bmp->h);
unload_datafile(dat);
}
If a datafile contains nested child datafiles, the header will prefix the
names of objects in the sub-files with the name of their parent datafile. It
will also define a count of the number of objects in the child file, which
may be useful if for example the child datafile contains several bitmaps
which form a 'run' animation, and you want your code to automatically
adjust to the number of frames in the datafile.
For example, the datafile:
"FILE" - NESTED_FILE
|- "BMP" - A_BITMAP
|- "FONT" - A_FONT
"DATA" - SOME_DATA
"DATA" - SOME_MORE_DATA
produces the header:
#define NESTED_FILE 0 /* FILE */
#define NESTED_FILE_A_BITMAP 0 /* BMP */
#define NESTED_FILE_A_FONT 1 /* FONT */
#define NESTED_FILE_COUNT 2
#define SOME_DATA 1 /* DATA */
#define SOME_MORE_DATA 2 /* DATA */
The main datafile contains three objects (NESTED_FILE, SOME_DATA, and
SOME_MORE_DATA) with consecutive indexes, while the child datafile contains
the two objects A_BITMAP and A_FONT. To access these objects you need to
reference both the parent and child datafiles, eg:
DATAFILE *dat = load_datafile("whatever.dat");
DATAFILE *nested = (DATAFILE *)dat[NESTED_FILE].dat;
FONT *thefont = (FONT *)nested[NESTED_FILE_A_FONT].dat;
If you need to access object property strings from within your program, you
can use the function:
char *get_datafile_property(DATAFILE *dat, int type);
This will return a pointer to the property string if it can be found, and an
empty string (not null!) if it does not exist. One possible use of this
function is to locate objects by name, rather than using the indexes from a
header file. The datafile array is ended by an object of type DAT_END, so to
search the datafile dat for the object "my_object" you could use the code:
for (i=0; dat[i].type != DAT_END; i++) {
if (stricmp(get_datafile_property(dat+i, DAT_ID('N','A','M','E')),
"my_object") == 0) {
/* found the object at index i */
}
}
/* not found... */
If you prefer to access objects by name rather than index number, you can
use the function:
DATAFILE *find_datafile_object(DATAFILE *dat, char *objectname);
This will search an already loaded datafile for an object with the specified
name, returning a pointer to it, or NULL if the object cannot be found. It
understands '/' and '#' separators for nested datafile paths.
It is also possible to selectively load individual objects from a datafile,
with the function:
DATAFILE *load_datafile_object(char *filename, char *objectname);
This searches the datafile for an object with the specified name, so
obviously it won't work if you strip the name properties out of the file.
Because this function needs to seek through the data, it will be extremely
slow if you have saved the file with global compression. If you are planning
to load objects individually, you should save the file uncompressed or with
individual compression per-object. Because the returned datafile points to a
single object rather than an array of objects, you should access it with the
syntax datafile->dat, rather than datafile[index].dat, and when you are done
you should free the object with the function:
void unload_datafile_object(DATAFILE *dat);
Alternatively, the packfile functions can open and read directly from the
contents of a datafile object. You do this by calling pack_fopen() with a
fake filename in the form "filename.dat#object_name". The contents of the
object can then be read in an identical way to a normal disk file, so any of
the file access functions in Allegro (eg. load_pcx() and set_config_file())
can be used to read from datafile objects. Note that you can't write to
datafiles in this way: the fake file is read only. Also, you should save the
file uncompressed or with per-object compression if you are planning on
using this feature. Finally, be aware that the special Allegro object types
aren't the same format as the files you import the data from, so if for
example you want to use load_pcx to read an image from a datafile, you
should import it as a binary data chunk rather than as a BITMAP object.
If you have appended a datafile to the end of your executable with the
exedat utility, use load_datafile("#") to read the entire thing into memory,
load_datafile_object("#", "object_name") to load a specific object, and
pack_fopen("#object_name", F_READ) to read one of the objects directly with
your own code.
By default, all graphic objects loaded from a datafile will be converted
into the current color depth. This conversion may be both lossy and very
slow, particularly when reducing from truecolor to 256 color formats, so you
may wish to disable it by calling set_color_conversion(COLORCONV_NONE) or
set_color_conversion(COLORCONV_PARTIAL) before your call to load_datafile().
=====================================
======== Compiling Datafiles ========
=====================================
The utility dat2s can be used to convert a datafile into an asm (.s) source
file, which can then be assembled and linked into your program. This avoids
the need for a separate datafile to accompany your program, and means the
data will automatically be loaded into memory at startup. You should be
aware, though, that large datafiles can take a long time to compile, and
that it is not possible to compress data which is compiled in this way.
The simplest way to invoke dat2s is with the command:
dat2s filename.dat -o output.s
The resulting asm file can then be assembled with the command:
gcc -c output.s
This will produce an object module called output.o, which can be linked into
your program, for example:
gcc myprog.c -o myprog.exe output.o -lalleg
Your program can then access the contents of the datafile as simple global
variables. Definitions for these variables can be obtained by telling dat2s
to output a header file as well as the asm file, with the '-h' option. You
can also use '-p' to set a prefix string for all the object names. For
example, when applied to the datafile:
"BMP" - A_BITMAP
"BMP" - ANOTHER_BITMAP
"SAMP" - EXPLODE
"PAL" - SOME_COLORS
"FONT" - THE_FONT
the command:
dat2s filename.dat -o output.s -h output.h -p item
produces the header:
extern BITMAP item_a_bitmap;
extern BITMAP item_another_bitmap;
extern SAMPLE item_explode;
extern PALETTE item_some_colors;
extern FONT item_the_font;
extern DATAFILE item_data[];
You can refer to these objects directly, for example:
blit(&item_a_bitmap, screen, 0, 0, 0, 0, SCREEN_W, SCREEN_H);
Alternatively, you can use the datafile array for compatibility with code
that was originally written for separately loaded datafiles, with the
standard syntax item_data[index].dat.
================================
======== Custom Objects ========
================================
Some of the objects in a datafile, for example palettes and FLI animations,
are simply treated as blocks of binary data, but others are loaded into
special formats such as bitmap structures or compiled sprites. It is
possible to extend the datafile system to support your own custom object
types, eg. map objects for a tile based engine, or level data for a platform
game. Obviously the grabber has no way of understanding this data, but it
will allow you to import binary data from external files, so you can grab
information produced by your own utilities. If you are happy with the data
being loaded as a simple binary block, that is all you need to do, but if
you need to load it into a specific structure, read on...
Your custom objects must be given a unique type ID, which is formed from
four ASCII characters (by convention all uppercase A-Z). If you don't use
all four characters, the string should be padded with spaces (ASCII 32). You
should use this ID when creating the objects in the grabber (select
New/Other and type in the ID string), and in your code you should define an
identifier for the type, eg:
#define DAT_MAPDATA DAT_ID('M','A','P','D')
You then need to write functions for loading and destroying objects of this
type, in the form:
void *load_mapdata(PACKFILE *f, long size)
{
/* Allegro will call this function whenever an object of your custom
* type needs to be loaded from a datafile. It will be passed a
* pointer to the file from which the data is to be read, and the size
* of the object in bytes. It should return a pointer to the loaded
* data, which will be stored in the dat field of the datafile object
* structure, or NULL if an error occurs. The file will have been
* opened as a sub-chunk of the main datafile, so it is safe to read
* past the end of the object (if you attempt this, Allegro will
* return EOF), and it is also safe to return before reading all the
* data in the chunk (if you do this, Allegro will skip any unused
* bytes before starting to read the next object). You should _not_
* close the file when you are done: this will be handled by the
* calling function. To clarify how all this works, here's an example
* implementation of a null-terminated string object:
*/
#define MAX_LEN 256
char buf[MAX_LEN];
char *p;
int c;
for (c=0; (c<MAX_LEN-1) && (!pack_feof(f)); c++)
buf[c] = pack_getc(f);
buf[c] = 0;
p = malloc(c+1);
strcpy(p, buf);
return p;
}
void destroy_mapdata(void *data)
{
/* Allegro will call this function whenever an object of your custom
* type needs to be destroyed. It will be passed a pointer to the
* object (as returned by the load function), and should free whatever
* memory the object is using. For example, the simple string object
* returned by the above loader could be destroyed with the code:
*/
if (data)
free(data);
}
Finally, before you load your datafile you must tell Allegro about the
custom format, by calling:
register_datafile_object(DAT_MAPDATA, load_mapdata, destroy_mapdata);
It is also possible to integrate support for custom object types directly
into the grabber and dat utilities, by copying some special files into the
tools/plugins directory. This can be used to add whole new object types and
menu commands, or to provide additional import/export routines for the
existing formats. See tools/plugins/plugins.txt for an overview of how to
write your own grabber plugins.
=================================
======== The File Format ========
=================================
In case anyone wants to do some serious hackery, and for my own future
reference, here are some details of the innards of the datafile format.
Note that this is different to the datafile format used by Allegro versions
2.1 and earlier. Allegro can still load files from the old format, but it
was much less flexible and didn't support nice things like object
properties, so you should load any old files into the grabber and save them
out again to convert to the new format.
Nb. if all you want to do is write a utility that manipulates datafiles in
some way, the easiest approach is probably to use the helper functions in
datedit.c, which are currently shared by the dat, dat2s, and grabber
programs. These functions handle loading, saving, inserting and deleting
objects, and modifying the contents of datafiles in various ways, but life
is too short for me to bother documenting them all here. Look at the
source...
Anyway. All numbers are stored in big-endian (Motorola) format. All text is
stored in UTF-8 encoding. A datafile begins with one of the 32 bit values
F_PACK_MAGIC or F_NOPACK_MAGIC, which are defined in allegro.h. If it starts
with F_PACK_MAGIC the rest of the file is compressed with the LZSS
algorithm, otherwise it is uncompressed. This magic number and optional
decompression can be handled automatically by using the packfile functions
and opening the file in F_READ_PACKED mode. After this comes the 32 bit
value DAT_MAGIC, followed by the number of objects in the root datafile (not
including objects nested inside child datafiles), followed by each of those
objects in turn.
Each object is in the format:
OBJECT =
var - <property list> - any properties relating to the object
32 bit - <type ID> - object type ID
32 bit - <compressed size> - size of the raw data in the file
32 bit - <uncompressed size> - see below
var - <data> - the contents of the object
The property list can contain zero or more object properties, in the form:
PROPERTY =
32 bit - <magic> - "prop"
32 bit - <type ID> - property type ID
32 bit - <size> - size of the property string, in bytes
var - <data> - property string, _not_ null-terminated
If the uncompressed size field in an object is positive, the contents of the
object are not compressed (ie. the raw and compressed sizes should be the
same). If the uncompressed size is negative, the object is LZSS compressed,
and will expand into -<uncompressed size> bytes of data. The easiest way to
handle this is to use the pack_fopen_chunk() function to read both the raw
and compressed sizes and the contents of the object.
The contents of an object vary depending on the type. Allegro defines the
standard types:
DAT_FILE =
32 bit - <object count> - number of objects in the sub-file
var - <object list> - objects in the same format as above
DAT_FONT =
16 bit - <font size> - 8, 16, -1, or 0
if font size == 8 { - obsolete as of version 3.9.x!
unsigned char[95][8] - 8x8 bit-packed font data
}
if font size == 16 { - obsolete as of version 3.9.x!
unsigned char[95][16] - 8x16 bit-packed font data
}
if font size == -1 { - obsolete as of version 3.9.x!
95x {
16 bit - <width> - character width
16 bit - <height> - character height
var - <data> - character data (8 bit pixels)
}
}
if font size == 0 { - new format introduced in version 3.9.x
16 bit - <ranges> - number of character ranges
for each range {
8 bit - <mono> - 1 or 8 bit format flag
32 bit - <start> - first character in range
32 bit - <end> - last character in range (inclusive)
for each character {
16 bit - <width> - character width
16 bit - <height> - character height
var - <data> - character data
}
}
}
DAT_SAMP =
16 bit - <bits> - sample bits (negative for stereo)
16 bit - <freq> - sample frequency
32 bit - <length> - sample length
var - <data> - sample data
DAT_MIDI =
16 bit - <divisions> - MIDI beat divisions
32x {
32 bit - <length> - track length, in bytes
var - <data> - MIDI track data
}
DAT_FLI =
var - <data> - FLI or FLC animation, standard format
DAT_BITMAP =
DAT_C_SPRITE =
DAT_XC_SPRITE =
16 bit - <bits> - bitmap color depth
16 bit - <width> - bitmap width
16 bit - <height> - bitmap height
var - <data> - bitmap data
Valid color depths are 8, 15, 16, 24, 32, and -32. Both 15 and 16 bit
images are stored in 5.6.5 RGB format, and 24 and 32 bit images as
8.8.8 RGB. The special -32 flag indicates that the data is in true 32
bit RGBA format.
DAT_RLE_SPRITE =
16 bit - <bits> - sprite color depth
16 bit - <width> - sprite width
16 bit - <height> - sprite height
32 bit - <size> - data size, in bytes
var - <data> - RLE compressed sprite data
Valid color depths are 8, 15, 16, 24, 32. and -32. Both 15 and 16 bit
images are stored in 5.6.5 RGB format with 16 bit skip counts and EOL
markers, and 24 and 32 bit images as 8.8.8 RGB. with 32 bit skip
counts and markers. The special -32 flag indicates that the data is in
true 32 bit RGBA format.
DAT_PALETTE =
256 x {
8 bit - <red> - red component, 0-63
8 bit - <green> - green component, 0-63
8 bit - <blue> - blue component, 0-63
8 bit - <pad> - alignment padding
}
I think that covers everything.
I don't think you needed all of that, most is just for developers, but oh well.