# Modding with Omnispeak

Commander Keen has a rich history of modding, to which Omnispeak is proud to contribute. Alas, Omnispeak does not work out-of-the-box with existing Keen mods, unless they were designed specifically to support Omnispeak.

However, it is possible to create mods using Omnispeak, or to (with a little effort) adapt existing mods. Depending on the complexity of the mod, this can range from being very easy, to quite difficult indeed.

There are two types of mods for Omnispeak:

1. Data mods: mods which only change data files. These can change levels, graphics, sounds (and music), as well as make simple adjustments to the game or enemy behaviour.
2. Source mods: mods which modify the Omnispeak source code. These can change literally _anything_, but require writing or re-writing source code in `C`, as well as re-compiling the game for every platform you wish to support.

A similar distinction exists between patch-based and source mods for the original Keen games, but there is one crucial difference: a CKPATCH mod _can_ change anything: Omnispeak data mods can only change a few specific things. These are often enough for simple levelpacks, or mods which don't change game or enemy behaviour much.


## Variables and EPISODE.CKx

The core of Omnispeak's modding features is the _variable system_, which is built around the `EPISODE.CKx` file. This is a text file, with a vaguely ckpatch-style syntax, which allows the configuration of a number of values which affect the game, including game text, enemy actions, and various tweakable numbers.

Examples of things stored in variables include:

- Other filenames (as mentioned above)
- On-screen text, including level names, menu items, dialogue from characters, etc.
- Values which tweak enemy behaviour, like jump heights, probabilities of taking various actions, speeds, etc.
- Chunk numbers (and/or names) used for various screens, sprites, and effects.
- Toggles for different bugs and bits of engine behaviour (e.g., the impossible bullet).
- Much more…

A variable file (like `EPISODE.CKx` consists of a number of lines containing either a comment, a variable declaration, or an instruction to load additional variables from another file.

Comments begin with the `#` character, and continue until the end of the line.

Include directives take the form `%include "filename"`, where `filename` is the name of another file to load variables from. Equally, it's possible to use `%override "filename"`, which will load the variables in "filename" at a lower priority: for example, you could have a `MOD.CKx` file which has all of your mod's changes, which ends with `%override `EPISODE.CKx`.

Variable declarations begin with the _type_ of the variable, followed by its value. The _type_ is one of the following:

- `%int` — an integer. Followed by a number. This is either in decimal, in octal (if it begins with a leading `0`), in hexadecimal (if it begins with either `$` or `0x`), or a reference to another variable (e.g., `@VariableName` would use whatever the value of  the integer variable `VariableName`).
- `%string` — a string (piece of text). This is followed by the value of the string, in double quotes (e.g. "Hello, World!"). If a new line is needed, it can be included as "\\n". If a backslash is needed, it must be escaped as "\\"
- `%intarray` — an _array_ (or list) of integers. Each integer is specified the same way as if it were an `%int`, and they are seprated by commas. This is used, for example, by the mapping of level number to music track (see `CK_LevelMusic`).
- `%stringarray` — an _array_ (or list) of strings. Strings are enclosed in double quotes, and separated by commas. Used by the level names and level entry text.
- `%action` — an _action_ or _state_ for a sprite. These consist of 13 values, as described below. This describes how one frame of a sprites animation and behaviour occurs. See the [detailed description on KeenWiki](https://keenwiki.shikadi.net/wiki/Patch:Sprite_actions) for more details. It's also possible to create an _alias_ for an action by setting `%action new = old` (though this requires `old` to already be defined, so can be complicated to use with `%override`.

By default, `EPISODE.CKx` uses `%include` to split some of the variables up into separate files:

- `STRINGS.CKx` contains all of the text for the game.
- `GFXCHUNK.CKx` contains integer variables for all of the chunks in `EGAGRAPH`. This allows other variables (and the game code) to reference them by name (e.g. `@KEENPOGOR1` instead of `211`). This file can be auto-generated by some tools.
- `ACTION.CKx` contains all of the actions (or states) used by the game. These allow fine-grained control of enemy animation.
- `EPISODE.CKx` contains all other variables.

You can select a new episode with the `/EPISODE <episode.ckx>` command-line argument to Omnispeak.

If you're using a Debug version of Omnispeak, any missing variables will be logged as warnings. Some of these may be harmless, but it can be a good way of spotting typos.

### %override and 'Patch' mods

There are two ways to make a data mod in Omnispeak:

1. Rewrite the entire EPISODE.CKx file (and any associated variable files, like STRINGS.CKx, ACTION.CKx, and GFXCHUNK.CKx).
2. Create a new variable file (e.g. MOD.CKx), which only contains the differences from the original game. This can then end with `%override EPISODE.CKx` to load the original data.

For a total conversion (where almost nothing is left unchanged), the former may be best. But for most data-only mods, it's best to do the latter. In particualar, these 'override' mods have the following advantages:

- It's easier to see what has changed in the mod.
- The mod is more likely to keep working with future Omnispeak versions (as any extra variables will be included).
- If porting a mod from a CKPATCH file, it's easier to match changes.

On the other hand, there are some trade-offs:

- Users will have to pass `/EPISODE MOD.CKx` to Omnispeak to teach it what file to load.
- You need to be careful not to name any of your variable files 'EPISODE.CKx', 'ACTION.CKx', 'GFXCHUNK.CKx', etc, as otherwise the `%override` command may load your file instead of the original.
- It can be a bit more work to create a new file, rather than modifying existing ones.
- Variables can only be referenced (in @VARNAME references or action aliases) if they appear earlier in the variable file, so if you need to reference a variable from the original game, it must be done after the `%override` statement. This can waste a small amount of memory (as the old and new versions must coexist), and results in an (otherwise harmless) warning being printed.

Still, it's worth giving it a go!

### Working with Actions

Actions are by far the most complicated type of variable used in Omnispeak, so it's worth going into them in more detail.

Every sprite has a series of actions, only one of which is valid at any time. The action describes which graphic the sprite uses for that frame of animation, how it moves, how quickly it animates, and what its behaviour is, both alone, and when it collides with another sprite.

(**Note**: The original Keen source code calls sprites "actors" and actions "states". You're likely to see these names used in the Reconstructed Keen 4–6 code, as well as in the Wolf3D community. As the Keen community has largely referred to these as actions, that's what Omnispeak — and this document — will do.)

An action in Omnispeak has the following values:

- A name: the Omnispeak code will refer to these names when setting the initial action for a sprite, as well as after some behaviours and interactions, so be careful when changing them.
- A memory address: this is a 16-bit integer, which represents the address in memory of the action data in the original DOS executable. This is how actions are written to savegames, so changing it will break compatibility. (This is why different versions of Keen can't load each others' savegames.) If you're porting an existing mod, you should try to keep these the same. If you're making a new one from scratch, you can set these to whatever you like, so long as no two actions share an address. (I like to number them is some memorable way, like `$0101` for action 1 in sprite 1.)
- The sprite chunk to use when the sprite is facing left. Typically this should be a reference to a chunk name in `GFXCHUNK.CKx`, such as `@SPR_KEENSTANDL`, but you can use a chunk number directly if you prefer.
- The sprite chunk to use when the sprite is facing right. As above. Note that the original names for chunks often include an 'L' or 'R' to make it easy to see which is which. This can be the same as the left one, if the sprite does not face to the side.
- Movement type. This can be one of: `UnscaledFrame`, `UnscaledOnce`, `ScaledFrame`, `ScaledOnce`, or `Frame`. The `Scaled` types ensure that the sprite moves smoothly across the time the action is in use, whereas `Unscaled` actions move just once per frame, in visible 'steps'. The `*Once` actions can call the `*Think` function (see below) only once, whereas `Frame` actions will `Think` multiple times while the action is active. The `Frame` action type is special, and ignores the action's speed and timers, running the `Think` function every single frame.
- Protect Animation: if this is 0, the action's sprite is guaranteed to be displayed on-screen for at least one frame (even if it should've already proceeded to another action). If 1, the frame may be skipped.
- Stick To Ground: If this is 1, the game will try to push the sprite down every time it moves left or right. This effectively means that the sprite always tries to move diagonally downwards. The purpose if this is to make sure that the sprite will follow slopes, rather than end up in midair. For actions which are not supposed to walk on the ground, set this to 0.
- Timer: how many ticks (1/70th of a second) the action should remain active for, before it transitions to the next action (see below). Also determines how often a `ScaledFrame` or `UnscaledFrame` action `think`s. Unused for `Frame` actions.
- `x` velocity: the speed at which the sprite moves left or right. This is measured in units (1/16th of a pixel, 1/256th of a tile), and is in the direction the sprite is facing. The way this is applied depends on the action type, and it's completely ignored for `Frame` actions.
- `y` velocity: the speed at which the sprite moves up or down. This is overwritten by the physics for most objects if `Stick To Ground` (above) is enabled.
- `Think` function: This is the name of a function (defined in the Omnispeak source code) for the `Think` behaviour for the sprite. The list of possible functions is hardcoded in the game executable, and determines the action's behaviour.
- `Collision` function: This is the name of a collision function (defined in the Omnispeak source code) called when the sprite collides with another sprite. Collision functions are not compatible with Think or Draw functions.
- `Draw` function: This is the name of a function (defined in the Omnispeak source code) which runs for every displayed frame. This is usually used to display the sprite on-screen, but can also be used for things which need to react instantly, like collisions with tiles.
- Next action: the name of the next action to use, once either the timer expires (above), or some other part of the game code triggers the next action. If `NULL`, there is no next action, and the sprite will disappear when the current action ends. The action must be defined somewhere in `ACTION.CKx`.

Most of these are described in even more detail on the [KeenWiki](https://keenwiki.shikadi.net/wiki/Patch:Galaxy_actions#Action_Type).

## Custom Levels and TileInfo

Using custom levels with Omnispeak is easy: Omnispeak supports the `GAMEMAPS` format natively. You can simply replace the `GAMEMAPS.CKx` file and its corresponding `MAPHEAD.CKx` file, and Omnispeak will load the new levels.

Note that these need to be the _Carmackized_ maps from TED5, Abiathar, or whichever level editor is used, and the `MAPHEAD.CKx` file needs to be the final _Carmackized_ header. TED5 does not generate this header as a separate file by default, but embeds it into an object file, `CKxMHEAD.OBJ` for linking with the original DOS executable. You can either extract the temporary `MTEMP.TMP` file, or use `FIXMHEAD` to create this. Other level editors will usually generate the correct `MAPHEAD`.

By default, Omnispeak assumed the `MAPHEAD` file also contains the TileInfo. However, if a separate `TILEINFO.CKx` file exists, Omnispeak will load that instead. This is more common for Keen mods, so may be the better choice. If you'd rather use a different name for the tileinfo file (e.g. `mymod.tli`), you can modify the `ca_tileInfo` variable in `EPISODE.CKx`.

Note that some mods (particularly those made with TED5) will name their modified map file `EDITMAPS.CKx`. You can either rename this to `GAMEMAPS.CKx`, or adjust the `ca_mapFile` variable in `EPISODE.CKx`. Similarly, if you'd rather use a different filename for the `MAPHEAD`, you can modify `ca_mapHead`.

The current version of Omnispeak does not support Keen Dreams-style Huffman compressed maps, nor does it support the RLEW-only compressed `MAPTEMP` and `MAPTHEAD` format. (But this may change if there's demand!)

## Graphics

Omnispeak only supports `EGA` graphics in the `EGAGRAPH` format. `CGA` or `VGA` graphics are not supported. Unlike the original games, the header and huffman dictionary are not compiled into the executable, but kept as separate files. Similarly, the layout of the file is stored in a TED5-compatible `GFXINFOE` file. (And further details, including the chunk names, are kept as variables in the `GFXCHUNK` file.)

If the mod uses the same `EGAGRAPH` layout as the original games (i.e., it has the same number and arrangement of bitmaps, sprites, and tiles — most non-source mods do), then you can simply replace the `EGAGRAPH.CKx` file, as well as the `EGAHEAD.CKx` and `EGADICT.CKx` support files. The existing `GFXINFOE.CKx` and `GFXCHUNK.CKx` files can be used unmodified.

If the layout has changed, you'll need to generate a new `GFXINFOE` file. This can be done manually (the format [is described on the ModdingWiki](https://moddingwiki.shikadi.net/wiki/TED5#GFXINFOE.xxx)), or can be generated by `TED5`, `TEDSETUP` or [idGrab](https://github.com/sulix/idgrab).

Similarly, if you've added or rearranged any of the chunks, you may need to update or regenerate `GFXCHUNK.CKx`. This file has all of the names of chunks, such as bitmaps, sprites, and help texts. It's a standard variable file, `%include`d from `EPISODE.CKx`, and you can either update the chunk numbers manually, or generate it from [idGrab](https://github.com/sulix/idgrab).

Omnispeak currently has no special support for uncompressed `EGAGRAPH` files, as used by some source mods. These can still be loaded by using a no-op huffman dictionary, or by modifying the Omnispeak source code.

## Sounds and Music

Similarly, Omnispeak supports the original games' `AUDIO` files, alongside their header and  Huffman dictionary.

By default, Omnispeak calls the header `AUDIOHHD.CKx`, in order to distinguish it from the uncompressed `AUDIOHED.CKx` version. However, most mods use `AUDIOHED.CKx` as the filename, regardless of compression. Either rename the file, or modify the filename in the `ca_audioHead` variable in `EPISODE.CKx`.

Omnispeak keeps information about the layout of `AUDIO.CKx` in the `AUDINFOE.CKx` file, as well as in `EPISODE.CKx`. If the layout of the audio is unchanged (i.e., no new sound effects or music was added), then the existing `AUDINFOE.CKx` can be used. Otherwise, it will need to be created by hand with a hex editor. The format is:

- 00: `numSongs` (16-bit): the number of music tracks,
- 02: `numSounds` (16-bit): the number of sound effects,
- 04: `numSoundChunks` (16-bit): the total number of chunks in the file
- 06: `startPCSounds` (16-bit): the chunk number of the first PC-speaker sound effect (probably 0)
- 08: `startAdlibSounds` (16-bit): the chunk number of the first Adlib sound effect (probably `numSounds`)
- 10: `startDigiSounds` (16-bit): the chunk number of the first digital sound effect. Keen and Omnispeak don't support digital effects, so this can be ignored, but it's usually 2×`numSounds`).
- 12: `startMusic` (16-bit): the chunk number of the first IMF music track. This is usually 2×`numSounds` (as there are no digital effects).

This may get replaced by variabes in `EPISODE.CKx`, or by a MUSE-compatible `MUSEINFO.CKx` file in future versions of Omnispeak.

## Source Modding With Omnispeak

Omnispeak provides a good (but not perfect) base from which to make 'source mods' — the Omnispeak code is a very accurate reimplementation of the game, and does not have the limitations imposed by the original game being 16-bit. Omnispeak can use all of the system's memory, can interact with modern operating system features, and be debugged and developed with modern tools.

This does however come at a cost — Omnispeak's system requirements are much higher than the original games' — a fast, 32-bit or better CPU is required (realistically, at least a 486 DX2/66), and the DOS version is known to be buggy. Additionally, Keen fans are often used to using DOSBox (or real hardware), and the Windows or Linux versions of Omnispeak may be less familiar.

Other choices for source mods include:

- [CKSRCMOD](https://keenwiki.shikadi.net/wiki/Keen:Galaxy_Source_Modding_Package) — this is a combination of the Catacomb 3D, Keen Dreams, and (older) Omnispeak code available as a base for 16-bit Keen games. It uses a mix of Omnispeak names and original id Software names for functions, types, etc. It has been used to develop several mods, most notably [Atroxian Realm](https://keenwiki.shikadi.net/wiki/Atroxian_Realm), and [Foray in the Forest](https://keenwiki.shikadi.net/wiki/Foray_in_the_Forest), both of which add many new features and optimisations, and may be useful starting points.
- [K1n9_Duk3's Recreated Keen4–6 Source Code](https://pckf.com/viewtopic.php?t=11505) (a.k.a `keensource456`) — this is a recreation of the original Keen 4–6 source code which can build perfect copies of the original game binaries, and which use the original id Software function and type names. This is more accurate than Omnispeak or CKSRCMOD, and is an excellent starting point if you're reusing lots of original enemies. It was used as the basis for [Return to the Shadowlands](https://keenwiki.shikadi.net/wiki/Return_to_the_Shadowlands).

It is possible to release multiple versions of a mod, each using different engines (to provide both a 16-bit version and an Omnispeak version, for example). Omnispeak has some tools to help, see below.

**Note:** Omnispeak is released under the GPL, so any mod _must_ have its source code released. This allows, for example, other players to recompile it for operating systems you don't have, or to take the effort to rebase it on top of newer versions of Omnispeak to fix bugs or add features.

### Basic Mod Layout

Omnispeak mostly keeps episode specific code in their own files, with names starting `ck4_`, `ck5_`, and `ck6_` — looking at these is a good start for writing your own mod. Usually, you'll want to have your mod behave as its own episode (and so have its own version of these files), but there are some parts of the code which check the episode number, so you'll still ultimately want to decide if you're basing your mod on one particular episode, or creating a "new " one from scratch.

Episode-specific bits of the common code are put behind `#ifdef` guards, like `WITH_KEEN4`. This can help you find other areas you may wish to change.

I'd recommend putting your mod's source files in `src/mod/<mymod>`, and putting any changes to common code behind a `MOD_MYMOD` #define. This will make it easier for you to see what you've changed, and opens the future possibility of packaging mods together.

Similarly, if you wish to keep a copy of the mod's data (either as a whole, or just Omnispeak support files) in the omnispeak source tree, `data/mod/<mymod>` is the place to do so.

### Accessing Variables

While there's no need to use Omnispeak's variable support in a source mode (as any values can be changed in the code directly), it's often useful to do so anyway, as it makes it easier to test and tweak values, without the need to either recompile or understand C.

Variables are accessed by using a series of macros, and may have default values, used if the variable is not set in `EPISODE.CKx`.

You can find these macros in `ck_act.h`:

- `CK_STRING(name)`: get the value of the string variable _name_.
- `CK_INT(name, default)`: get the value of the integer variable _name_, or the value _default_, if the variable is not set.
- `CK_CHUNKNUM(name)`: Get the graphics chunk number for the chunk _name_. This is just an integer variable behind the scenes.
- `CK_CHUNKID(name)`: Get an ID for the chunk _name_, which can later be passed to `CK_LookupChunk()` to get the chunk number. Unlike `CK_CHUNKNUM()`, this ID (of type `chunk_id_t`) is a compile-time constant, so can be used in statically-initialised variables like arrays.
- `CK_SOUNDNUM(name)`: Get the sound effect number for the effect _name_. This is the effect number, not the chunk number (i.e., it's the same for the PC Speaker and Adlib versions), and can be passed to `SD_PlaySound()`.
- `CK_INTARRAY(name)`: Get the integer array variable _name_, which is an array of `intptr_t`. You are encouraged to use `CK_INTELEMENT` instead if you can.
- `CK_INTELEMENT(name, i)`: Returns element _i_ of the integer array _name_. This is preferable to `CK_INTARRAY`, as it will be bounds-checked in debug or hardened builds.
- `CK_STRINGARRAY(name)`: Get the string array variable _name_. You may prefer to use `CK_STRINGELEMENT` instead, if you can.
- `CK_STRINGELEMENT(name, i)`: Returns element _i_ of the string array _name_. This is preferable to `CK_STRINGARRAY`, as it will be bounds-checked in debug or hardened builds.
- `CK_FILENAME(name, default)`: Returns the filename stored in the string variable _name_, or — if the variable doesn't exist — _default_, with its extension adjusted if required.
- `CK_ACTION(name)`: Returns the action with the name _name_.
- `CK_FUNCTION(name)`: Returns the function with the name _name_. This is an action function (i.e., it accepts a single `CK_object` pointer).

Note that, for strings, chunk ids, actions, and arrays, which may be returned as pointers, you should not free them — they are owned by the variable system, and will be valid for the runtime of the game.

Note that Think, Draw, and Collision functions need to be registered before the game loads `EPISODE.CKx`. 

### Porting Between Omnispeak and other Engines

It's possible to port mods written for Omnispeak to CKSRCMOD or keensource456 (or vice-versa) with a little bit of effort. For the most part, this involves fairly mechanical porting from 32/64-bit to 16-bit, but there are a few things to keep in mind.

Porting between Omnispeak and CKSRCMOD is slightly easier than with keensource456, as more of the names are consistent. For most of the gameplay code (which is what mods change most often), CKSRCMOD and Omnispeak are very similar.

There are some in-progress tools which can help generate useful headers:

- idGrab can generate not only Omnispeak-style `GFXCHUNK` and `GFXINFOE` files, but also iGrab headers (both for C and Assembly), which can be used directly with CKSRCMOD or keensource456.
- Omnispeak comes with an (unfinished) tool called `varparser` in the `tools/varparser` directory. With a bit of tweaking, this can convert `EPISODE.CKx` into a series of header and source files which can be used in a CKSRCMOD project so you can access variables and actions.
- CKSRCMOD's action type does not include the `compatDosPointer`. If you want to maintain savegame compatibility, you'll need some other way of doing so. `varparser` can be made to spit out an array mapping the actual action pointer to the `compatDosPointer` and vice-versa. This can be used to implement a slow, but compatible save/load function:
```
uint16_t CK_ActionToPtr16(CK_action *act)
{
	int i = 0;
	for (;CompatPtrs[i].realPtr;i++) {
		if (act == CompatPtrs[i].realPtr)
			return CompatPtrs[i].dosPtr;
	}
	return 0;
}

CK_action *CK_Ptr16ToAction(uint16_t ptr)
{
	int i = 0;
	for (;CompatPtrs[i].realPtr;i++) {
		if (ptr == CompatPtrs[i].dosPtr)
			return CompatPtrs[i].realPtr;
	}
	return 0;
}
```
- You'll probably want to implement a Borland C++ compatible `stdbool.h` and/or `stdint.h` header. You'll also need to be careful: Borland C++ limits variable and function names to 30 characters, and it's easy to exceed that with mod actions.
- You almost certainly will run out of memory, if not overall, then in the data segment. As Keen keeps actions and the objarray in the data segment, reducing the size of these (e.g., by making boolean values 8-bits wide, rather than 16-bits) can save some space.


[Operation Station Infiltration](https://keenwiki.shikadi.net/wiki/Operation_Station_Infiltration) has a CKSRCMOD/Atroxian Realm based version which uses these tricks.