Cached tables

By specifying _KDB_CACHE_TABLE for the optional eCacheOpts flag in KI_OPEN, a programmer to tell KCML to cache a read-only table locally on the application server. This works both for client-server databases like Oracle and for native KDB and can result in a significant performance advantage over accessing the tables in the database directly. Caching is only supported on the default connection as specified in KI_ALLOC_CONNECT.

Suitable candidates for caching are tables that are frequently accessed but rarely updated. They should also be relatively small as there is a limit of at most 65533 rows in a cached table.

Cached tables are created with the database tablename and a .kcache extension in a directory specified by the _KDB_CONN_ATTR_CACHE_DIR connection attribute. The default handle will have this attribute preset to the value of the KCACHE environment variable. If this environment variable is not set then /tmp is used for the default. Any directory must exist and the KCML process must have the privileges necessary to be able to create files in it.

The _KDB_CONN_ATTR_CACHE_DIR connection attribute is defaulted to blank for connection handles other than the default and requests to open a table in cached mode on these connections will be silently ignored with the database table being opened in read mode instead. However the application can set the attribute to a suitable directory at any point up to the KI_OPEN to ensure caching. Cache directories should be unique per database to avoid name clashes. Caching can be disabled for any connection by setting the attribute to blank.

If the cache file for the table already exists in the directory it is mapped into memory and future accesses on that handle will be directed to the cache. If the file does not exist then KI_OPEN will create it and copy the database tables rows into it.

There is a check in KI_OPEN to see if the master table for a cached table has been updated since the cache was created. A change to the master table will only be logged (for caching purposes) once the table handle that was used to make the change is closed. If the master table is newer than the cache the old cache copy will be dropped and a new one created. Other KCML processes on the application server that already have the older cache open and mapped into memory will continue to use the old copy until such time as they close and reopen it.

If you want to force a rebuild of the cache table then pass _KDB_CACHE_REMOVE_FILE as the option in KI_OPEN and any existing table will be dropped and recreated.

Stale cache files should be regularly purged by renaming the directory. Memory mapped files cannot be removed on Windows if they are in use.

Setting eCacheOpts to _KDB_CACHE_PASSTHROUGH enables a passthrough mode whereby all reads come from the cache but some other operations not implemented in the cached table, come from the master table. This requires more resources and should only be used if necessary.

Word search lookups on cached tables are performed with a table-scan.

Compatibility note:

Caching was introduced in KCML 6.60. Word search indexing for cache tables came with KCML 6.65.