Transactions in the native KDB database

KDB transactions guarantee that several changes to several KDB tables are atomic. The KDB journal guarantees the logging of whole KDB transactions. It is intended to preserve the consistency of KDB databases. It has an advantage over just the KDB journal that in addition to guaranteed internal consistency of KCML tables transactions between KCML tables are also guaranteed. The start of a KDB transaction is marked by a call to KI_BEGIN. From this point onwards all transactions are made to an in memory buffer. No updates are made to the KCML tables. The end of the transaction is marked by a call to either KI_COMMIT or KI_ROLLBACK. KI_ROLLBACK throws away the buffer. KI_COMMIT makes all the changes to all the KCML tables. If the KDB journal exists and is enabled KI_COMMIT copies the buffer to the journal log before doing the updates to the KCML tables.

Locking

Various modes of locking can be set with krecover -x or KI_SET_MODE. There are two extremes of locking. One is that locking is expected to be done by the application code. This means that locks should be taken on all the KCML tables before the transaction, and not removed until after a KI_COMMIT. This may require application code changes. The other extreme of locking takes a lock on the KDB log file at a KI_BEGIN and releases it after a KI_COMMIT or KI_ROLLBACK. This form of locking means that there can only be one transaction executed on the system at a time. It is very coarse but is done by the KDB transaction logging code and requires no application code changes.

A better solution than these two extremes is available. This should make migration of existing application code to the use of transactions easier. In this mode, locks taken by the application code are implemented and recorded. When the application releases these locks KDB does not release the locks but records the fact. When a KI_COMMIT or KI_ROLLBACK has been executed the locks taken and released in the transaction are released.

Restrictions

There is a compiled in maximum size to the transaction buffer. If a KI_BEGIN is executed by accident without a matching KI_COMMIT or KI_ROLLBACK, or too many updates are attempted then an update will fail with a KE_TRANSSIZE (32) error. KDB transaction logging is not of much use for whole table type updates. These could be re-executed in the event of a system crash anyway. To enable transaction logging KI_BEGIN, KI_COMMIT and KI_ROLLBACK calls should be added to the application programs. In addition krecover should be started with the appropriate flags to select transaction logging and its locking mode.

It is not possible to nest transactions. Issuing a second KI_BEGIN before a KI_COMMIT or KI_ROLLBACK gives a KE_TRANSSYNTAX (31) error.

KI_OPEN, KI_CLOSE, KI_CREATE, KI_BU_CREATE, KI_GROW and KI_EXTEND are currently not permitted within transactions. All the tables required for a transaction must be opened sometime before the KI_BEGIN and closed sometime after the KI_COMMIT or KI_ROLLBACK. The reason for this is that the journal mechanism depends on the file descriptors remaining unchanged. This restriction may be relaxed in the future.

Additional Functionality

In addition to KCML tables, it may be necessary to update plain files and have these changes included in the journal log and/or in transactions. For this purpose KI_BU_CREATE, KI_BU_OPEN, KI_BU_READ and KI_BU_WRITE have been added.

A feature of some DBMS is automatic rollback. If there is an error in the course of a transaction then the changes made in the transaction are automatically rolled back. This can be enabled in KDB through krecover -x or KI_SET_MODE

There is also a KI_LOG_SYNC call which can be used to issue a sync() and mark the sync in the log in the normal way. This is an expensive call.

For development it may be desirable to have several different KCML database journal systems. The KLOGKEY environment parameter can be used to keep these systems separate. It must be exactly 4 characters long. Its value is used as the key for the shared memory and semaphore that control the journal system. Its default value is "KLOG", 0x424C4F47. For more information see KCML environment variables

Security

The device driver entry points /dev/KDBlogread and /dev/KDBlogwrite give access to the raw disk partition that is used for the KDB journal. This partition contains a copy of all the data written to KCML tables, while the journal has been enabled. In order to maintain security of the information written to KCML tables these devices should only be readable by privileged users. All users must be able to write to the writing device.

Transaction Logging Without the Journal

The KDB journal has an obvious performance overhead. Every write requires a physical write to the journal log. Some situations require the functionality of transactions, but do not want the performance overhead of the KDB journal. This is catered for by having KDB perform all the operations explained above, except actually writing to the log. In addition, if the chosen locking mode is not the coarse KDB log lock, then krecover and its shared memory segment are not required. KDB transaction logging without the KDB journal can be enabled by with KI_SET_MODE.

Setting Up Transaction Locking

If the KDB journal is being used then transaction logging can be enabled with krecover. A useful locking mode could be enabled by adding a -x 0x23 argument. This enables transaction logging with locking checks and automatic rollback on error. The -x argument is a bit field where,

If the KDB journal is not being used then transaction logging can be programmatically enabled with the following code

mode = VAL(HEX(23))
CALL KI_SET_MODE, 1, mode, TO oldMode, ki_status