Every KCML data type — write in KCML, read in Python (worked example)
This example builds on the KISAM create/write/read page. It writes a
single KISAM record carrying one field of every writable KCML data type, then
reads the raw file back in Python with struct — decoding each field from its
on-disk bytes and checking the result against a ground-truth file the KCML program
writes alongside it. It's a guided tour of how KCML actually stores each type.
Verified by execution on KCML 06.00.88 (KClient direct mode); Python 3.12.
What it demonstrates
- Encoding one field of every writable KCML type into a fixed-length KISAM record.
- The exact on-disk byte format of each type — big-endian binary integers, packed BCD numbers, Julian and BCD dates, times, flags and raw bytes.
- Building packed values in KCML with
BIN,HEXPACKandCONVERT DATE(because thePACKstatement is broken in 06.00.88). - Reading the file in Python with
structand verifying every field matches what KCML wrote.
The record layout
A fixed 58-byte record. Byte 1 is the KISAM status byte; the typed columns follow. The "dict" column is the KDB data-dictionary type code (see KDB data types).
| Bytes | Dict | Type | Field | On-disk encoding |
|---|---|---|---|---|
| 1 | — | (status) | — | HEX(00) = live record |
| 2–7 | C | VARCHAR | code (key) | ASCII, space-padded |
| 8–27 | C | VARCHAR | name | ASCII, space-padded |
| 28 | B | INTEGER(1) | b1 | 1-byte big-endian unsigned |
| 29–30 | B | INTEGER(2) | b2 | 2-byte big-endian unsigned |
| 31–34 | B | INTEGER(4) | b4 | 4-byte big-endian unsigned |
| 35–37 | J | DATE | jdate | 3-byte big-endian Julian day number |
| 38–41 | L | BCDDATE | bcddate | 4-byte packed BCD CCYYMMDD |
| 42–49 | K | DECIMAL(15,2) | amount | 8-byte IBM packed BCD, sign nibble last |
| 50–52 | T | TIME | time | 3-byte big-endian seconds since midnight |
| 53 | G | BOOL | active | Y/N byte |
| 54 | F | BIT | flag | Y/N/space byte |
| 55–58 | H | HEX(4) | rawhex | 4 raw bytes |
One record, byte by byte
The first record (code = "P00001") on disk, with each slice labelled:
00 byte 1 status live record
50 30 30 30 30 31 2-7 C code "P00001"
41 6c 69 63 65 ... 8-27 C name "Alice Walker" (space padded)
2a 28 B b1 0x2a = 42
03 e8 29-30 B b2 0x03e8 = 1000
00 0f 42 40 31-34 B b4 0x000f4240 = 1000000
25 8d cf 35-37 J jdate 0x258dcf=2461135 -> 2026-04-04
20 26 04 18 38-41 L bcddate BCD -> 2026-04-18
00 00 00 00 12 34 56 7c 42-49 K amount BCD 000000001234567 + sign C -> 12345.67
00 cb f7 50-52 T time 0x00cbf7=52215 -> 14:30:15
59 53 G active 'Y'
4e 54 F flag 'N'
de ad be ef 55-58 H rawhex DEADBEEF
Two things to notice: every multi-byte number is big-endian (most significant
byte first), and the packed-decimal amount keeps its sign in the last nibble
(c = positive, d = negative).
Writing it in KCML
The program creates the file (see the KISAM page for KI_CREATE
details), then for each record fills every field with the right encoding and calls
KI_WRITE. It also writes a pipe-delimited types_truth.txt so the Python reader
has something to check itself against.
01000 REM demo_kisam_types - KISAM record with one field of every writable KCML type
: REM Layout reclen 58 - byte 1 status, 2-7 code C, 8-27 name C, 28 b1 B1
: REM 29-30 b2 B2, 31-34 b4 B4, 35-37 jdate J, 38-41 bcddate L, 42-49 amount K
: REM 50-52 time T, 53 active G, 54 flag F, 55-58 rawhex H
: DIM result, ki_status, handle, ki_sym, ki_dataptr$6, ki_key$64
: DIM keyspec$(1)33, nrows, reclen, ftype, dictfile$2, file$120, rec$58
: DIM sv_i, jval, secs, bcd_val, hh, mm, ss
: DIM n_s$20, pad_s$40, sign$1, hex_s$20, kpacked$8, bcdp$4, rawp$4, gt$200
: DIM t_code$(2)6, t_name$(2)20, t_b1(2), t_b2(2), t_b4(2)
: DIM t_jd$(2)10, t_bcd$(2)8, t_amt(2), t_time(2), t_act$(2)1, t_flg$(2)1, t_raw$(2)8
: file$ = "C:/kisam_data/PEOPLE_TYPES"
: DIM gtfile$120
: gtfile$ = "C:/kisam_data/types_truth.txt"
: nrows = 100 : reclen = 58 : ftype = 6 : dictfile$ = " "
: t_code$(1) = "P00001" : t_name$(1) = "Alice Walker"
: t_b1(1) = 42 : t_b2(1) = 1000 : t_b4(1) = 1000000
: t_jd$(1) = "2026-04-04" : t_bcd$(1) = "20260418" : t_amt(1) = 12345.67
: t_time(1) = 52215 : t_act$(1) = "Y" : t_flg$(1) = "N" : t_raw$(1) = "DEADBEEF"
: t_code$(2) = "P00002" : t_name$(2) = "Bob Stone"
: t_b1(2) = 7 : t_b2(2) = 255 : t_b4(2) = 70000
: t_jd$(2) = "2025-12-25" : t_bcd$(2) = "20251225" : t_amt(2) = -89.50
: t_time(2) = 32700 : t_act$(2) = "N" : t_flg$(2) = "Y" : t_raw$(2) = "0BADF00D"
01010 - DEFFORM TypesDemo()=\
{.form,.form$,.Style=0x50c000c4,.Width=520,.Height=130,.Text$="KISAM all types",.Id=1024},\
{.lblOut,.static$,.Style=0x50000000,.Left=10,.Top=20,.Width=500,.Height=50,.Text$="(running)",.Id=2001,.Font=.UI},\
{.btnClose,.button$,.Style=0x50010001,.Left=425,.Top=90,.Width=80,.Height=15,.Text$="Close",.Id=1,.Font=.UI},\
{.paneStatus,.status$,.Width=520,.Style=0x50000000,.Text$="Ready"},\
{.UI,.dlgfont$,.Name$="Segoe UI",.Size=10}
: + DEFEVENT TypesDemo.Enter()
: keyspec$(1) = ALL(HEX(00))
: STR(keyspec$(1), 1, 1) = "N"
: STR(keyspec$(1), 2, 2) = BIN(2, 2)
: STR(keyspec$(1), 4, 1) = BIN(6)
: CALL KI_ALLOC_HANDLE 0, 1 TO handle, ki_status
: CALL KI_CREATE handle, file$, dictfile$, nrows, reclen, SYM(keyspec$()), 4, 50, ftype TO ki_status
: OPEN #1, gtfile$, "w"
: FOR sv_i = 1 TO 2
: rec$ = ALL(HEX(20))
: STR(rec$, 1, 1) = HEX(00)
: STR(rec$, 2, 6) = t_code$(sv_i)
: STR(rec$, 8, 20) = t_name$(sv_i)
: STR(rec$, 28, 1) = BIN(t_b1(sv_i), 1)
: STR(rec$, 29, 2) = BIN(t_b2(sv_i), 2)
: STR(rec$, 31, 4) = BIN(t_b4(sv_i), 4)
: CONVERT DATE t_jd$(sv_i) TO jval
: STR(rec$, 35, 3) = BIN(jval, 3)
: HEXPACK bcdp$ FROM t_bcd$(sv_i)
: STR(rec$, 38, 4) = bcdp$
: bcd_val = ROUND(ABS(t_amt(sv_i)) * 100, 0)
: n_s$ = $PRINTF("%d", bcd_val)
: pad_s$ = "000000000000000" & RTRIM(n_s$)
: sign$ = "C"
: IF t_amt(sv_i) < 0 THEN sign$ = "D"
: hex_s$ = STR(RTRIM(pad_s$), LEN(RTRIM(pad_s$)) - 14, 15) & sign$
: HEXPACK kpacked$ FROM hex_s$
: STR(rec$, 42, 8) = kpacked$
: STR(rec$, 50, 3) = BIN(t_time(sv_i), 3)
: STR(rec$, 53, 1) = t_act$(sv_i)
: STR(rec$, 54, 1) = t_flg$(sv_i)
: HEXPACK rawp$ FROM t_raw$(sv_i)
: STR(rec$, 55, 4) = rawp$
: ki_sym = SYM(rec$)
: CALL KI_WRITE handle, ki_sym, 0 TO ki_status, ki_dataptr$
: hh = INT(t_time(sv_i) / 3600)
: mm = INT((t_time(sv_i) - hh * 3600) / 60)
: ss = t_time(sv_i) - hh * 3600 - mm * 60
: gt$ = $PRINTF("%s|%s|%d|%d|%d", RTRIM(t_code$(sv_i)), RTRIM(t_name$(sv_i)), t_b1(sv_i), t_b2(sv_i), t_b4(sv_i))
: gt$ = gt$ & $PRINTF("|%s|%s-%s-%s", RTRIM(t_jd$(sv_i)), STR(t_bcd$(sv_i),1,4), STR(t_bcd$(sv_i),5,2), STR(t_bcd$(sv_i),7,2))
: gt$ = gt$ & $PRINTF("|%.2f|%02d:%02d:%02d|%s|%s|%s", t_amt(sv_i), hh, mm, ss, RTRIM(t_act$(sv_i)), RTRIM(t_flg$(sv_i)), RTRIM(t_raw$(sv_i)))
: PRINT #1, RTRIM(gt$)
: NEXT sv_i
: CALL KI_CLOSE handle TO ki_status
: CLOSE #1
: .lblOut.Text$ = "Wrote 2 all-type records + types_truth.txt. reclen=58."
: .paneStatus.Text$ = $PRINTF("create status %d", ki_status)
: END EVENT
: FORM END TypesDemo
01020 result = TypesDemo.Open()
: $END
How each type is written:
- C (string) —
STR(rec$, off, len) = "text". Fixed-length, space-padded. - B (integer) —
STR(rec$, off, n) = BIN(value, n)builds ann-byte big-endian unsigned integer (the inverse ofVAL). - J (Julian date) —
CONVERT DATE "2026-04-04" TO jvalgives the Julian day number (a plain integer), stored as 3 bytes withBIN(jval, 3). - L (BCD date) —
HEXPACK bcdp$ FROM "20260418"packs the 8 digits into 4 BCD bytes (CCYYMMDD). - K (packed decimal) —
PACKis broken in 06.00.88, so build the BCD hex string by hand (15 digits + a sign nibbleC/D) andHEXPACKit. - T (time) — seconds since midnight via
BIN(seconds, 3). - G/F (flags) — a single
"Y"/"N"byte. - H (raw) —
HEXPACK rawp$ FROM "DEADBEEF"for arbitrary bytes.
Reading it in Python
No KCML here — just struct and a few small decoders. The reader pulls reclen
from the header, finds the data records, and decodes each field by type, then
verifies every value against types_truth.txt.
#!/usr/bin/env python3
"""
Read a type-6 KISAM file written by KCML using struct.
The companion KCML program writes a 58-byte record carrying one field of every
writable KCML data type. This script reads it with no KCML involved: it parses
the header with struct, locates the data records, decodes every field by its
on-disk encoding, and checks the result against the ground-truth file.
"""
import struct
import sys
from pathlib import Path
HERE = Path(__file__).resolve().parent
DATA = HERE.parent / "forms" / "kisam_data"
KISAM_FILE = DATA / "PEOPLE_TYPES"
TRUTH_FILE = DATA / "types_truth.txt"
JDN_GREGORIAN_OFFSET = 1721425 # ordinal(date) == JDN - this; date(1,1,1)==ordinal 1
# ---- field decoders ---------------------------------------------------------
def dec_str(b): # C / VARCHAR
return b.decode("ascii", "replace").rstrip()
def dec_uint(b): # B / INTEGER (1, 2 or 4 bytes), big-endian
# struct has no 3-byte int, so left-pad to 4 bytes and read as >I
return struct.unpack(">I", b.rjust(4, b"\x00"))[0]
def dec_julian(b): # J / DATE -- 3-byte big-endian Julian day
jdn = struct.unpack(">I", b.rjust(4, b"\x00"))[0]
import datetime
return datetime.date.fromordinal(jdn - JDN_GREGORIAN_OFFSET).isoformat()
def dec_bcd_date(b): # L / BCDDATE -- 4-byte packed BCD CCYYMMDD
d = b.hex() # e.g. "20260418"
return f"{d[0:4]}-{d[4:6]}-{d[6:8]}"
def dec_packed_decimal(b, scale=2): # K / DECIMAL -- IBM packed BCD, sign nibble last
nibbles = b.hex() # 16 hex chars for 8 bytes
digits, sign = nibbles[:-1], nibbles[-1].lower()
value = int(digits) / (10 ** scale)
if sign in ("d", "b"): # C/A/F/E = positive, D/B = negative
value = -value
return value
def dec_time(b): # T / TIME -- 3-byte BE seconds since midnight
secs = struct.unpack(">I", b.rjust(4, b"\x00"))[0]
return f"{secs // 3600:02d}:{secs % 3600 // 60:02d}:{secs % 60:02d}"
def dec_char(b): # G / F -- single byte
return b.decode("ascii", "replace").strip()
def dec_hex(b): # H -- raw bytes as hex
return b.hex().upper()
# field = (name, offset_1based, length, decoder)
FIELDS = [
("code", 2, 6, dec_str),
("name", 8, 20, dec_str),
("b1", 28, 1, dec_uint),
("b2", 29, 2, dec_uint),
("b4", 31, 4, dec_uint),
("jdate", 35, 3, dec_julian),
("bcddate",38, 4, dec_bcd_date),
("amount", 42, 8, dec_packed_decimal),
("time", 50, 3, dec_time),
("active", 53, 1, dec_char),
("flag", 54, 1, dec_char),
("rawhex", 55, 4, dec_hex),
]
# ---- KISAM container parsing ------------------------------------------------
def read_header(buf):
"""Return (magic, reclen) from the type-6 KISAM header.
Magic 'K6' at bytes 0-1; record length as big-endian uint16 at bytes 4-5."""
magic = buf[0:2].decode("ascii", "replace")
(reclen,) = struct.unpack_from(">H", buf, 4)
return magic, reclen
def looks_like_record(buf, off, reclen):
"""A live data record: status 0x00, printable key, printable name. This
rejects index-block entries (same keys, but pointer bytes where the name
would be)."""
if off + reclen > len(buf):
return False
if buf[off] != 0x00:
return False
key = buf[off + 1:off + 7]
name = buf[off + 7:off + 27]
ok = lambda bs: all(0x20 <= c <= 0x7E for c in bs)
return ok(key) and key[0:1] != b" " and ok(name)
def find_records(buf, reclen):
"""Yield record byte-slices: find the first valid record, step by reclen
while they stay valid, resume scanning across gaps."""
off, n = 0, len(buf)
while off + reclen <= n:
if looks_like_record(buf, off, reclen):
yield buf[off:off + reclen]
off += reclen
else:
off += 1
# ---- main -------------------------------------------------------------------
def decode_record(rec):
out = {}
for name, pos, length, fn in FIELDS:
out[name] = fn(rec[pos - 1:pos - 1 + length]) # 1-based -> 0-based slice
return out
def main():
buf = KISAM_FILE.read_bytes()
magic, reclen = read_header(buf)
print(f"file : {KISAM_FILE.name} ({len(buf)} bytes)")
print(f"header : magic={magic!r} reclen={reclen}")
if magic != "K6":
sys.exit("not a type-6 KISAM file (magic != 'K6')")
records = [decode_record(r) for r in find_records(buf, reclen)]
print(f"records: {len(records)} found\n")
cols = [f[0] for f in FIELDS]
widths = {c: max(len(c), *(len(str(r[c])) for r in records)) for c in cols}
header = " ".join(c.ljust(widths[c]) for c in cols)
print(header)
print("-" * len(header))
for r in records:
print(" ".join(str(r[c]).ljust(widths[c]) for c in cols))
# verify against the KCML-written ground truth
if not TRUTH_FILE.exists():
return
truth = [ln for ln in TRUTH_FILE.read_text().splitlines() if ln.strip()]
print("\nVerification vs KCML ground truth:")
all_ok = True
for r, line in zip(records, truth):
expected = dict(zip(cols, line.split("|")))
for c in cols:
got, want = r[c], expected[c]
if c in ("b1", "b2", "b4"):
ok = int(got) == int(want)
elif c == "amount":
ok = abs(float(got) - float(want)) < 1e-9
elif c == "time": # KCML %02d space-pads -> normalize
ok = [int(x) for x in str(got).split(":")] == [int(x) for x in want.split(":")]
else:
ok = str(got) == want
all_ok &= ok
if not ok:
print(f" MISMATCH {expected['code']}.{c}: decoded={got!r} truth={want!r}")
print(" ALL FIELDS MATCH" if all_ok else " *** MISMATCHES ABOVE ***")
if __name__ == "__main__":
main()
Running it against the file the KCML program wrote:
file : PEOPLE_TYPES (13312 bytes)
header : magic='K6' reclen=58
records: 2 found
code name b1 b2 b4 jdate bcddate amount time active flag rawhex
P00001 Alice Walker 42 1000 1000000 2026-04-04 2026-04-18 12345.67 14:30:15 Y N DEADBEEF
P00002 Bob Stone 7 255 70000 2025-12-25 2025-12-25 -89.5 09:05:00 N Y 0BADF00D
Verification vs KCML ground truth:
ALL FIELDS MATCH
struct does the fixed-width binary fields directly (>H for the header,
>I over a left-padded slice for the 1/2/3/4-byte integers and the Julian/time
values). BCD isn't a struct format, so the packed date and packed-decimal fields
use small nibble decoders.
Types not covered (and why)
| Dict | Type | Why it's omitted |
|---|---|---|
P |
NUMERIC (KCML native packed) | The PACK statement is broken in 06.00.88 (S12); the IBM-packed K type covers packed decimals. |
N |
INTEGER(4) internal | Documented as a non-portable internal representation — not meant to be decoded outside KCML. |
M |
TIMESTAMP | Milliseconds since year 0000 GMT — an awkward epoch with no clean writer here. |
O / Q |
BLOB / CBLOB | Type-7 tables only, which require a KDB database connection (unavailable in a plain direct-mode session). |
See also
- KISAM files — create, write and read — the foundation example
- KDB data types — the full dictionary type catalogue