(CSW ONLY) To generate additional errors on adjacent pairs of blocks with inconsistent polarities, add +p: $ php -f lsblocks.php +p If there is a particular block that is of interest (usually because it contains errors), you can request verbose details of that particular block based on its block ID (which will have been displayed on the prior run): $ php -f lsblocks.php +b +v +e (+v provides a verbose block listing; +e provides a verbose error listing.) The +k option will LIST any tokenised BASIC files that are found on the tape. One other useful trick that this tool can perform is to extract all of the 8N1 blocks from a CSW or TIBET into a directory, as individual files: $ php -f lsblocks.php +x

At this time, this software will automatically create the directory if it does not exist, and it has no qualms about overwriting existing files, so be careful. This behaviour can be altered easily (change "TRUE, TRUE" to "FALSE, FALSE" in the call to save_blocks), but these choices are not exposed via the command line right now. Currently, this software provides no way to extract whole CFS files rather than individual blocks, but the blocks it does write can be manually concatenated into complete files fairly easily. On Unixalikes a shell command such as the following will probably do the job, once this software has extracted the blocks to, say, "/tmp/src", assuming you want the files in "/tmp/dst": SRC="/tmp/src" ; DST="/tmp/dst" for N in `ls "${SRC}" | cut -d _ -f 3-4 | sort | uniq` ; do cat "${SRC}/"*${N}* >> "${DST}/`echo ${N} | cut -d _ -f 1`" ; done WHY? ---- This software was written for testing CSWs produced by Quadbike. I was prevously using beebjit in an ad-hoc way for this purpose, but decided I needed something better. It is expected that this software will be useful for anyone else who is unfortunate enough to be developing software to convert an audio file into CSW or TIBET data. From this perspective, it offers two key innovations: i) Every byte decoded from the CSW is stored along with the sample number within the CSW where that byte originated. For TIBET, bytes are stored along with the TIBET line number whence they came. As such, it is very useful for determining whereabouts in the original audio file certain features lie. Header field locations, data locations, CRC locations and any error locations are all displayed to the user, in terms of the number of samples into the source audio file that was used to generate the CSW. For TIBET files, it will display the line number. ii) An error count and block count is provided at the end of the block decoding process. In my adventures with Quadbike, I have found that making a change to an algorithm often improves transcription of some tapes, but only at the expense of other ones. By having a large corpus of test data, and by summing errors over this entire corpus, it should be possible to measure scientifically the overall efficacy of any change to an audio decoding algorithm (hopefully including my own). Unlike beebjit's CSW error reporting, this software only counts errors that occur within blocks -- so errors caused by transients on the tape during silent sections or leader sections will not contribute to the error count. It also reports the sample number of stop bit errors, which beebjit does not at this time. Additionally, beebjit's CSW-loading heuristic doesn't play nicely with Quadbike; it was intended for dealing with output from CSW.exe, which measures pulse lengths from a tape. Quadbike, however, artificially synthesises pairs of pulses based on frequency data, so using a hard threshold between 1-pulses and 0-pulses, as e.g. B-Em does, works much better for Quadbike's CSW output. */ // returned errors define ("X_E_OK", 0); define ("X_E_BUG", 1); define ("X_E_CLI", 2); define ("X_E_CSW_NOT_FOUND", 3); define ("X_E_CSW_LOAD_ERROR", 4); define ("X_E_CSW_COMPRESSION_TYPE", 5); define ("X_E_CSW_FLAGS", 6); define ("X_E_CSW_MAGIC", 7); define ("X_E_CSW_UNZIP", 8); define ("X_E_CSW_FILENAME_LONG", 9); define ("X_E_HELP", 10); define ("X_E_CSW_DECODE_BODY", 11); define ("X_E_CSW_VERSION", 12); define ("X_E_SAVE_NO_DIR", 50); define ("X_E_SAVE_MKDIR", 51); define ("X_E_SAVE_WRITE", 52); define ("X_E_SAVE_FILE_EXISTS", 53); define ("X_E_SAVE_DIR_IS_FILE", 56); // internal block errors define ("BLK_E_NONE", 100); define ("BLK_E_BAD_STOP_BIT", 101); define ("BLK_E_PULSE_SEQUENCE", 102); define ("BLK_E_PULSE_LEN", 103); define ("BLK_E_FILENAME_LONG", 104); define ("BLK_E_SILENCE", 105); // internal synthetic errors define ("BLK_E_DATA_CRC", 200); // } CRC errors do not occur in bytestreams, hence have no smpnum, define ("BLK_E_HEADER_CRC", 201); // } but if they are detected, they will be attached to the block define ("BLK_E_POLARITY", 202); // - same for unexpected polarity switches define ("BLK_E_BLOCKLEN_LARGE", 203); // internal data errors define ("DAT_E_OK", 300); define ("DAT_E_SHORT", 301); define ("DAT_E_CRC", 302); define ("TBT_E_PARSE_VERSION", 400); define ("TBT_E_BAD_VERSION", 401); define ("TBT_E_PARSE_BAD_LINE", 402); define ("TBT_E_PARSE_SILENCE", 403); define ("TBT_E_BAD_SILENCE", 404); define ("TBT_E_PARSE_LEADER", 405); define ("TBT_E_BAD_LEADER", 406); define ("TBT_E_PARSE_DATA", 407); define ("TBT_E_BAD_PHASE", 408); define ("TBT_E_PARSE_CYCLES", 409); define ("TBT_E_BUG", 410); define ("TBT_E_WRITE_FILE", 411); define ("TBT_E_PARSE_HINT", 412); define ("TBT_E_BAD_FRAMING", 413); define ("TBT_E_PARSE_DUP_VERSION", 414); define ("TBT_E_BAD_INT", 415); define ("TBT_E_BAD_FLOAT", 416); define ("TBT_E_GZIP", 417); // 0.7: handle void chunks define ("TBT_E_ZL_CHUNK", 418); // 0.8: major version mismatch define ("TBT_E_INCOMPATIBLE", 419); //define ("TK_E_OK", 0); define ("TK_E_CLI", 500); define ("TK_E_NULL", 501); define ("TK_E_FILE_NOT_FILE", 502); define ("TK_E_FOPEN_READ_FILE", 503); define ("TK_E_FILE_TOOLARGE", 504); define ("TK_E_FREAD", 505); define ("TK_E_FREAD_EOF", 506); define ("TK_E_STAT", 507); define ("TK_E_OVERFLOW", 508); define ("TK_E_TOKEN", 509); define ("TK_E_MALLOC", 510); define ("TK_E_BUG", 511); $block_error_names = array( BLK_E_BAD_STOP_BIT => "Bad stop bit", BLK_E_PULSE_SEQUENCE => "Bad pulse seq", BLK_E_PULSE_LEN => "Bad pulse len", BLK_E_FILENAME_LONG => "Bad filename", BLK_E_SILENCE => "Silence", // internal, not printed BLK_E_DATA_CRC => "Data CRC mismatch", BLK_E_HEADER_CRC => "Header CRC mismatch", BLK_E_POLARITY => "Polarity switch", // if +p //BLK_E_WACKY_BLOCKLEN => "Bad blocklen" BLK_E_BLOCKLEN_LARGE => "Block len > 256 [high byte was zeroed]" ); define ("TIBET_MAJOR_VERSION", "0"); define ("CSW_FREQ_1", 1201.9); define ("CSW_FREQ_2", CSW_FREQ_1 * 2.0); define ("MAX_OP_FILE_LEN", 1000000000); // 1 GB define ("MAX_PULSE_DURATION_SMPS", 60000000); define ("MIN_VALID_LEADER_LEN_S", 0.1); // require 0.1s leadertone before block define ("MIN_SILENCE_SMPS", 200); // block decode state machine define ("STATE_NONE", 0); define ("STATE_IDLE", 1); define ("STATE_FILENAME", 2); define ("STATE_LOAD", 3); define ("STATE_EXECUTE", 4); define ("STATE_BLOCKNUM", 5); define ("STATE_DATA_LEN", 6); define ("STATE_NEXTFILE", 7); define ("STATE_FLAGS", 8); define ("STATE_HCRC", 9); define ("STATE_DATA", 10); define ("STATE_DCRC", 11); $state_names = array( STATE_NONE => "", STATE_IDLE => "need sync", STATE_FILENAME => "filename", STATE_LOAD => "load address", STATE_EXECUTE => "execute address", STATE_BLOCKNUM => "block number", STATE_DATA_LEN => "data length", STATE_NEXTFILE => "next file address", STATE_FLAGS => "block flags", STATE_HCRC => "header CRC", STATE_DATA => "data", STATE_DCRC => "data CRC" ); define ("CSW_BLOCK_FLAGS_FINAL", 0x80); // bit 7 of flags set if final block define ("CSW_BLOCK_FLAGS_EMPTY", 0x40); // bit 6 set if data length is 0 define ("CSW_BLOCK_FLAGS_LOCKED", 0x1); // bit 0 set if locked define ("SAVE_BIN_EXTENSION", ".bin"); define ("DETOK_STATE_ASCII", 0); define ("DETOK_STATE_EXPECT_LINENUM_1", 1); define ("DETOK_STATE_EXPECT_LINENUM_2", 2); define ("DETOK_STATE_EXPECT_LINELEN", 3); define ("DETOK_STATE_EXPECT_TOKEN", 4); define ("DETOK_STATE_DECODE_LINENUM_1", 5); define ("DETOK_STATE_DECODE_LINENUM_2", 6); define ("DETOK_STATE_DECODE_LINENUM_3", 7); // *** BEGIN TIBET STUFFS *** define ("TBT_STATE_VERSION", 0); define ("TBT_STATE_IDLE", 1); define ("TBT_STATE_CYCLES", 2); class Span { var $linenum; var $span_ix; } class TimeHint extends Span { var $timestamp; } class ParsedTibet { var $version; var $spans; function __construct() { $this->spans = array(); $this->version = ""; } } class TibetSilence extends Span { var $secs; } class TibetLeader extends Span { var $cycles; } class DataFraming extends Span { var $framelen; // 7 or 8: FIXME: rename to wordlen var $parity; // string; "N", "O", "E" var $stops; // 1 or 2 //var $autodetected; function to_string() : string { return "$this->framelen$this->parity$this->stops"; } function __construct() { // defaults $this->framelen = 8; $this->parity = "N"; $this->stops = 1; } } class BaudRate extends Span { var $rate; function to_string() : string { return "$this->rate"; } function __construct() { // default $this->rate = 1200; } } class TibetData extends Span { var $squawk; var $cycles; // array var $bits; var $framing; // DataFraming function __construct() { $this->cycles = array(); // TibetCycles $this->bits = array(); // also TibetCycles ... $this->squawk = 0; } } class TibetCycle extends Span { var $value; // 0 or 1 var $smpnum; // CSW var $tibet_linenum; // or TIBET (indexed from 0; add 1 for printout) var $tibet_line; } class DummyByte extends Span { var $pre_leader_cycs; var $post_leader_cycs; var $byte_value; } // *** END TIBET STUFF *** // NOT USED YET -- 8N1 ASSUMED class BlockFraming { var $data; var $parity; var $stop; function __construct() { $this->data = 8; $this->parity = FALSE; $this->stop = 1; } function parse($s) : int { print "B: FIXME: block framing not parsed yet\n"; return X_E_CLI; } } class CswGaps { // all measured in samples: var $thresh; // the critical length at which a fast 1200 cycle becomes a slow 2400 cycle var $max_valid_1200; var $min_valid_2400; var $ideal_2400; function __construct ($rate) { $rate = (float) $rate; $tape_speed = 1.0; // ideal half-cycle (one pulse) lengths $ideal_2400 = $rate / (CSW_FREQ_2 * 2.0 * $tape_speed); $ideal_1200 = $ideal_2400 * 2.0; $this->thresh = (int) round (1.5 * $ideal_2400); // walk mode can need a surprisingly big number here; // 0.6 was needed to load electron snapper side A $tol = 0.6 * $ideal_2400; $this->max_valid_1200 = (int) round ($ideal_1200 + $tol); $this->min_valid_2400 = (int) round ($ideal_2400 - $tol); $this->ideal_2400 = (int) $ideal_2400; if ($this->min_valid_2400 < 0) { print "BUG: tol is set too high\n"; exit(X_E_BUG); } return $this; } } class CswPulse { var $len_smps; var $smpnum; // CSW var $tibet_linenum; // or TIBET (indexed from 0; add 1 for printout) var $tibet_line; } class ByteError { var $e; var $smpnum; // CSW var $tibet_linenum; // or TIBET (indexed from 0; add 1 for printout) var $tibet_line; var $pulses; // CswPulse[]: we'll use this for TIBET too var $state; // state machine value on error //static function get (int $type, int $smpnum) { // print "ByteError::get() is dead! Bailing\n"; // die(); //} static function get_csw (int $type, int $smpnum) : ByteError { $e = new ByteError; $e->e = $type; $e->smpnum = $smpnum; $e->state = STATE_NONE; return $e; } static function get_tibet (int $type, int $tibet_linenum, string $tibet_line) : ByteError { $e = new ByteError; $e->e = $type; $e->tibet_linenum = $tibet_linenum; $e->tibet_line = $tibet_line; $e->state = STATE_NONE; return $e; } static function get2 (int $type) : ByteError { return ByteError::get_csw($type, -1); } function __construct() { $this->e = BLK_E_NONE; $this->smpnum = -1; $this->tibet_linenum = -1; $this->tibet_line = ""; $this->pulses = array(); $this->state = STATE_IDLE; } function get_name() : string { global $block_error_names; return $block_error_names[$this->e]; } function is_pulse_error() : bool { return ($this->e == BLK_E_PULSE_LEN) || ($this->e == BLK_E_PULSE_SEQUENCE); } function to_string() : string { global $state_names; //$s = $this->get_name().": ".$this->smpnum; if (-1 != $this->tibet_linenum) { // TIBET $s = "{ L".sprintf("%5d",(1+$this->tibet_linenum))." } "; } else if (-1 == $this->smpnum) { //print_r($this); $s = "[ N/A ] "; } else { $s = "[".sprintf("%8d", $this->smpnum)."] "; } $s .= $this->get_name(); if ($this->state != STATE_NONE) { $s .= " (" . $state_names[$this->state] . ") "; } if ($this->is_pulse_error()) { $s .= ": ".$this->pulses[0]->len_smps.","; $s .= $this->pulses[1]->len_smps.","; $s .= $this->pulses[2]->len_smps.","; $s .= $this->pulses[3]->len_smps." "; } return $s; } } class DecodedByte { var $value_int; var $value_chr; var $smpnum; // CSW only var $tibet_linenum; // ixed from 0 var $tibet_line; var $error; var $leader_len_smps; var $polarity; // CSW only function __construct() { $this->error = new ByteError; } } class BlockField { public int $i; public array $a; public int $smpnum; public int $tibet_linenum; public string $tibet_line; public string $s; function __construct() { $this->i = 0; $this->a = array(); $this->s = ""; $this->smpnum = -1; $this->tibet_linenum = -1; $this->tibet_line = ""; } function format_ctxt() : string { if ($this->tibet_linenum != -1) { // TIBET return "{ L".sprintf("%5d",(1+$this->tibet_linenum))." } "; } else if ($this->smpnum > 0) { return "[".sprintf("%8d",$this->smpnum)."]"; } else { return " "; } } } class BeebBlock { var $ix; var $smpnum; var $tibet_linenum; var $tibet_line; // fields will be of type BlockField to store sample nums: public BlockField $name; public BlockField $load; public BlockField $ex; public BlockField $blknum; public BlockField $blklen; public BlockField $flag; public BlockField $nextaddr; public BlockField $hcrc_read; public BlockField $data; public BlockField $dcrc_read; // TODO: var $len, $parity, $stop var $hcrc_computed; var $dcrc_computed; var $errors; function __construct($blkn) { $this->ix = $blkn; $this->smpnum = -1; $this->tibet_line = ""; $this->tibet_linenum = -1; $this->name = new BlockField; $this->load = new BlockField; $this->ex = new BlockField; $this->blknum = new BlockField; $this->blklen = new BlockField; $this->flag = new BlockField; $this->nextaddr = new BlockField; $this->hcrc_read = new BlockField; $this->dcrc_read = new BlockField; $this->data = new BlockField; $this->dcrc_computed = 0; $this->hcrc_computed = 0; $this->errors = array(); } function printable_name() : string { $name = ""; for ($i=0; $i < strlen($this->name->s); $i++) { $c = $this->name->s[$i]; if (printable($c)) { $name .= $c; } else { $name .= "?"; $badchar = 1; } } return $name; } static function heading (bool $is_tibet) : string { // - block ID // - sample number // - MOS filename // - load address // - execution address // - MOS block number // - block length // - next file address // - flags (L=locked, E=empty, F=final) // - header CRC error (HC) // - data CRC error (DC), or data incomplete (DI) $s = ""; if ( ! $is_tibet ) { $s = "----|----------|------------|--------|--------|----|----|--------|---|--|--\n". " id smpnum filename load exec num len nextfile flg hE dE\n". "----|----------|------------|--------|--------|----|----|--------|---|--|--"; } else { $s = "----|----------|------------|--------|--------|----|----|--------|---|--|--\n". " id linenum filename load exec num len nextfile flg hE dE\n". "----|----------|------------|--------|--------|----|----|--------|---|--|--"; } return $s; } function to_string (bool $vrb, bool $vrb_e) : string { if ( ! $vrb ) { return $this->get_summary ($vrb_e); } else { return $this->get_multiline ($vrb_e); } } function flags_string() : string { return (($this->flag->i & CSW_BLOCK_FLAGS_FINAL) ? "F" : " "). (($this->flag->i & CSW_BLOCK_FLAGS_EMPTY) ? "E" : " "). (($this->flag->i & CSW_BLOCK_FLAGS_LOCKED)? "L" : " "); } function get_summary(bool $vrb_e) : string { global $block_error_names, $state_names; $s = ""; $badchar = 0; // one-line summary $name = $this->printable_name(); $name_q = str_pad("\"$name\"", 12, " ", STR_PAD_LEFT); //print "tibet_linenum = $this->tibet_linenum\n"; $s = sprintf("#%3d",$this->ix)." ". (($this->tibet_linenum == -1) ? sprintf("[%8d]",$this->smpnum) : sprintf("{ L%5d }",(1+$this->tibet_linenum)))." ". $name_q." ". sprintf("%8x",$this->load->i)." ". sprintf("%8x",$this->ex->i)." ". sprintf("%4x",$this->blknum->i)." ". sprintf("%4x",$this->blklen->i)." ". sprintf("%8x",$this->nextaddr->i)." ". $this->flags_string()." ". (($this->hcrc_read->i == $this->hcrc_computed) ? " " : "HC")." "; if ($this->data_ok() == DAT_E_SHORT) { $s.="DI"; //$s.=" ".sprintf("%3x",count($this->data->a)); } else if ($this->data_ok() == DAT_E_CRC) { $s.="DC"; } else { $s.=" "; } $s .= $this->print_errors($vrb_e); return $s; } function detokenise(string &$s_out) : int { $detok = new Detokeniser($this); return $detok->go_baby_go($s_out); } function print_errors (bool $multi) : string { $c = count($this->errors); $d = $c; $s=""; if ( ! $multi && ($c>0) ) { $d = 1; // limit to 1 } for ($i=0; $i < $d; $i++) { $s.= "\n"; $s.= " > ".$this->errors[$i]->to_string(); } if ( ! $multi && ($c > 1) ) { $s .= " (".($c-1)." more errors ...)"; // more errors not shown } if ($c>0) { $s .= "\n"; } return $s; } function get_multiline (bool $vrb_e) : string { // convert name string to array: $i=0; $name_a = array(); for ($i=0; $i < strlen($this->name->s); $i++) { $cb = new DecodedByte; $cb->value_int = ord($this->name->s[$i]); $cb->value_chr = $this->name->s[$i]; if ($this->name->tibet_linenum != -1) { $cb->tibet_linenum = $this->name->tibet_linenum; $cb->tibet_line = $this->name->tibet_line; } else { $cb->smpnum = $this->name->smpnum; } $name_a[$i] = $cb; } $s=""; $s .= " ID #".$this->ix."\n"; $s .= $this->name->format_ctxt()." name (hexdump follows):\n"; // ".str_pad("\"".$this->name->s."\"",12," ",STR_PAD_LEFT)."\n"; $s .= my_hexdump($name_a, ($this->name->tibet_linenum != -1), FALSE); // FALSE => don't include offset $s .= $this->load->format_ctxt()." load address ".sprintf("%8x",$this->load->i)."\n"; $s .= $this->ex->format_ctxt()." execution address ".sprintf("%8x",$this->ex->i)."\n"; $s .= $this->blknum->format_ctxt()." MOS block number ".sprintf("%8x",$this->blknum->i)."\n"; $s .= $this->blklen->format_ctxt()." data length ".sprintf("%8x",$this->blklen->i)."\n"; $s .= $this->nextaddr->format_ctxt()." next file address ".sprintf("%8x",$this->nextaddr->i)."\n"; $s .= $this->flag->format_ctxt()." flags (final/empty/lock) ".sprintf("%8x",$this->flag->i)." (".$this->flags_string().")\n"; $s .= $this->hcrc_read->format_ctxt()." hCRC (read/computed) ". sprintf("%04x / ",$this->hcrc_read->i). sprintf("%04x",$this->hcrc_computed). "\n"; $s .= $this->dcrc_read->format_ctxt()." dCRC (read/computed) ". sprintf("%04x / ",$this->dcrc_read->i). sprintf("%04x",$this->dcrc_computed). "\n"; if (count($this->data->a)) { $s .= $this->data->format_ctxt()." data (hexdump follows):\n".my_hexdump($this->data->a, ($this->data->tibet_linenum != -1), TRUE); // TRUE => include offset } $s .= $this->print_errors($vrb_e); return $s; } function data_ok() : int { if (count($this->data->a) != ($this->blklen->i)) { return DAT_E_SHORT; } if ($this->dcrc_read->i != $this->dcrc_computed) { return DAT_E_CRC; } return DAT_E_OK; } function get_framing_string() : string { return "8N1"; // FIXME } } class Detokeniser { private BeebBlock $bb; private static array $tokens = array( 0x80 => "AND", 0x81 => "DIV", 0x82 => "EOR", 0x83 => "MOD", 0x84 => "OR", 0x85 => "ERROR", 0x86 => "LINE", 0x87 => "OFF", 0x88 => "STEP", 0x89 => "SPC", 0x8a => "TAB", 0x8b => "ELSE", 0x8c => "THEN", // 0x8d 0x8d => "", 0x8e => "OPENIN", 0x8f => "PTR", 0x90 => "PAGE", 0x91 => "TIME", 0x92 => "LOMEM", 0x93 => "HIMEM", 0x94 => "ABS", 0x95 => "ACS", 0x96 => "ADVAL", 0x97 => "ASC", 0x98 => "ASN", 0x99 => "ATN", 0x9a => "BGET", 0x9b => "COS", 0x9c => "COUNT", 0x9d => "DEG", 0x9e => "ERL", 0x9f => "ERR", 0xa0 => "EVAL", 0xa1 => "EXP", 0xa2 => "EXT", 0xa3 => "FALSE", 0xa4 => "FN", 0xa5 => "GET", 0xa6 => "INKEY", 0xa7 => "INSTR(", 0xa8 => "INT", 0xa9 => "LEN", 0xaa => "LN", 0xab => "LOG", 0xac => "NOT", 0xad => "OPENUP", 0xae => "OPENOUT", 0xaf => "PI", 0xb0 => "POINT", 0xb1 => "POS", 0xb2 => "RAD", 0xb3 => "RND", 0xb4 => "SGN", 0xb5 => "SIN", 0xb6 => "SQR", 0xb7 => "TAN", 0xb8 => "TO", 0xb9 => "TRUE", 0xba => "USR", 0xbb => "VAL", 0xbc => "VPOS", 0xbd => "CHR$", 0xbe => "GET$", 0xbf => "INKEY$", 0xc0 => "LEFT$(", 0xc1 => "MID$(", 0xc2 => "RIGHT$(", 0xc3 => "STR$(", 0xc4 => "STRING$(", 0xc5 => "EOF", 0xc6 => "AUTO", 0xc7 => "DELETE", 0xc8 => "LOAD", 0xc9 => "LIST", 0xca => "NEW", 0xcb => "OLD", 0xcc => "RENUMBER", 0xcd => "SAVE", // 0xce 0xce => "", 0xcf => "PTR", 0xd0 => "PAGE", 0xd1 => "TIME", 0xd2 => "LOMEM", 0xd3 => "HIMEM", 0xd4 => "SOUND", 0xd5 => "BPUT", 0xd6 => "CALL", 0xd7 => "CHAIN", 0xd8 => "CLEAR", 0xd9 => "CLOSE", 0xda => "CLG", 0xdb => "CLS", 0xdc => "DATA", 0xdd => "DEF", 0xde => "DIM", 0xdf => "DRAW", 0xe0 => "END", 0xe1 => "ENDPROC", 0xe2 => "ENVELOPE", 0xe3 => "FOR", 0xe4 => "GOSUB", 0xe5 => "GOTO", 0xe6 => "GCOL", 0xe7 => "IF", 0xe8 => "INPUT", 0xe9 => "LET", 0xea => "LOCAL", 0xeb => "MODE", 0xec => "MOVE", 0xed => "NEXT", 0xee => "ON", 0xef => "VDU", 0xf0 => "PLOT", 0xf1 => "PRINT", 0xf2 => "PROC", 0xf3 => "READ", 0xf4 => "REM", 0xf5 => "REPEAT", 0xf6 => "REPORT", 0xf7 => "RESTORE", 0xf8 => "RETURN", 0xf9 => "RUN", 0xfa => "STOP", 0xfb => "COLOUR", 0xfc => "TRACE", 0xfd => "UNTIL", 0xfe => "WIDTH", 0xff => "OSCLI", ); function __construct (BeebBlock $bb) { $this->bb = $bb; } function go_baby_go (string &$s_out) : int { //size_t i; //u8_t state; //tk_err_t e; //s16_t tk; //char c[2]; //s32_t linenum; #define LINENUM_S_LEN 10 //char linenum_s[10]; //u8_t ln_decode[3]; $state = DETOK_STATE_ASCII; //$c[1] = '\0'; $c=""; $tk = -1; $linenum = -1; $e = X_E_OK; $s_out = ""; $ln_decode = array(0=>0,1=>0,2=>0); //for (i=0; i < inlen; i++) { for ($i=0; $i < count($this->bb->data->a); $i++) { //u8_t b; //b = in[i]; $b = $this->bb->data->a[$i]->value_int; switch ($state) { case DETOK_STATE_ASCII: case DETOK_STATE_EXPECT_TOKEN: // fixme???? // FIXME: rearrange into two blocks, one for definitely a token, // one for maybe a token if (0xd == $b) { // newline; expect line number $state = DETOK_STATE_EXPECT_LINENUM_1; //c[0] = '\n'; //e = append (out, alloc, fill, c, 1); $s_out .= "\n"; } else if ($b == 0x8d) { // line number needs decoding $state = DETOK_STATE_DECODE_LINENUM_1; } else if (Detokeniser::is_token($b)) { // assume token $e = Detokeniser::token($b, $s_out); // $s updated $tk = $b; // *** // TODO: tokens requiring special states, GOTO etc // *** switch ($tk) { default: break; } } else if (ctype_print(chr($b))) { //} || (b==0xa)) { //} else if (isascii(b)) { $s_out .= chr($b); //e = append (out, alloc, fill, c, 1); //} else if (b==0x1c) { } else { //TK_SNPRINTF(linenum_s, LINENUM_S_LEN, 5, "", b); //e = append(out, alloc, fill, linenum_s, strlen(linenum_s)); $s_out .= sprintf("", $b); } // ??? just ignore? /* } else { // not a token, not printable, not a newline ... fprintf(stderr, "desync at 0x%zx, byte %x\n", i, b); #define SNIP "\n\n" e = append(out, alloc, fill, SNIP, strlen(SNIP)); } */ //if (TK_E_OK != e) { return e; } break; case DETOK_STATE_EXPECT_LINENUM_1: $linenum = $b; $linenum <<= 8; $state = DETOK_STATE_EXPECT_LINENUM_2; break; case DETOK_STATE_EXPECT_LINENUM_2: $linenum |= $b; $state = DETOK_STATE_EXPECT_LINELEN; // ??? //TK_SNPRINTF(linenum_s, LINENUM_S_LEN, 6, "%u ", linenum & 0x7fff); //e = append(out, alloc, fill, linenum_s, strlen(linenum_s)); $s_out .= sprintf("%u ", $linenum & 0x7fff); break; case DETOK_STATE_EXPECT_LINELEN: // FIXME // just ignore it $state = DETOK_STATE_EXPECT_TOKEN; break; case DETOK_STATE_DECODE_LINENUM_1: $ln_decode[0] = $b; $state = DETOK_STATE_DECODE_LINENUM_2; break; case DETOK_STATE_DECODE_LINENUM_2: $ln_decode[1] = $b; $state = DETOK_STATE_DECODE_LINENUM_3; break; case DETOK_STATE_DECODE_LINENUM_3: $ln_decode[2] = $b; // we have all three bytes, go $linenum = Detokeniser::decode_linenum ($ln_decode[0], $ln_decode[1], $ln_decode[2]); //TK_SNPRINTF(linenum_s, LINENUM_S_LEN, 6, "%u", linenum & 0x7fff); //e = append(out, alloc, fill, linenum_s, strlen(linenum_s)); $s_out .= sprintf("%u", $linenum & 0x7fff); $state = DETOK_STATE_ASCII; $ln_decode = array(0=>0,1=>0,2=>0); break; default: //fprintf (stderr, "???\n"); print "???\n"; return TK_E_BUG; } // next input character if (X_E_OK != $e) { return $e; } } return $e; } // thanks, Matt Godbolt static function decode_linenum (int $a, int $b, int $c) : int { //int r0, r1, r10; $r10 = $a; $r0 = $r10 << 2; $r1 = $r0 & 0xc0; $r10 = $b; $r1 ^= $r10; $r10 = $c; $r0 = $r10 ^ ($r0 << 2); $r0 &= 0xff; $r0 = $r1 | ($r0<<8); return $r0; } static function is_token (int $b) : bool { //return ($b != 0x8d) && ($b != 0xce) && ($b >= 0x80); //return isset($tokens[$b]); return $b >= 0x80; } static function token (int $t, string &$s_inout) : int { //int $e; $e = X_E_OK; if ( ! Detokeniser::is_token($t) ) { //print(stderr, "bad token (0x%x)\n", t); //$s_inout .= sprintf("", $t); return TK_E_TOKEN; //return TK_E_OK; } // pre-append a space? // FIXME: this test should be a method on Detokeniser, needPrintPreSpace() or something if ($t == 0xb8) { // TO, ... //e = append (out, alloc, fill, " ", 1); $s_inout .= " "; } //if (TK_E_OK != e) { return e; } //e = append (out, alloc, fill, tokens[t - 0x80], 0xffff & strlen(tokens[t - 0x80])); $s_inout .= Detokeniser::$tokens[$t]; //if (TK_E_OK != e) { return e; } // post-append a space? // FIXME: this test should be a method on Detokeniser, needPrintPostSpace() or something if (($t != 0xdd) && ($t != 0xf2) && ($t != 0xa4)) { // don't add a space for DEF, others //e = append (out, alloc, fill, " ", 1); $s_inout .= " "; } return $e; } } define ("CLI_MODE_INSPECT", 1); define ("CLI_MODE_EXTRACT", 2); define ("CLI_OPT_BLK_ID", "+b"); // +arg define ("CLI_OPT_VRB", "+v"); define ("CLI_OPT_VRB_ERR", "+e"); define ("CLI_OPT_REPAIR_BITFLIP", "+r"); define ("CLI_OPT_FRAMING", "+f"); // +arg, not used define ("CLI_OPT_HELP", "+h"); define ("CLI_OPT_POLARITY_ERRORS", "+p"); define ("CLI_OPT_X_DIR", "+x"); // +arg define ("CLI_OPT_X_ALLOW_OVERWRITE", "+!"); define ("CLI_OPT_X_MKDIR", "+m"); define ("CLI_OPT_DETOK", "+k"); class Cli { var $mode; // extract or inspect var $block_id; var $verbose; var $verbose_errors; var $bitflip; var $extract_dir; var $framing; var $ip_filename; var $polarity_errors; var $allow_mkdir; var $allow_overwrite; var $basic_detok; function __construct() { $this->mode = CLI_MODE_INSPECT; $this->block_id = -1; $this->verbose = FALSE; $this->verbose_errors = FALSE; $this->bitflip = FALSE; $this->extract_dir = NULL; $this->framing = new BlockFraming; // not used $this->ip_filename = NULL; $this->polarity_errors = FALSE; $this->allow_mkdir = FALSE; $this->allow_overwrite = FALSE; $this->basic_detok = FALSE; } static function arg_is_opt ($s) { return ($s[0] == CLI_OPT_BLK_ID[0]); } static function dupe (string $v) { print "\nE: Illegal duplicate command-line option: $v\n"; } function parse (array $argv) : int { $c = count($argv); if ($c < 2) { return X_E_HELP; } if (($c == 2) && ($argv[1] == CLI_OPT_HELP)) { return X_E_HELP; } $have_block_id = FALSE; $have_verbose = FALSE; $have_verbose_errors = FALSE; $have_polarity_errors = FALSE; $have_repair = FALSE; $have_extract_dir = FALSE; $have_overwrite = FALSE; $have_mkdir = FALSE; $have_framing = FALSE; $have_detok = FALSE; for ($i=1, $final=0, $state=""; ($i < $c) && (NULL == $this->ip_filename); // finish once filename is found $i++) { $final = ($i == ($c-1)); $v = $argv[$i]; if ($final) { // final opt => CSW filename, but state is bad if ($state != "") { return X_E_CLI; } // CSW filename OK $this->ip_filename = $v; } else { // opts if ($state == "") { if ( ! Cli::arg_is_opt($v) ) { // no '+' found return X_E_CLI; } else if (CLI_OPT_VRB == $v) { if ($have_verbose) { Cli::dupe($v); return X_E_CLI; } $have_verbose = TRUE; $this->verbose = TRUE; } else if (CLI_OPT_VRB_ERR == $v) { if ($have_verbose_errors) { Cli::dupe($v); return X_E_CLI; } $have_verbose_errors = TRUE; $this->verbose_errors = TRUE; } else if (CLI_OPT_REPAIR_BITFLIP == $v) { if ($have_repair) { Cli::dupe($v); return X_E_CLI; } $have_repair = TRUE; $this->bitflip = TRUE; } else if (CLI_OPT_DETOK == $v) { if ($have_detok) { Cli::dupe($v); return X_E_CLI; } $have_detok = TRUE; $this->basic_detok = TRUE; } else if (CLI_OPT_HELP == $v) { return X_E_HELP; } else if (CLI_OPT_POLARITY_ERRORS == $v) { if ($have_polarity_errors) { Cli::dupe($v); return X_E_CLI; } $have_polarity_errors = TRUE; $this->polarity_errors = TRUE; } else if (CLI_OPT_X_MKDIR == $v) { if ($have_mkdir) { Cli::dupe($v); return X_E_CLI; } $have_mkdir = TRUE; $this->allow_mkdir = TRUE; } else if (CLI_OPT_X_ALLOW_OVERWRITE == $v) { if ($have_overwrite) { Cli::dupe($v); return X_E_CLI; } $have_overwrite = TRUE; $this->allow_overwrite = TRUE; } else { $state = $v; // argument expected, set state } } else { if ($state == CLI_OPT_BLK_ID) { if ($have_block_id) { Cli::dupe(CLI_OPT_BLK_ID." ".$v); return X_E_CLI; } $have_block_id = TRUE; if ( ! ctype_digit ($v) ) { return X_E_CLI; } $this->block_id = (int) $v; } else if ($state == CLI_OPT_X_DIR) { if ($have_extract_dir) { Cli::dupe(CLI_OPT_X_DIR." ".$v); return X_E_CLI; } $have_extract_dir = TRUE; $this->extract_dir = $v; $this->mode = CLI_MODE_EXTRACT; } else if ($state == CLI_OPT_FRAMING) { if ($have_framing) { Cli::dupe(CLI_OPT_FRAMING." ".$v); return X_E_CLI; } $have_framing = TRUE; $f = new BlockFraming; $e = $f->parse($v); // FIXME: not implemented, always fails if (X_E_OK != $e) { return $e; } } else { print "B: CLI illegal state: $state\n"; return X_E_BUG; } $state = ""; } } } if ( ($this->mode == CLI_MODE_EXTRACT) && ( $have_block_id || $have_verbose || $have_verbose_errors) ) { print "E: Cannot use ".CLI_MODE_EXTRACT. " with ".CLI_OPT_VRB.", ". CLI_OPT_VRB_ERR." or ". CLI_OPT_BLK_ID."\n"; return X_E_CLI; } if ($this->mode != CLI_MODE_EXTRACT) { if ($have_overwrite) { print "E: ".CLI_OPT_X_ALLOW_OVERWRITE." requires ".CLI_OPT_X_DIR."\n"; return X_E_CLI; } else if ($have_mkdir) { print "E: ".CLI_OPT_X_MKDIR." requires ".CLI_OPT_X_DIR."\n"; return X_E_CLI; } } return X_E_OK; } static function usage($argv0) : string { $argv0 = basename($argv0); $s=""; $s.="Usage:\n\n php -f $argv0 [options] \n\n"; $s.="where [options] may be:\n"; $s.=" ".CLI_OPT_BLK_ID. " inspect one particular block only\n"; $s.=" ".CLI_OPT_VRB. " print verbose block details\n"; $s.=" ".CLI_OPT_VRB_ERR. " print verbose error details\n"; $s.=" ".CLI_OPT_DETOK. " print block(s) as detokenised BASIC\n"; $s.=" ".CLI_OPT_POLARITY_ERRORS. " generate errors if adjacent blocks have different polarities \n (CSW only)\n"; $s.=" ".CLI_OPT_REPAIR_BITFLIP. " attempt bad-CRC repair by flipping bits\n"; $s.=" ".CLI_OPT_X_DIR. " extract blocks to target directory\n"; $s.=" ".CLI_OPT_X_ALLOW_OVERWRITE." with ".CLI_OPT_X_DIR.", allow overwriting existing files\n"; $s.=" ".CLI_OPT_X_MKDIR. " with ".CLI_OPT_X_DIR.", create directory if it doesn't exist\n"; //$s.=" ".CLI_OPT_FRAMING." select block framing (default is 8N1)\n"; return $s; } } $argv = $_SERVER['argv']; $e = csw_main ($argv); exit($e); function csw_main ($argv) { print "\nlsblocks.php, v".LSBLKS_VERSION."\n\n"; $cli = new Cli; $e = $cli->parse($argv); if ($e == X_E_HELP) { print Cli::usage($argv[0]); exit($e); } else if ($e == X_E_CLI) { print "For help, try:\n php -f $argv[0] +h\n\n"; exit($e); } else if ($e != X_E_OK) { exit($e); } if ( ! file_exists ($cli->ip_filename) ) { print "E: File not found: $cli->ip_filename\n"; exit(X_E_CSW_NOT_FOUND); } print "M: Loading $cli->ip_filename ... "; $ip = file_get_contents($cli->ip_filename); if (FALSE === $ip) { print "E: Could not load: $csw_fn\n"; exit(X_E_CSW_LOAD_ERROR); } $ip_len = strlen($ip); print "$ip_len bytes.\n"; $pulses = array(); $csw_polarity = FALSE; $csw_rate = 0; $bytes = array(); // try CSW $e = parse_csw ($ip, $csw_polarity, $csw_rate, $pulses); // csw_data, polarity, rate out // csw_data is an array of pulse lengths $is_tibet = (X_E_CSW_VERSION == $e) || (X_E_CSW_MAGIC == $e); if ($is_tibet) { print "CSW magic not found; assuming TIBET ...\n"; $csw_rate = 44100; $e = X_E_OK; } else if (X_E_OK != $e) { exit($e); } $gaps = new CswGaps($csw_rate); if ($is_tibet) { $e = parse_tibet ($ip, $gaps, $pulses); } if (X_E_OK != $e) { exit($e); } // this is called for both CSW and TIBET, and decodes the pulse train // (faked in the case of TIBET) $e = csw_decode ($pulses, $gaps, $is_tibet, $bytes); // bytes out: is array of DecodedByte if (X_E_OK != $e) { exit($e); } // $s=""; // foreach ($bytes as $a=>$b) { // $s .= $b->value_chr; // } // print my_hexdump_from_string($s); print "Decoded ".count($bytes)." bytes.\n"; $blocks = array(); $e = block_decode ($bytes, $blocks, $cli, $csw_rate, $is_tibet); //if ($e != X_E_OK) { // exit($e); //} $error_metric = 0; foreach ($blocks as $meh=>$b) { $error_metric += count($b->errors); } // print metrics for corpus analysis print "M: Metrics|Errors|$error_metric|Blocks|".count($blocks)."\n"; if ( $cli->bitflip ) { for ($i=0; $i < count($blocks); $i++) { if ( $blocks[$i]->data_ok() == DAT_E_CRC ) { print "Block #$i: bad data CRC: read ". sprintf("%04x", $blocks[$i]->dcrc_read->i). ", computed ". sprintf("%04x", $blocks[$i]->dcrc_computed). ".\n". " Try bitflip ...\n"; data_try_bitflips ($blocks[$i]); if ( $blocks[$i]->data_ok() == DAT_E_CRC ) { print "Bitflip failed.\n"; } } } } if ($cli->mode == CLI_MODE_EXTRACT) { $e = save_blocks ($blocks, $cli->extract_dir, $cli->allow_overwrite, $cli->allow_mkdir); } return $e; } function save_blocks (array $blocks, string $path, bool $overwrite, bool $mkdir) : int { if ( ! file_exists ($path) ) { if ( ! $mkdir ) { print "E: Block save directory not found: $path\n"; return X_E_SAVE_NO_DIR; } else { // create output directory if ( ! mkdir ($path) ) { print "E: Could not create block save directory: $path\n"; return X_E_SAVE_MKDIR; } print "M: Created block output directory $path\n"; } } else if ( ! is_dir ($path) ) { print "E: Block save directory was actually a file: $path\n"; return X_E_SAVE_DIR_IS_FILE; } print "M: Saving blocks to $path\n"; for ($i=0; $i < count($blocks); $i++) { $b = $blocks[$i]; $fn = $b->name->s; for ($j=0, $fn_clean="", $fn_hex=""; $j < strlen($fn); $j++) { if ( ! char_is_host_filename_legal ($fn[$j]) ) { $fn_clean.="-"; } else { $fn_clean.=$fn[$j]; } $fn_hex .= sprintf("%02x", ord($fn[$j])); } $fn_host = sprintf ("%04d_%d_%s_%s_%04x_%x_%x_%s%s%s_%s".SAVE_BIN_EXTENSION, $b->ix, $b->smpnum, $fn_clean, $fn_hex, $b->blknum->i, $b->load->i, $b->ex->i, ($b->flag->i & CSW_BLOCK_FLAGS_FINAL) ? "F" : "", ($b->flag->i & CSW_BLOCK_FLAGS_EMPTY) ? "E" : "", ($b->flag->i & CSW_BLOCK_FLAGS_LOCKED) ? "L" : "", $b->get_framing_string()); $fn_qualified = $path."/".$fn_host; if (file_exists($fn_qualified) && ! $overwrite) { print "E: Refusing to overwrite: $fn_qualified\n"; return X_E_SAVE_FILE_EXISTS; } $payload = ""; foreach ($b->data->a as $meh=>$v) { $payload .= $v->value_chr; } if (FALSE === file_put_contents ($fn_qualified, $payload)) { print "E: Failed to write file %s\n"; return X_E_SAVE_WRITE; } print "X: $fn_qualified\n"; } // next block return X_E_OK; } function block_decode (array $bytes, array &$blocks, Cli $cli, int $rate, bool $is_tibet) : int { global $state_names; $state = STATE_IDLE; $fn = ""; $blkn = 0; $lab = 0; $i = 0; $extract = ($cli->mode == CLI_MODE_EXTRACT); $prev_polarity = 0; $e = BLK_E_NONE; if ( ! $cli->verbose && ! $extract) { print BeebBlock::heading($is_tibet)."\n"; } for ($i=0; $i < count($bytes); $i++) { $byteval = $bytes[$i]; if (($byteval->error->e != BLK_E_NONE) && ($state != STATE_IDLE)) { // error token in byte stream, and we aren't in STATE_IDLE // make a note of where we are $byteval->error->state = $state; // attach error to block $blocks[$blkn]->errors[] = $byteval->error; // cswblks v3.2: new logic: // - bad stop bits are never fatal // - errors are not fatal to the block EXCEPT if we're in the data len field in the header // (we don't want a corrupted data length field // resulting in us trying to read an insane amount of data, and miss // subsequent blocks) if (($byteval->error->e != BLK_E_BAD_STOP_BIT) || ($state == STATE_DATA_LEN)) { block_finished ($blocks[$blkn], $cli); $blkn++; $state = STATE_IDLE; // abort block continue; } } $iv = $byteval->value_int; $cv = $byteval->value_chr; $sn = $byteval->smpnum; $leader = $byteval->leader_len_smps; $polarity = $byteval->polarity; $e = X_E_OK; $tln = isset($byteval->tibet_linenum) ? $byteval->tibet_linenum : -1; $tls = isset($byteval->tibet_line) ? $byteval->tibet_line : ""; if ($state == STATE_IDLE) { // reset everything here $load_pos = 0; $ex_pos = 0; $blknum_pos = 0; $blklen_pos = 0; $nextaddr_pos = 0; $hcrc_pos = 0; $dcrc_pos = 0; $start_of_hdr = 0; $data = array(); $load_tmp = array(); $ex_tmp = array(); $blknum_tmp = array(); $blklen_tmp = array(); $nextaddr_tmp = array(); $dcrc_tmp = array(); $hcrc_tmp = array(); $blocks[$blkn] = new BeebBlock($blkn); $blocks[$blkn]->tibet_linenum = $byteval->tibet_linenum; $blocks[$blkn]->tibet_line = $byteval->tibet_line; $blocks[$blkn]->tibet_linenum = $byteval->tibet_linenum; // awaiting sync if ($iv != 0x2a) { continue; } if ($leader < (MIN_VALID_LEADER_LEN_S * $rate)) { // we didn't get enough prior leader // (probably a squawk) // ignore it continue; } if ( ( ! $is_tibet ) && ($blkn > 0) && ($polarity != $prev_polarity) && $cli->polarity_errors) { // unexpected polarity switch detected // synthesise an error for this, and attach it to the block $blkerr = ByteError::get_csw (BLK_E_POLARITY, $sn); $blkerr->state = $state; $blocks[$blkn]->errors[] = $blkerr; } $prev_polarity = $polarity; $blocks[$blkn]->smpnum = $sn; // store the block start smpnum $state = STATE_FILENAME; $start_of_hdr = $i + 1; } else if ($state == STATE_FILENAME) { // file name if (strlen($blocks[$blkn]->name->s) == 0) { if ( ! $is_tibet ) { // start of filename, store its smpnum $blocks[$blkn]->name->smpnum = $sn; } else { $blocks[$blkn]->name->tibet_linenum = $byteval->tibet_linenum; $blocks[$blkn]->name->tibet_line = $byteval->tibet_line; } } if (0 == $iv) { // NULL terminator // end of file name $state = STATE_LOAD; } else { $blocks[$blkn]->name->s .= $cv; } if (strlen($blocks[$blkn]->name->s) > 10) { //print "E: [".$blocks[$blkn]->name->smpnum." -> ".$sn."]: File name unterminated.\n"; // unterminated filename // can't decide what to do about this // we could try to continue decoding the block // but for now we'll just invalidate the block // and try to get the next one if ($is_tibet) { $blkerr = ByteError::get_tibet(BLK_E_FILENAME_LONG, $byteval->tibet_linenum, $byteval->tibet_line); } else { $blkerr = ByteError::get_csw(BLK_E_FILENAME_LONG, $sn); } $blkerr->state = $state; $blocks[$blkn]->errors[] = $blkerr; $e = X_E_CSW_FILENAME_LONG; // terminate block } } else if ($state == STATE_LOAD) { // load address $e = block_add_le4_byte ($iv, $load_pos, $load_tmp, $blocks[$blkn]->load, $state, // advanced $sn, $tln, $tls); } else if ($state == STATE_EXECUTE) { // ex address $e = block_add_le4_byte ($iv, $ex_pos, $ex_tmp, $blocks[$blkn]->ex, $state, // advanced $sn, $tln, $tls); } else if ($state == STATE_BLOCKNUM) { // blocknum $e = block_add_le2_byte ($iv, $blknum_pos, $blknum_tmp, $blocks[$blkn]->blknum, $state, // advanced $sn, $tln, $tls); } else if ($state == STATE_DATA_LEN) { // block len $e = block_add_le2_byte ($iv, $blklen_pos, $blklen_tmp, $blocks[$blkn]->blklen, $state, // advanced $sn, $tln, $tls); if (/*(BLK_E_NONE == $e) &&*/ ($blocks[$blkn]->blklen->i > 256)) { // looks like Ultron (Viper) at least does this? // high byte is set in the block len, we'll throw // a warning and then just zero it out if ($is_tibet) { $blkerr = ByteError::get_tibet(BLK_E_BLOCKLEN_LARGE, $byteval->tibet_linenum, $byteval->tibet_line); } else { $blkerr = ByteError::get_csw(BLK_E_BLOCKLEN_LARGE, $sn); } $blkerr->state = ($state - 1); // need prior phase, not current phase $blocks[$blkn]->errors[] = $blkerr; $blocks[$blkn]->blklen->i &= 0xff; } } else if ($state == STATE_NEXTFILE) { // block flag $blocks[$blkn]->flag->i = $iv; if ($is_tibet) { $blocks[$blkn]->flag->tibet_linenum = $byteval->tibet_linenum; $blocks[$blkn]->flag->tibet_line = $byteval->tibet_line; } else { $blocks[$blkn]->flag->smpnum = $sn; } $state = STATE_FLAGS; } else if ($state == STATE_FLAGS) { // next file address $e = block_add_le4_byte ($iv, $nextaddr_pos, $nextaddr_tmp, $blocks[$blkn]->nextaddr, $state, // advanced $sn, $tln, $tls); if ($nextaddr_pos == 4) { // compute header CRC now $hdr = array_slice ($bytes, $start_of_hdr, $i + 1 - $start_of_hdr); $blocks[$blkn]->hcrc_computed = acorn_crc($hdr); } } else if ($state == STATE_HCRC) { // header CRC $hcrc = new BlockField; // temp $e = block_add_le2_byte ($iv, $hcrc_pos, $hcrc_tmp, $hcrc, $state, // advanced $sn, $tln, $tls); if ($hcrc_pos == 1) { if ($is_tibet) { $blocks[$blkn]->hcrc_read->tibet_linenum = $byteval->tibet_linenum; $blocks[$blkn]->hcrc_read->tibet_line = $byteval->tibet_line; } else { $blocks[$blkn]->hcrc_read->smpnum = $sn; } } else { // CRC is MSB first, so $blocks[$blkn]->hcrc_read->i = (($hcrc->i << 8) & 0xff00) | (($hcrc->i >> 8) & 0xff); // compare CRCs if ($blocks[$blkn]->hcrc_computed != $blocks[$blkn]->hcrc_read->i) { // bad header CRC detected. This is an error, but not one // in the bytestream -- we'll just attach it to the block $blocks[$blkn]->errors[] = ByteError::get2(BLK_E_HEADER_CRC); } $len = $blocks[$blkn]->blklen->i; if ($len == 0) { // no data, no data CRC => finished, so reset $state = STATE_IDLE; //print $blocks[$blkn]->to_string()."\n"; block_finished($blocks[$blkn], $cli); $blkn++; } } } else if ($state == STATE_DATA) { // if the header is valid, check the polarity // data if (count($blocks[$blkn]->data->a) == 0) { if ($is_tibet) { //print "blocks[$blkn]->data->tibet_linenum = $byteval->tibet_linenum\n"; $blocks[$blkn]->data->tibet_linenum = $byteval->tibet_linenum; $blocks[$blkn]->data->tibet_line = $byteval->tibet_line; } else { $blocks[$blkn]->data->smpnum = $sn; } } $blocks[$blkn]->data->a[] = $byteval; if (count($blocks[$blkn]->data->a) == $blocks[$blkn]->blklen->i) { // got all the data $blocks[$blkn]->dcrc_computed = acorn_crc($blocks[$blkn]->data->a); $state = STATE_DCRC; } } else if ($state == STATE_DCRC) { // data CRC $dcrc = new BlockField; // temp $e = block_add_le2_byte ($iv, $dcrc_pos, $dcrc_tmp, $dcrc, $state, // advanced $sn, $tln, $tls); if ($dcrc_pos == 1) { if ($is_tibet) { $blocks[$blkn]->dcrc_read->tibet_linenum = $byteval->tibet_linenum; $blocks[$blkn]->dcrc_read->tibet_line = $byteval->tibet_line; } else { $blocks[$blkn]->dcrc_read->smpnum = $sn; } } else { // CRC is MSB first, so byteswap $blocks[$blkn]->dcrc_read->i = (($dcrc->i << 8)&0xff00) | (($dcrc->i >> 8)&0xff); // compare CRCs if ($blocks[$blkn]->dcrc_computed != $blocks[$blkn]->dcrc_read->i) { // bad data CRC detected. This is another error, but not one // in the bytestream -- we'll just attach it to the block $blocks[$blkn]->errors[] = ByteError::get2(BLK_E_DATA_CRC); } // block finished, reset $state = STATE_IDLE; block_finished($blocks[$blkn], $cli); //print $blocks[$blkn]->to_string()."\n"; $blkn++; } } else { print "B: impossible state ".$state."\n"; $e = X_E_BUG; } if ($e != X_E_OK) { $state = STATE_IDLE; $e = X_E_OK; // error, but keep the incomplete block anyway block_finished($blocks[$blkn], $cli); $blkn++; } } // next byte if ($state != STATE_IDLE) { print "E: State machine finished in state $state. Possible source truncation.\n"; } return $e; // should be 0 unless something fatal happens } function block_finished (BeebBlock $b, Cli $c) { // don't print anything for extract mode $inspect = ($c->mode == CLI_MODE_INSPECT); if ( $inspect && (( $c->block_id == -1 ) || ($c->block_id == $b->ix) ) ) { print $b->to_string($c->verbose==1, $c->verbose_errors==1)."\n"; } if ( $inspect && $c->basic_detok && (( $c->block_id == -1 ) || ($c->block_id == $b->ix) ) ) { $s=""; $e = $b->detokenise($s); if (X_E_OK != $e) { print "W: detokenise failed, code $e\n"; } else { print "\n----- BEGIN BASIC -----\n"; print $s."\n"; print "------ END BASIC ------\n\n"; } } } // this is horrible function block_add_le4_byte (int $b, int &$le4_pos, array &$tmp_array, BlockField &$target, int &$state, int $smpnum, int $tibet_linenum, string $tibet_line) : int { if ($le4_pos == 0) { $tmp_array = array(); // store smpnum $target->smpnum = $smpnum; $target->tibet_linenum = $tibet_linenum; $target->tibet_line = $tibet_line; } $tmp_array[] = $b; if ($le4_pos == 3) { $e = parse_le4_i ($tmp_array, $target->i); if (X_E_OK != $e) { return $e; } $state++; // advance state if no error } $le4_pos++; return X_E_OK; } // this is also horrible function block_add_le2_byte (int $b, int &$le2_pos, array &$tmp_array, BlockField &$target, int &$state, int $smpnum, int $tibet_linenum, string $tibet_line) : int { if ($le2_pos == 0) { $tmp_array = array(); // store smpnum $target->smpnum = $smpnum; $target->tibet_linenum = $tibet_linenum; $target->tibet_line = $tibet_line; } $tmp_array[] = $b; if ($le2_pos == 1) { $e = parse_le2_i ($tmp_array, $target->i); if (X_E_OK != $e) { return $e; } $state++; // advance state if no error } $le2_pos++; return X_E_OK; } function data_try_bitflips (BeebBlock &$block) { $data = $block->data->a; // make copy $len = count($data); for ($i=0; $i < $len; $i++) { // bytes $mask = 1; $byte_i = $data[$i]->value_int; for ($j=0; $j < 8; $j++) { // bits $b = $byte_i; // make copy $b = $b ^ $mask; // flip bit $data[$i]->value_int = $b; // replace modified byte in data $crc = acorn_crc($data); // CRC it if ($block->dcrc_read->i == $crc) { //print "[".($data[$i]->smpnum)."] "; if ($data[$i]->tibet_linenum != -1) { printf ("{ L%5d } ", 1+$data[$i]->tibet_linenum); } else { printf ("[%08d] ", $data[$i]->smpnum); } print "BITFLIP \"". $block->printable_name()."\", block ID ". $block->ix.": flip byte $i (".sprintf("&%x -> &%x",$byte_i,$b)."), bit $j (".($byte_i & $mask)."->".($b & $mask).")\n"; $data[$i]->value_chr = chr($b); $block->data->a = $data; // replace data $block->dcrc_computed = $crc; return; } $mask <<= 1; } $data[$i]->value_int = $byte_i; // restore byte and continue } } function decode_pulse_seq (array $four_pulses, CswGaps $gaps, int &$value) : int { $v = array(); for ($i=0; $i < 4; $i++) { $plen = $four_pulses[$i]->len_smps; if ($plen > MIN_SILENCE_SMPS) { return BLK_E_SILENCE; } else if ( ($plen < $gaps->min_valid_2400) || ($plen > $gaps->max_valid_1200) ) { // pulse length out of range return BLK_E_PULSE_LEN; } else if ($plen < $gaps->thresh) { // shorter than thresh // 2400 $v[$i] = 1; } else { // 1200 // longer than thresh $v[$i] = 0; } } // zero-bit: two long pulses required if ($v[0] == 0) { if ($v[0] != $v[1]) { return BLK_E_PULSE_SEQUENCE; } $value = 0; return BLK_E_NONE; // ok } // one-bit: four short pulses required for ($i=0; $i < 4; $i++) { if ($v[$i] != 1) { return BLK_E_PULSE_SEQUENCE; } $value = 1; } return BLK_E_NONE; // ok } // expects 8N1 // bytevals is array of DecodedByte // pulses is CswPulse[] function csw_decode (array $pulses, CswGaps $gaps, bool $is_tibet, array &$bytevals) : int { $i = 0; // sample count $bitpos = 0; $b = 0; $frame_start_smps = 0; $p = 0; $num_2400_run_cycs = 0; //$polarity_at_start_of_frame = 0; //$polarity_at_start_of_prev_frame = 0; $polarity = 0; for ($p=0; $p < (count($pulses) - 3); $i += $pulses[$p]->len_smps, $p++) { // increase total sample count // examine (up to) four consecutive pulses $v = 0; $e = decode_pulse_seq (array_slice($pulses, $p, 4), $gaps, $v); if ($e == BLK_E_SILENCE) { // reset leader counters, but do not insert an error token $pending_leader_run_len_smps = 0; $num_2400_run_cycs = 0; continue; } else if ($e != BLK_E_NONE) { // bad pulse sequence $bitpos = 0; $error_byteval = new DecodedByte; // this condition will interrupt a block // if it occurs during one, so we need // to mark it in the byte stream $error_byteval->error = new ByteError; $error_byteval->error->e = $e; $error_byteval->error->smpnum = $i; //smps; $error_byteval->error->pulses = array_slice($pulses, $p, 4); $bytevals[] = $error_byteval; continue; // next single pulse } if ($v == 0) { // update sample count for skipped pulses $i += $pulses[$p]->len_smps; // zero bit encountered if (0 == $bitpos) { // preserve leader length if we're outside of a block $pending_leader_run_len_smps = $num_2400_run_cycs; $polarity = ($p & 1); } $num_2400_run_cycs = 0; $p++; // valid long pair, skip to p+2 } else { // update sample count for skipped pulses $i += $pulses[$p]->len_smps; $i += $pulses[$p+1]->len_smps; $i += $pulses[$p+2]->len_smps; $num_2400_run_cycs += $pulses[$p]->len_smps + $pulses[$p+1]->len_smps + $pulses[$p+2]->len_smps + $pulses[$p+3]->len_smps; $p+=3; // valid short quad, skip to p+4 } if (0 == $bitpos) { // start of frame // expect start bit $b = 0; $frame_start_smps = $i; // record smpnum of start bit if ($v == 1) { //print "E: [$i]: invalid start bit\n"; continue; // no start bit } $bitpos = 1; // start bit found, carry on //$polarity_at_start_of_frame = $polarity; } else if (($bitpos >= 1) && ($bitpos <= 8)) { $b = ($b >> 1) & 0x7f; $b |= ($v ? 0x80 : 0x0); $bitpos++; } else { // bitpos == 9 // expect stop bit $byteval = new DecodedByte; //$byteval->bad_stop_bit_smpnum = -1; // copy the leader length from the start of the block $byteval->leader_len_smps = $pending_leader_run_len_smps; $byteval->tibet_linenum = $pulses[$p]->tibet_linenum; if ( $is_tibet && ! isset($pulses[$p]->tibet_linenum) ) { print "B: pulses[$p]->tibet_linenum is unset\n"; return TBT_E_BUG; }// else { //print_r($pulses[$p]); //} $byteval->tibet_line = $pulses[$p]->tibet_line; $pending_leader_run_len_smps = 0; // check stop bit is OK if ($v != 1) { // attach error to byte (no independent sentinel error for this now, v3.2) if ($is_tibet) { $byteval->error = ByteError::get_tibet(BLK_E_BAD_STOP_BIT, $byteval->tibet_linenum, $byteval->tibet_line); } else { $byteval->error = ByteError::get_csw(BLK_E_BAD_STOP_BIT, $frame_start_smps); } } // v3.2: modified this so that stop bit errors are no longer fatal; // the byte is marked with the error, but the byte is still resolved //} else { $byteval->value_int = $b; $byteval->value_chr = chr($b); $byteval->smpnum = $frame_start_smps; $byteval->polarity = $polarity; //} // append either a byte or an error token $bytevals[] = $byteval; $bitpos = 0; // start again } } // next pulse pair return X_E_OK; } function parse_csw (string $csw, bool &$polarity, int &$rate, array &$pulses_out) : int { $rate = 0; $h_len = 0; $zip = false; $polarity = false; $num_pulses = 0; // rate, h_len, zip, polarity, num_pulses out $e = csw_parse_header ($csw, $rate, $h_len, $zip, $polarity, $num_pulses); if ($e != X_E_OK) { return $e; } print "\nM: Body starts at 0x".sprintf("%x", $h_len)."\n"; $e = parse_body (substr($csw, $h_len), $zip, $pulses_out); // data is an array of pulse lengths if ($e != X_E_OK) { return $e; } if (count($pulses_out) != $num_pulses) { print "W: Pulse count mismatch: $num_pulses in header, ".count($pulses_out)." in body.\n"; } else { print "M: Counted ".count($pulses_out)." pulses (OK).\n"; } return $e; } function csw_parse_header (string $csw, int &$rate, int &$h_len, bool &$zipped, bool &$polarity, int &$num_pulses) : int { $magic_s = substr($csw, 0, 23); // v3.2: $version_s = substr($csw, 23, 2); $rate_s = substr($csw, 25, 4); $num_pulses_s = substr($csw, 29, 4); $cmp_type_s = substr($csw, 33, 1); $flags_s = substr($csw, 34, 1); $hdr_extlen_s = substr($csw, 35, 1); $appdesc_s = substr($csw, 36, 16); $hdr_extlen = ord($hdr_extlen_s); $hdr_ext = substr($csw, 52, $hdr_extlen); $rate=0; $num_pulses=0; // v3.2: now parse version properly $vmaj = 0; $vmin = 0; if ($version_s === "\x02\x00") { $vmaj = 2; $vmin = 0; } else if ($version_s === "\x02\x01") { $vmaj = 2; $vmin = 1; } else { print "E: Unknown CSW file version: $vmaj.$vmin\n"; return X_E_CSW_VERSION; //18; } if (X_E_OK != ($e = parse_le4_s ($rate_s, $rate))) { return $e; } if (X_E_OK != ($e = parse_le4_s ($num_pulses_s, $num_pulses))) { return $e; } $cmp_type = ord($cmp_type_s); $flags = ord($flags_s); if (($cmp_type != 1) && ($cmp_type != 2)) { print "E: Bad compression type: 0x".sprintf("%02x", $cmp_type)."\n"; return X_E_CSW_COMPRESSION_TYPE; } // v3.2: flag legality mask changed from 0xfe, made nonzero bits 0-2 legal if ($vmin == 1) { $flags_legal_mask = 0xf8; printf("W: CSW 2.1: use of flags bits 1 & 2 is unknown; flags are 0x%x\n", $flags); } else if ($vmin == 0) { $flags_legal_mask = 0xfe; } if ($flags & $flags_legal_mask) { //0xfe) { print "E: Bad flags: 0x".sprintf("%02x", $flags)."\n"; return 6; } $polarity = (1 == ($flags & 1)); $hdr_extlen = ord($hdr_extlen_s); // v3.2: if ($magic_s !== "Compressed Square Wave\x1a") { //print "E: Bad juju\n"; return X_E_CSW_MAGIC; } if ($hdr_extlen != 0) { print "W: Unknown header extension data: ".my_hexdump_simple($hdr_ext)."\n"; } print "\nCSW Version: $vmaj.$vmin\n"; print "App: \"$appdesc_s\"\n"; print "Rate: $rate\n"; print "Pulses: $num_pulses\n"; print "Polarity: ".($polarity?"starts high":"starts low")."\n"; $zipped = ($cmp_type == 2); print "Zipped: ". ($zipped ? "yes" : "no")."\n"; print "Hdr. ext. len.: $hdr_extlen\n"; $h_len = 52 + $hdr_extlen; return X_E_OK; } function my_hexdump_simple(string $s) : string { $len = strlen($s); $o = ""; for ($i=0; $i < $len; $i++) { $o.=sprintf("%02x ", ord($s[$i])); } return $o; } function parse_body (string $body, bool $zip, array &$pulses_out) : int { $e = X_E_OK; if ($zip) { $e = body_unzip($body); // body modified if (X_E_OK != $e) { return $e; } } $len = strlen($body); $smpnum=0; for ($i=0; $i < $len; $i++) { $b = ord($body[$i]); if ($b == 0) { if (($i+4) >= $len) { print "E: long pulse overflows buffer\n"; return X_E_CSW_DECODE_BODY; } $e = parse_le4_s (substr($body, $i+1, 4), $b); //print "M: Long pulse, body offset $i (0x".sprintf("%x",$i)."): $b\n"; if (X_E_OK != $e) { return $e; } $i+=4; } $p = new CswPulse; $p->len_smps = $b; $p->smpnum = $smpnum; $smpnum += $b; $pulses_out[] = $p; } return $e; } function body_unzip (string &$b) : int { $in = $b; $b = zlib_decode($in); if (false === $b) { print "E: zlib decode failed.\n"; return X_E_CSW_UNZIP; } print "M: Unzipped body from ".strlen($in)." to ".strlen($b)." bytes.\n"; return X_E_OK; } function parse_le4_s (string $s, int &$i) : int { $len = strlen($s); if ($len != 4) { print "E: parse_le4_s: string has len $len, should be 4\n"; return X_E_BUG; } $b = array(); for ($j=0; $j < 4; $j++) { $b[$j] = ord($s[$j]); } parse_le4_i ($b, $i); return X_E_OK; } function parse_le4_i (array $b, int &$i) { $i=0; $i |= $b[0]; $i |= ($b[1] << 8) & 0xff00; $i |= ($b[2] << 16) & 0xff0000; $i |= ($b[3] << 24) & 0xff000000; } function parse_le2_i (array $b, int &$i) { $i=0; $i |= $b[0]; $i |= ($b[1] << 8) & 0xff00; } function acorn_crc (array $ip) : int { $c = 0; foreach ($ip as $k=>$bv) { $b = $bv->value_int; $h = ($c>>8) & 0xff; $h = $b ^ $h; $c = ($c & 0xff) | (($h << 8) & 0xff00); for ($i=0; $i < 8; $i++) { $t=0; if ($c & 0x8000) { $c = $c ^ 0x810; $t = 1; } $c = 0xffff & ($t | (($c<<1) & 0xfffe)); } } return $c; } function my_hexdump_from_string (string $stg) : string { $s=""; $start_of_line = 1; if (!isset($stg) || (strlen($stg)==0)) { return ""; } $l=strlen($stg); if (defined("HEXDUMP_MAX") && $l>HEXDUMP_MAX) { $l=HEXDUMP_MAX; } $sbuf=""; for ($i=0;$i<$l;$i++) { if ($start_of_line) { //$s .= sprintf("[%8d] ", $mem[$i]->smpnum); $start_of_line = 0; } if (!($i%16)) { //if ($include_offset) { $s.=sprintf ("%5x ", $i); //} else { // $s.=" "; //} } $s.=sprintf ("%02x ", $z=ord($stg[$i])); if (!ctype_print($w=($stg[$i]))||$z>127||$w==="\r"||$w==="\n") { $sbuf.="."; } else { $sbuf.=$w; } if ($i%16 == 15) { // append text bit yet? $s .= " ".$sbuf."\n"; $sbuf=""; $start_of_line = 1; } } // ending // append any remaining text bit if ($i%16!=0) { for ($i=(16-$i%16);$i;$i--) { // pad up to start of text bit $s.= " "; } $s.= " ".$sbuf."\n"; } return $s; } function my_hexdump (array $mem, bool $is_tibet, bool $include_offset) { $s=""; $start_of_line = 1; if (!isset($mem) || (count($mem)==0)) { return; } $l=count($mem); if (defined("HEXDUMP_MAX") && $l>HEXDUMP_MAX) { $l=HEXDUMP_MAX; } $sbuf=""; for ($i=0;$i<$l;$i++) { if ($start_of_line) { if ( ! $is_tibet ) { $s .= sprintf("[%8d] ", $mem[$i]->smpnum); } else { $s .= sprintf("{ L%5d } ", $mem[$i]->tibet_linenum); } $start_of_line = 0; } if (!($i%16)) { if ($include_offset) { $s.=sprintf ("%02x ", $i); } else { $s.=" "; } } $s.=sprintf ("%02x ", $z=ord($mem[$i]->value_chr)); if (!ctype_print($w=($mem[$i]->value_chr))||$z>127||$w==="\r"||$w==="\n") { $sbuf.="."; } else { $sbuf.=$w; } if ($i%16 == 15) { // append text bit yet? $s .= " ".$sbuf."\n"; $sbuf=""; $start_of_line = 1; } } // ending // append any remaining text bit if ($i%16!=0) { for ($i=(16-$i%16);$i;$i--) { // pad up to start of text bit $s.= " "; } $s.= " ".$sbuf."\n"; } return $s; } function printable($c) : bool { if (strlen($c)!=1) { return FALSE; } return ctype_alnum($c) || ctype_punct($c) || ($c==" "); } function char_is_host_filename_legal ($c) : bool { // from Cornfield again // (amend as appropriate) return ctype_alnum ($c) || ('_'==$c) || ('-'==$c) || (':'==$c) || (','==$c) || ('^'==$c) || ('('==$c) || (')'==$c) || (' '==$c); } // ***** BEGIN TIBET STUFF FROM TIBETUEF.PHP ***** function parse_tibet (string $ip, CswGaps $cswgaps, array &$pulses_out) : int { $pulses_out = array(); // try gzdecode $ip_unz = @gzdecode($ip); if (FALSE === $ip_unz) { print "Input is not gzipped.\n"; } else { print "Decompressed TIBET: ".strlen($ip)." -> ".strlen($ip_unz)." bytes.\n"; $ip = $ip_unz; } $tbt = new ParsedTibet; $e = tibet_process ($ip, FALSE, $tbt); // tbt populated if (X_E_OK != $e) { return $e; } //foreach($tbt->spans as $a=>$b) { printf("[%d]: %s\n", $a, gettype($b)); } //die(); // copy results from tibetuef's ParsedTibet to cswblks's array of DecodedByte foreach ($tbt->spans as $i=>$span) { if (gettype($span) != "object") { print "B: parse_tibet: Bad span type: \"".gettype($span)."\"\n"; return TBT_E_BUG; } $t = get_class($span); // (TimeHint), TibetSilence, TibetLeader, (DataFraming), (BaudRate), TibetData, (TibetCycle), DummyByte if ($t == "TibetData") { //$num_1200ths = $t->secs * CSW_FREQ_1; for ($n=0; $n < count($span->cycles); $n++) { $c = $span->cycles[$n]; // TibetCycle (one 2400th) $p = new CswPulse; if ( ! isset ($c->tibet_linenum) ) { print "B: span $i cycle $n has no tibet_linenum\n"; return TBT_E_BUG; } $p->tibet_linenum = $c->tibet_linenum; $p->tibet_line = $c->tibet_line; if (1 == $c->value) { $p->len_smps = (int) ($cswgaps->ideal_2400); $pulses_out[] = $p; $pulses_out[] = $p; } else { $p->len_smps = (int) ($cswgaps->ideal_2400 * 2); $pulses_out[] = $p; } } } // the block decoder currently fails unless a decent amount of // leader exists between blocks, so if ($t == "TibetLeader") { $p = new CswPulse; $p->tibet_linenum = 0; //$span->tibet_linenum; $p->tibet_line = ""; //$span->tibet_line; $p->len_smps = (int) ($cswgaps->ideal_2400); for ($z=0; $z < 1000; $z++) { $pulses_out[] = $p; } } } return X_E_OK; } function tibet_process_line (int $ln, string $line, int &$state, int &$span_ix, bool $insert_timestamps, ParsedTibet &$tbt) : int { // FIXME: doesn't quite meet TIBET specifications // more checking needed ... // eliminate comments $line_tmp = explode("#", $line); $line = $line_tmp[0]; // split by space $words_tmp = explode(" ", $line); $words = array(); // remove any blank words foreach ($words_tmp as $tmp=>$w) { $w = trim($w); if (strlen($w) > 0) { $words[] = $w; } } $wc = count($words); // skip empty lines if ($wc == 0) { return X_E_OK; } $e = X_E_OK; $w0 = $words[0]; // the default state at the start of a parse is TBT_STATE_VERSION ... if (TBT_STATE_VERSION == $state) { // version line must be the first non-comment, non-blank // line in the file. // any subsequent version lines will simply be ignored. // (this is deliberate and makes concatenating files easy) $e = tibet_parse_version ($words, $ln, $tbt->version, $line); $state = TBT_STATE_IDLE; } else if (TBT_STATE_IDLE == $state) { // this is a whitelist; we could ignore unknown keywords // instead, but we'll leave it like this for now if ($w0 == "tibet") { // duplicate version line; just check it for validity $dummy = ""; $e = tibet_parse_version ($words, $ln, $dummy, $line); if ($dummy != $tbt->version) { print "E: line $ln, span $span_ix: Mismatched duplicate version: $line\n"; return TBT_E_PARSE_DUP_VERSION; } // TIBET 0.4: reset framing and baud hints for file concatenation $df = new DataFraming; // constructor defaults to 8N1 $df->linenum = $ln; $df->span_ix = $span_ix; $tbt->spans[] = $df; // token rather than span $br = new BaudRate; // constructor defaults to 1200 $br->linenum = $ln; $br->span_ix = $span_ix; $tbt->spans[] = $br; // token rather than span } else if ($w0 == "silence") { if ($wc != 2) { print "E: line $ln, span $span_ix: Bad silence line: $line\n"; return TBT_E_PARSE_SILENCE; } $silence = new TibetSilence; $silence->linenum = $ln; $f = 0.0; $e = tibet_parse_float ($words[1], $f); // f populated if (X_E_OK != $e) { return $e; } $silence->secs = $f; //(float) $words[1]; $silence->span_ix = $span_ix; $span_ix++; if (($silence->secs <= 0.0) || ($silence->secs > 1000000.0)) { print "E: line $ln, span $span_ix: Illegal silence length: $words[1]\n"; return TBT_E_BAD_SILENCE; } $tbt->spans[] = $silence; } else if ($w0 == "leader") { if ($wc != 2) { print "E: line $ln: Bad leader line: $line\n"; return TBT_E_PARSE_LEADER; } $leader = new TibetLeader; $leader->linenum = $ln; $num_cycs = 0; $e = tibet_parse_int($words[1], $num_cycs); // $num_cycs populated if (X_E_OK != $e) { print "E: line $ln, span $span_ix: Non-integer leader cycles count: $words[1]\n"; return $e; } $leader->cycles = $num_cycs; //(int) $words[1]; $leader->span_ix = $span_ix; $span_ix++; if (($leader->cycles < 1) || ($leader->cycles > 30000000)) { print "E: line $ln, span $span_ix: Illegal leader length: $words[1]\n"; return TBT_E_BAD_LEADER; } $tbt->spans[] = $leader; } else if (($w0 == "squawk") || ($w0 == "data")) { if ($wc != 1) { print "E: line $ln, span $span_ix: Illegal $w0 line: $line\n"; return TBT_E_PARSE_DATA; } $data = new TibetData; $data->linenum = $ln; $data->span_ix = $span_ix; $data->squawk = ($w0 == "squawk"); $span_ix++; $state = TBT_STATE_CYCLES; $tbt->spans[] = $data; } else if ($w0 == "/phase") { // don't care; it's partially a function of playback, // so I disagree that it should be regarded } else if ($w0 == "/speed") { // don't care; again, it's a function of playback, // not of the source } else if ($w0 == "/time") { if ($insert_timestamps) { $timehint = new TimeHint; $timestamp = 0.0; $e = tibet_parse_float($words[1], $timestamp); // timestamp populated if (X_E_OK != $e) { print "E: line $ln, span $span_ix: Illegal /time hint: $line\n"; return $e; } $timehint->timestamp = $timestamp; //(float) $words[1]; $timehint->linenum = $ln; $timehint->span_ix = $span_ix; $tbt->spans[] = $timehint; // token rather than span } } else if ($w0 == "/framing") { // quadbike doesn't export this, as it can't detect framing, // but it could be added by manually editing a TIBET file. // UEF chunk 104 needs to know about non-standard framings, and // if they e.g. change in the middle of a block, we stand no // chance of detecting them automatically, so $df = new DataFraming; $df->linenum = $ln; $df->span_ix = $span_ix; $e = tibet_parse_framing ($words[1], $df); // df populated if (X_E_OK != $e) { return $e; } $tbt->spans[] = $df; // token rather than span } else if ($w0 == "/baud") { // again, not exported by QB $br = new BaudRate; $br->linenum = $ln; $br->span_ix = $span_ix; $e = tibet_parse_baudrate ($words[1], $br); // br populated if (X_E_OK != $e) { return $e; } $tbt->spans[] = $br; // token rather than span } else { print "E: line $ln, span $span_ix: Unrecognised: $line\n"; return TBT_E_PARSE_BAD_LINE; } } else if (TBT_STATE_CYCLES == $state) { if ($w0 == "end") { $state = TBT_STATE_IDLE; } else { $len = strlen($words[0]); for ($i=0; $i < $len; $i++) { $span_ix = count($tbt->spans) - 1; $span = $tbt->spans[$span_ix]; // TibetData if ($words[0][$i] == "-") { $cyc = new TibetCycle; $cyc->value = 0; $cyc->tibet_linenum = $ln; //$span->linenum; $cyc->tibet_line = $line; $cyc->span_ix = $span->span_ix; $span->cycles[] = $cyc; } else if ($words[0][$i] == ".") { $cyc = new TibetCycle; $cyc->value = 1; $cyc->tibet_linenum = $ln; //$span->linenum; $cyc->tibet_line = $line; $cyc->span_ix = $span->span_ix; $span->cycles[] = $cyc; } else if ($words[0][$i] == "P") { // P cannot be turned into a bit, so decoders just skip it. } else { print "E: line $ln, span $span_ix: Bad cycle line: $line\n"; return TBT_E_PARSE_CYCLES; } $tbt->spans[$span_ix] = $span; // replace the modified value } } } if (X_E_OK != $e) { return $e; } //print "\n"; return X_E_OK; } function tibet_parse_float (string $w, float &$f) : int { $len = strlen($w); $dp_count=0; if ($len > 50) { return TBT_E_BAD_FLOAT; } for ($i=0; $i < $len; $i++) { $c = $w[$i]; if ($c == ".") { if ($dp_count != 0) { // only one decimal point allowed return TBT_E_BAD_FLOAT; } else if ($i == ($len - 1)) { // decimal point may not be the final character return TBT_E_BAD_FLOAT; } $dp_count++; } else if ( ! ctype_digit ($c) ) { // chars other than digits and decimal point are illegal return TBT_E_BAD_FLOAT; } } $f = (float) $w; return X_E_OK; } function tibet_parse_int (string $w, int &$i) : int { $len = strlen($w); if ($len > 19) { return TBT_E_BAD_INT; } for ($j=0; $j < $len; $j++) { $v = $w[$j]; if (!ctype_digit($v)) { return TBT_E_BAD_INT; } } $i = (int) $w; return X_E_OK; } function tibet_parse_framing (string $w, DataFraming &$f) : int { if (strlen($w) != 3) { return FALSE; } if (($w[0] != "7") && ($w[0] != "8")) { return FALSE; } if (($w[1] != "N") && ($w[1] != "O") && ($w[1] != "E")) { return FALSE; } if (($w[2] != "1") && ($w[2] != "2")) { return FALSE; } $framelen = 0; $e = tibet_parse_int ($w[0], $framelen); // framelen populated if (X_E_OK != $e) { return $e; } $f->framelen = $framelen; $f->parity = $w[1]; $num_stops = 0; $e = tibet_parse_int ($w[2], $num_stops); // num_stops populated if (X_E_OK != $e) { return $e; } $f->stops = $num_stops; return X_E_OK; } function tibet_parse_baudrate (string $w, BaudRate &$r) : int { $i = 0; $e = tibet_parse_int ($w, $i); // i populated if (X_E_OK != $e) { return $e; } $r->rate = $i; return X_E_OK; } function tibet_parse_version (array $words, int $ln, string &$tbt_version, string $line) : int { $wc = count($words); if ($words[0] != "tibet") { print "E: Version line not found: $line\n"; return TBT_E_PARSE_VERSION; } if ($wc != 2) { print "E: line $ln: Bad version line: $line\n"; return TBT_E_PARSE_VERSION; } // 0.8: switched to major/minor version paradigm $tbt_version = $words[1]; $v = explode(".", $tbt_version); if (count($v) != 2) { print "E: line $ln: Bad version: $tbt_version\n"; return TBT_E_BAD_VERSION; } //if (TIBET_VERSION_STG != $tbt_version) { if ($v[0] != TIBET_MAJOR_VERSION) { print "E: line $ln: Incompatible TIBET version: $tbt_version\n"; return TBT_E_INCOMPATIBLE; } return X_E_OK; } function tibet_process (string $ip, bool $insert_timestamps, ParsedTibet &$tbt) : int { $state = TBT_STATE_VERSION; $lines = explode ("\n", $ip); $span_ix = 0; print count($lines)." lines.\n"; foreach ($lines as $ln=>$v) { $ln++; // linenum; 1-indexed $e = tibet_process_line ($ln, $v, $state, $span_ix, $insert_timestamps, $tbt); // state, tbt, span_ix modified if (X_E_OK != $e) { return $e; } } if ((TBT_STATE_IDLE != $state) && (STATE_DATA != $state)) { print "W: Finished in unexpected state $state\n"; } print "TIBET version: $tbt->version\n"; return X_E_OK; } ?>