diff --git a/README.md b/README.md index d902a472..158ae9f3 100644 --- a/README.md +++ b/README.md @@ -2,12 +2,12 @@ [![Circle CI](https://circleci.com/gh/keichi/binary-parser.svg?style=svg)](https://circleci.com/gh/keichi/binary-parser) -Binary-parser is a binary parser builder for [node](http://nodejs.org) that -enables you to write efficient parsers in a simple and declarative manner. +Binary-parser is a binary parser/encoder builder for [node](http://nodejs.org) that +enables you to write efficient parsers/encoders in a simple and declarative manner. It supports all common data types required to analyze a structured binary -data. Binary-parser dynamically generates and compiles the parser code -on-the-fly, which runs as fast as a hand-written parser (which takes much more +data. Binary-parser dynamically generates and compiles the parser and encoder code +on-the-fly, which runs as fast as a hand-written parser/encoder (which takes much more time and effort to write). Supported data types are: - Integers (supports 8, 16, 32 bit signed- and unsigned integers) @@ -31,11 +31,13 @@ $ npm install binary-parser ## Quick Start 1. Create an empty Parser object with `new Parser()`. -2. Chain builder methods to build the desired parser. (See +2. Chain builder methods to build the desired parser and/or encoder. (See [API](https://github.com/Keichi/binary-parser#api) for detailed document of each methods) 3. Call `Parser.prototype.parse` with an `Buffer` object passed as argument. 4. Parsed result will be returned as an object. +5. Or call `Parser.prototype.encode` with an object passed as argument. +6. Encoded result will be returned as a `Buffer` object. ```javascript // Module import @@ -68,19 +70,43 @@ var buf = Buffer.from("450002c5939900002c06ef98adc24f6c850186d1", "hex"); // Parse buffer and show result console.log(ipHeader.parse(buf)); + +var anIpHeader = { + version: 4, + headerLength: 5, + tos: 0, + packetLength: 709, + id: 37785, + offset: 0, + fragOffset: 0, + ttl: 44, + protocol: 6, + checksum: 61336, + src: [ 173, 194, 79, 108 ], + dst: [ 133, 1, 134, 209 ] }; + +// Encode an IP header object and show result as hex string +console.log(ipHeader.encode(anIpHeader).toString("hex")); ``` ## API -### new Parser() +### new Parser([options]) Constructs a Parser object. Returned object represents a parser which parses -nothing. +nothing. `options` is an optional object to pass options to this declarative +parser. + - `smartBufferSize` The chunk size of the encoding (smart)buffer (when encoding is used) (default is 256 bytes). ### parse(buffer) Parse a `Buffer` object `buffer` with this parser and return the resulting object. When `parse(buffer)` is called for the first time, parser code is compiled on-the-fly and internally cached. +### encode(obj) +Encode an `Object` object `obj` with this parser and return the resulting +`Buffer`. When `encode(obj)` is called for the first time, encoder code is +compiled on-the-fly and internally cached. + ### create(constructorFunction) Set the constructor function that should be called to create the object returned from the `parse` method. @@ -129,15 +155,18 @@ the following keys: `"utf8"`, `"ascii"`, `"hex"` and else are valid. See [`Buffer.toString`](http://nodejs.org/api/buffer.html#buffer_buf_tostring_encoding_start_end) for more info. -- `length ` - (Optional) Length of the string. Can be a number, string or a +- `length ` - (Optional) (Bytes)Length of the string. Can be a number, string or a function. Use number for statically sized arrays, string to reference another variable and function to do some calculation. + Note: when encoding the string is padded with spaces (0x20) at end to fit the length requirement. - `zeroTerminated` - (Optional, defaults to `false`) If true, then this parser reads until it reaches zero. - `greedy` - (Optional, defaults to `false`) If true, then this parser reads - until it reaches the end of the buffer. Will consume zero-bytes. + until it reaches the end of the buffer. Will consume zero-bytes. (Note: has + no effect on encoding function) - `stripNull` - (Optional, must be used with `length`) If true, then strip - null characters from end of the string + null characters from end of the string. (Note: has no effect on encoding, but + when used, then the parse() and encode() functions are not the exact opposite) ### buffer(name[, options]) Parse bytes as a buffer. `name` should consist only of alpha numeric @@ -154,7 +183,8 @@ the following keys: sized buffers, string to reference another variable and function to do some calculation. - `readUntil` - (either `length` or `readUntil` is required) If `"eof"`, then - this parser will read till it reaches end of the `Buffer` object. + this parser will read till it reaches end of the `Buffer` object. (Note: has no + effect on encoding.) ### array(name, options) Parse bytes as an array. `options` is an object which can have the following @@ -271,7 +301,8 @@ current object. `options` is an object which can have the following keys: - `type` - (Required) A `Parser` object. ### skip(length) -Skip parsing for `length` bytes. +Skip parsing for `length` bytes. (Note: when encoding, the skipped bytes will be filled +with zeros) ### endianess(endianess) Define what endianess to use in this parser. `endianess` can be either @@ -381,26 +412,44 @@ var buffer = Buffer.from([2, /* left */ 1, 1, 0, /* right */ 0]); parser.parse(buffer); ``` -### compile() -Compile this parser on-the-fly and cache its result. Usually, there is no need -to call this method directly, since it's called when `parse(buffer)` is +### compile() and compileEncode() +Compile this parser/encoder on-the-fly and cache its result. Usually, there is no need +to call this method directly, since it's called when `parse(buffer)` or `encode(obj)` is executed for the first time. -### getCode() -Dynamically generates the code for this parser and returns it as a string. +### getCode() and getCodeEncode() +Dynamically generates the code for this parser/encoder and returns it as a string. Usually used for debugging. ### Common options These are common options that can be specified in all parsers. - `formatter` - Function that transforms the parsed value into a more desired - form. + form. *formatter*(value, obj, buffer, offset) → *new value* \ + where `value` is the value to be formatted, `obj` is the current object being generated, `buffer` is the buffer currently beeing parsed and `offset` is the current offset in that buffer. + ```javascript + var parser = new Parser().array("ipv4", { + type: uint8, + length: "4", + formatter: function(arr, obj, buffer, offset) { + return arr.join("."); + } + }); + ``` + +- `encoder` - Function that transforms an object property into a more desired + form for encoding. This is the opposite of the above `formatter` function. \ + *encoder*(value) → *new value* \ + where `value` is the value to be encoded (de-formatted) and `obj` is the object currently being encoded. ```javascript var parser = new Parser().array("ipv4", { type: uint8, length: "4", - formatter: function(arr) { + formatter: function(arr, obj, buffer, offset) { return arr.join("."); + }, + encoder: function(str, obj) { + return str.split("."); } }); ``` diff --git a/lib/binary_parser.js b/lib/binary_parser.js index 12ece7ee..fadfaed7 100644 --- a/lib/binary_parser.js +++ b/lib/binary_parser.js @@ -2,6 +2,8 @@ // Globals //======================================================================================== var vm = require("vm"); +const SmartBuffer = require("smart-buffer").SmartBuffer; +global.SmartBuffer = SmartBuffer; var Context = require("./context").Context; @@ -34,6 +36,7 @@ var SPECIAL_TYPES = { var aliasRegistry = {}; var FUNCTION_PREFIX = "___parser_"; +var FUNCTION_ENCODE_PREFIX = "___encoder_"; var BIT_RANGE = []; (function() { @@ -59,7 +62,7 @@ Object.keys(PRIMITIVE_TYPES) // constructor //---------------------------------------------------------------------------------------- -var Parser = function() { +var Parser = function(opts) { this.varName = ""; this.type = ""; this.options = {}; @@ -69,14 +72,18 @@ var Parser = function() { this.endian = "be"; this.constructorFn = null; this.alias = null; + this.smartBufferSize = + opts && typeof opts === "object" && opts.smartBufferSize + ? opts.smartBufferSize + : 256; }; //---------------------------------------------------------------------------------------- // public methods //---------------------------------------------------------------------------------------- -Parser.start = function() { - return new Parser(); +Parser.start = function(opts) { + return new Parser(opts); }; Object.keys(PRIMITIVE_TYPES).forEach(function(type) { @@ -272,6 +279,23 @@ Parser.prototype.getCode = function() { return ctx.code; }; +Parser.prototype.getCodeEncode = function() { + var ctx = new Context(); + + ctx.pushCode("if (!obj || typeof obj !== 'object') {"); + ctx.generateError('"argument obj is not an object"'); + ctx.pushCode("}"); + + if (!this.alias) { + this.addRawCodeEncode(ctx); + } else { + this.addAliasedCodeEncode(ctx); + ctx.pushCode("return {0}(obj);", FUNCTION_ENCODE_PREFIX + this.alias); + } + + return ctx.code; +}; + Parser.prototype.addRawCode = function(ctx) { ctx.pushCode("var offset = 0;"); @@ -288,6 +312,20 @@ Parser.prototype.addRawCode = function(ctx) { ctx.pushCode("return vars;"); }; +Parser.prototype.addRawCodeEncode = function(ctx) { + ctx.pushCode("var vars = obj;"); + ctx.pushCode( + "var smartBuffer = SmartBuffer.fromOptions({size: {0}, encoding: 'utf8'});", + this.smartBufferSize + ); + + this.generateEncode(ctx); + + this.resolveReferences(ctx, "encode"); + + ctx.pushCode("return smartBuffer.toBuffer();"); +}; + Parser.prototype.addAliasedCode = function(ctx) { ctx.pushCode("function {0}(offset) {", FUNCTION_PREFIX + this.alias); @@ -308,12 +346,36 @@ Parser.prototype.addAliasedCode = function(ctx) { return ctx; }; -Parser.prototype.resolveReferences = function(ctx) { +Parser.prototype.addAliasedCodeEncode = function(ctx) { + ctx.pushCode("function {0}(obj) {", FUNCTION_ENCODE_PREFIX + this.alias); + + ctx.pushCode("var vars = obj;"); + ctx.pushCode( + "var smartBuffer = SmartBuffer.fromOptions({size: {0}, encoding: 'utf8'});", + this.smartBufferSize + ); + + this.generateEncode(ctx); + + ctx.markResolved(this.alias); + this.resolveReferences(ctx, "encode"); + + ctx.pushCode("return smartBuffer.toBuffer();"); + ctx.pushCode("}"); + + return ctx; +}; + +Parser.prototype.resolveReferences = function(ctx, encode) { var references = ctx.getUnresolvedReferences(); ctx.markRequested(references); references.forEach(function(alias) { var parser = aliasRegistry[alias]; - parser.addAliasedCode(ctx); + if (encode) { + parser.addAliasedCodeEncode(ctx); + } else { + parser.addAliasedCode(ctx); + } }); }; @@ -322,6 +384,11 @@ Parser.prototype.compile = function() { this.compiled = vm.runInThisContext(src); }; +Parser.prototype.compileEncode = function() { + var src = "(function(obj) { " + this.getCodeEncode() + " })"; + this.compiledEncode = vm.runInThisContext(src); +}; + Parser.prototype.sizeOf = function() { var size = NaN; @@ -379,6 +446,15 @@ Parser.prototype.parse = function(buffer) { return this.compiled(buffer, this.constructorFn); }; +// Follow the parser chain till the root and start encoding from there +Parser.prototype.encode = function(obj) { + if (!this.compiledEncode) { + this.compileEncode(); + } + + return this.compiledEncode(obj); +}; + //---------------------------------------------------------------------------------------- // private methods //---------------------------------------------------------------------------------------- @@ -416,6 +492,30 @@ Parser.prototype.generate = function(ctx) { return this.generateNext(ctx); }; +Parser.prototype.generateEncode = function(ctx) { + var varName = ctx.generateVariable(this.varName); + var savVarName = ctx.generateTmpVariable(); + + // Transform with the possibly provided encoder before encoding + if (this.options.encoder) { + // Save varName before applying encoder + ctx.pushCode("var {0} = {1};", savVarName, varName); + this.generateEncoder(ctx, varName, this.options.encoder); + } + + if (this.type) { + this["generate_encode" + this.type](ctx); + } + + if (this.options.encoder) { + // Restore varName after encoder transformation so that next parsers will + // have access to original field value (but not nested ones) + ctx.pushCode("{0} = {1};", varName, savVarName); + } + + return this.generateEncodeNext(ctx); +}; + Parser.prototype.generateAssert = function(ctx) { if (!this.options.assert) { return; @@ -455,6 +555,15 @@ Parser.prototype.generateNext = function(ctx) { return ctx; }; +// Recursively call code generators and append results +Parser.prototype.generateEncodeNext = function(ctx) { + if (this.next) { + ctx = this.next.generateEncode(ctx); + } + + return ctx; +}; + Object.keys(PRIMITIVE_TYPES).forEach(function(type) { Parser.prototype["generate" + type] = function(ctx) { ctx.pushCode( @@ -464,6 +573,13 @@ Object.keys(PRIMITIVE_TYPES).forEach(function(type) { ); ctx.pushCode("offset += {0};", PRIMITIVE_TYPES[type]); }; + Parser.prototype["generate_encode" + type] = function(ctx) { + ctx.pushCode( + "smartBuffer.write{0}({1});", + type, + ctx.generateVariable(this.varName) + ); + }; }); Parser.prototype.generateBit = function(ctx) { @@ -523,11 +639,77 @@ Parser.prototype.generateBit = function(ctx) { } }; +Parser.prototype.generate_encodeBit = function(ctx) { + // TODO find better method to handle nested bit fields + var parser = JSON.parse(JSON.stringify(this)); + parser.varName = ctx.generateVariable(parser.varName); + ctx.bitFields.push(parser); + + if ( + !this.next || + (this.next && ["Bit", "Nest"].indexOf(this.next.type) < 0) + ) { + var sum = 0; + ctx.bitFields.forEach(function(parser) { + sum += parser.options.length; + }); + + if (sum <= 8) { + sum = 8; + } else if (sum <= 16) { + sum = 16; + } else if (sum <= 24) { + sum = 24; + } else if (sum <= 32) { + sum = 32; + } else { + throw new Error( + "Currently, bit field sequence longer than 4-bytes is not supported." + ); + } + + var tmpVal = ctx.generateTmpVariable(); + ctx.pushCode("var {0} = 0;", tmpVal); + var bitOffset = 0; + ctx.bitFields.forEach(function(parser) { + ctx.pushCode( + "{0} |= ({1} << {2});", + tmpVal, + parser.varName, + sum - parser.options.length - bitOffset + ); + ctx.pushCode("{0} = {0} >>> 0;", tmpVal); + bitOffset += parser.options.length; + }); + if (sum == 8) { + ctx.pushCode("smartBuffer.writeUInt8({0});", tmpVal); + } else if (sum == 16) { + ctx.pushCode("smartBuffer.writeUInt16BE({0});", tmpVal); + } else if (sum == 24) { + var val1 = ctx.generateTmpVariable(); + var val2 = ctx.generateTmpVariable(); + ctx.pushCode("var {0} = ({1} >>> 8);", val1, tmpVal); + ctx.pushCode("var {0} = ({1} & 0x0ff);", val2, tmpVal); + ctx.pushCode("smartBuffer.writeUInt16BE({0});", val1); + ctx.pushCode("smartBuffer.writeUInt8({0});", val2); + } else if (sum == 32) { + ctx.pushCode("smartBuffer.writeUInt32BE({0});", tmpVal); + } + + ctx.bitFields = []; + } +}; + Parser.prototype.generateSkip = function(ctx) { var length = ctx.generateOption(this.options.length); ctx.pushCode("offset += {0};", length); }; +Parser.prototype.generate_encodeSkip = function(ctx) { + var length = ctx.generateOption(this.options.length); + ctx.pushCode("smartBuffer.writeBuffer(Buffer.alloc({0}));", length); +}; + Parser.prototype.generateString = function(ctx) { var name = ctx.generateVariable(this.varName); var start = ctx.generateTmpVariable(); @@ -578,6 +760,45 @@ Parser.prototype.generateString = function(ctx) { } }; +Parser.prototype.generate_encodeString = function(ctx) { + var name = ctx.generateVariable(this.varName); + var maxLen = ctx.generateTmpVariable(); + + // Get the length of string to encode + if (this.options.length) { + var optLength = ctx.generateOption(this.options.length); + // Encode the string to a temporary buffer + var tmpBuf = ctx.generateTmpVariable(); + ctx.pushCode( + "var {0} = Buffer.from({1}, '{2}');", + tmpBuf, + name, + this.options.encoding + ); + // Truncate the buffer to specified (Bytes) length + ctx.pushCode("{0} = {0}.slice(0, {1});", tmpBuf, optLength); + // Compute padding length + var padLen = ctx.generateTmpVariable(); + ctx.pushCode("{0} = {1} - {2}.length;", padLen, optLength, tmpBuf); + // Copy the temporary string buffer to current smartBuffer + ctx.pushCode("smartBuffer.writeBuffer({0});", tmpBuf); + // Add padding spaces + ctx.pushCode( + "if ({0} > 0) {smartBuffer.writeString(' '.repeat({0}));}", + padLen + ); + } else { + ctx.pushCode( + "smartBuffer.writeString({0}, '{1}');", + name, + this.options.encoding + ); + } + if (this.options.zeroTerminated) { + ctx.pushCode("smartBuffer.writeUInt8(0x00);"); + } +}; + Parser.prototype.generateBuffer = function(ctx) { if (this.options.readUntil === "eof") { ctx.pushCode( @@ -598,6 +819,13 @@ Parser.prototype.generateBuffer = function(ctx) { } }; +Parser.prototype.generate_encodeBuffer = function(ctx) { + ctx.pushCode( + "smartBuffer.writeBuffer({0});", + ctx.generateVariable(this.varName) + ); +}; + Parser.prototype.generateArray = function(ctx) { var length = ctx.generateOption(this.options.length); var lengthInBytes = ctx.generateOption(this.options.lengthInBytes); @@ -662,6 +890,95 @@ Parser.prototype.generateArray = function(ctx) { } }; +Parser.prototype.generate_encodeArray = function(ctx) { + var length = ctx.generateOption(this.options.length); + var lengthInBytes = ctx.generateOption(this.options.lengthInBytes); + var type = this.options.type; + var name = ctx.generateVariable(this.varName); + var item = ctx.generateTmpVariable(); + var itemCounter = ctx.generateTmpVariable(); + var maxItems = ctx.generateTmpVariable(); + var maxOffset = ctx.generateTmpVariable(); + var isHash = typeof this.options.key === "string"; + + if (isHash) { + ctx.generateError('"Encoding associative array not supported"'); + } + + // Compute the desired count of array items to encode (min of array size and length option) + if (length !== undefined) { + var tmpLength = ctx.generateTmpVariable(); + ctx.pushCode("var {0} = {1};", tmpLength, length); + ctx.pushCode( + "var {0} = {1}.length > {2} ? {2} : {1}.length;", + maxItems, + name, + tmpLength + ); + } else { + ctx.pushCode("var {0} = {1}.length;", maxItems, name); + } + + // Save current encoding smartBuffer and allocate a new one + var savSmartBuffer = ctx.generateTmpVariable(); + ctx.pushCode( + "var {0} = smartBuffer; smartBuffer = SmartBuffer.fromOptions({size: {1}, encoding: 'utf8'});", + savSmartBuffer, + this.smartBufferSize + ); + + ctx.pushCode("var {0} = 0;", itemCounter); + if (typeof this.options.readUntil === "function") { + ctx.pushCode("do {"); + } else { + ctx.pushCode("for ( ; {0} < {1}; ) {", itemCounter, maxItems); + } + + ctx.pushCode("var {0} = {1}[{2}];", item, name, itemCounter); + ctx.pushCode("{0}++;", itemCounter); + + if (typeof type === "string") { + if (!aliasRegistry[type]) { + ctx.pushCode("smartBuffer.write{0}({1});", NAME_MAP[type], item); + } else { + ctx.pushCode( + "smartBuffer.writeBuffer({0}({1}));", + FUNCTION_ENCODE_PREFIX + type, + item + ); + if (type !== this.alias) { + ctx.addReference(type); + } + } + } else if (type instanceof Parser) { + ctx.pushScope(item); + type.generateEncode(ctx); + ctx.popScope(); + } + + ctx.pushCode("}"); // End of 'do {' or 'for(...) {' + + if (typeof this.options.readUntil === "function") { + ctx.pushCode( + " while (!({0}).call(this, {1}, {2}.toBuffer()));", + this.options.readUntil, + item, + savSmartBuffer + ); + } + + var tmpBuffer = ctx.generateTmpVariable(); + ctx.pushCode("var {0} = smartBuffer.toBuffer()", tmpBuffer); + if (lengthInBytes !== undefined) { + // Truncate the tmpBuffer so that it will respect the lengthInBytes option + ctx.pushCode("{0} = {0}.slice(0, {1});", tmpBuffer, lengthInBytes); + } + // Copy tmp Buffer to saved smartBuffer + ctx.pushCode("{0}.writeBuffer({1});", savSmartBuffer, tmpBuffer); + // Restore current smartBuffer + ctx.pushCode("smartBuffer = {0};", savSmartBuffer); +}; + Parser.prototype.generateChoiceCase = function(ctx, varName, type) { if (typeof type === "string") { if (!aliasRegistry[type]) { @@ -688,6 +1005,32 @@ Parser.prototype.generateChoiceCase = function(ctx, varName, type) { } }; +Parser.prototype.generate_encodeChoiceCase = function(ctx, varName, type) { + if (typeof type === "string") { + if (!aliasRegistry[type]) { + ctx.pushCode( + "smartBuffer.write{0}({1});", + NAME_MAP[type], + ctx.generateVariable(this.varName) + ); + } else { + var tempVar = ctx.generateTmpVariable(); + ctx.pushCode( + "var {0} = {1}({2});", + tempVar, + FUNCTION_ENCODE_PREFIX + type, + ctx.generateVariable(this.varName) + ); + ctx.pushCode("smartBuffer.writeBuffer({0});", tempVar); + if (type !== this.alias) ctx.addReference(type); + } + } else if (type instanceof Parser) { + ctx.pushPath(varName); + type.generateEncode(ctx); + ctx.popPath(varName); + } +}; + Parser.prototype.generateChoice = function(ctx) { var tag = ctx.generateOption(this.options.tag); if (this.varName) { @@ -710,6 +1053,29 @@ Parser.prototype.generateChoice = function(ctx) { ctx.pushCode("}"); }; +Parser.prototype.generate_encodeChoice = function(ctx) { + var tag = ctx.generateOption(this.options.tag); + ctx.pushCode("switch({0}) {", tag); + Object.keys(this.options.choices).forEach(function(tag) { + var type = this.options.choices[tag]; + + ctx.pushCode("case {0}:", tag); + this.generate_encodeChoiceCase(ctx, this.varName, type); + ctx.pushCode("break;"); + }, this); + ctx.pushCode("default:"); + if (this.options.defaultChoice) { + this.generate_encodeChoiceCase( + ctx, + this.varName, + this.options.defaultChoice + ); + } else { + ctx.generateError('"Met undefined tag value " + {0} + " at choice"', tag); + } + ctx.pushCode("}"); +}; + Parser.prototype.generateNest = function(ctx) { var nestVar = ctx.generateVariable(this.varName); @@ -732,9 +1098,39 @@ Parser.prototype.generateNest = function(ctx) { } }; +Parser.prototype.generate_encodeNest = function(ctx) { + var nestVar = ctx.generateVariable(this.varName); + + if (this.options.type instanceof Parser) { + ctx.pushPath(this.varName); + this.options.type.generateEncode(ctx); + ctx.popPath(this.varName); + } else if (aliasRegistry[this.options.type]) { + var tempVar = ctx.generateTmpVariable(); + ctx.pushCode( + "var {0} = {1}({2});", + tempVar, + FUNCTION_ENCODE_PREFIX + this.options.type, + nestVar + ); + ctx.pushCode("smartBuffer.writeBuffer({0});", tempVar); + if (this.options.type !== this.alias) ctx.addReference(this.options.type); + } +}; + Parser.prototype.generateFormatter = function(ctx, varName, formatter) { if (typeof formatter === "function") { - ctx.pushCode("{0} = ({1}).call(this, {0});", varName, formatter); + ctx.pushCode( + "{0} = ({1}).call(this, {0}, vars, buffer, offset);", + varName, + formatter + ); + } +}; + +Parser.prototype.generateEncoder = function(ctx, varName, encoder) { + if (typeof encoder === "function") { + ctx.pushCode("{0} = ({1}).call(this, {0}, vars);", varName, encoder); } }; diff --git a/package-lock.json b/package-lock.json index 572f2edb..8f413f2f 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,6 +1,6 @@ { "name": "binary-parser", - "version": "1.3.1", + "version": "1.3.2", "lockfileVersion": 1, "requires": true, "dependencies": { @@ -602,6 +602,11 @@ "align-text": "0.1.4" } }, + "smart-buffer": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/smart-buffer/-/smart-buffer-4.0.1.tgz", + "integrity": "sha512-RFqinRVJVcCAL9Uh1oVqE6FZkqsyLiVOYEZ20TqIOjuX7iFVJ+zsbs4RIghnw/pTs7mZvt8ZHhvm1ZUrR4fykg==" + }, "source-map": { "version": "0.2.0", "resolved": "https://registry.npmjs.org/source-map/-/source-map-0.2.0.tgz", diff --git a/package.json b/package.json index be2f6595..998e8211 100644 --- a/package.json +++ b/package.json @@ -34,8 +34,10 @@ "url": "http://github.com/keichi/binary-parser.git" }, "bugs": "http://github.com/keichi/binary-parser/issues", - "dependencies": {}, + "dependencies": { + "smart-buffer": "^4.0.1" + }, "engines": { - "node": ">=5.10.0" + "node": ">=5.10.0" } } diff --git a/test/yy_primitive_encoder.js b/test/yy_primitive_encoder.js new file mode 100644 index 00000000..95fd7848 --- /dev/null +++ b/test/yy_primitive_encoder.js @@ -0,0 +1,385 @@ +var assert = require("assert"); +var util = require("util"); +var Parser = require("../lib/binary_parser").Parser; + +describe("Primitive encoder", function() { + describe("Primitive encoders", function() { + it("should nothing", function() { + var parser = Parser.start(); + + var buffer = parser.encode({ a: 0, b: 1 }); + assert.deepEqual(buffer.length, 0); + }); + it("should encode integer types", function() { + var parser = Parser.start() + .uint8("a") + .int16le("b") + .uint32be("c"); + + var buffer = Buffer.from([0x00, 0xd2, 0x04, 0x00, 0xbc, 0x61, 0x4e]); + var parsed = parser.parse(buffer); + var encoded = parser.encode(parsed); + assert.deepEqual(parsed, { a: 0, b: 1234, c: 12345678 }); + assert.deepEqual(encoded, buffer); + }); + it("should use encoder to transform to integer", function() { + var parser = Parser.start() + .uint8("a", { + formatter: function(val) { + return val * 2; + }, + encoder: function(val) { + return val / 2; + } + }) + .int16le("b", { + formatter: function(val) { + return "test" + String(val); + }, + encoder: function(val) { + return parseInt(val.substr("test".length)); + } + }); + + var buffer = Buffer.from([0x01, 0xd2, 0x04]); + var parsed = parser.parse(buffer); + var parsedClone = Object.assign({}, parsed); + var encoded = parser.encode(parsedClone); + assert.deepEqual(parsed, { a: 2, b: "test1234" }); + assert.deepEqual(encoded, buffer); + }); + it("should encode floating point types", function() { + var parser = Parser.start() + .floatbe("a") + .doublele("b"); + + var FLT_EPSILON = 0.00001; + var buffer = Buffer.from([ + 0x41, + 0x45, + 0x85, + 0x1f, + 0x7a, + 0x36, + 0xab, + 0x3e, + 0x57, + 0x5b, + 0xb1, + 0xbf + ]); + var result = parser.parse(buffer); + + assert(Math.abs(result.a - 12.345) < FLT_EPSILON); + assert(Math.abs(result.b - -0.0678) < FLT_EPSILON); + var encoded = parser.encode(result); + assert.deepEqual(encoded, buffer); + }); + it("should handle endianess", function() { + var parser = Parser.start() + .int32le("little") + .int32be("big"); + + var buffer = Buffer.from([ + 0x4e, + 0x61, + 0xbc, + 0x00, + 0x00, + 0xbc, + 0x61, + 0x4e + ]); + var parsed = parser.parse(buffer); + assert.deepEqual(parsed, { + little: 12345678, + big: 12345678 + }); + var encoded = parser.encode(parsed); + assert.deepEqual(encoded, buffer); + }); + it("should skip when specified", function() { + var parser = Parser.start() + .uint8("a") + .skip(3) + .uint16le("b") + .uint32be("c"); + + var buffer = Buffer.from([ + 0x00, + 0x00, // Skipped will be encoded as Null + 0x00, // Skipped will be encoded as Null + 0x00, // Skipped will be encoded as Null + 0xd2, + 0x04, + 0x00, + 0xbc, + 0x61, + 0x4e + ]); + var parsed = parser.parse(buffer); + assert.deepEqual(parsed, { a: 0, b: 1234, c: 12345678 }); + var encoded = parser.encode(parsed); + assert.deepEqual(encoded, buffer); + }); + }); + + describe("Bit field encoders", function() { + var binaryLiteral = function(s) { + var i; + var bytes = []; + + s = s.replace(/\s/g, ""); + for (i = 0; i < s.length; i += 8) { + bytes.push(parseInt(s.slice(i, i + 8), 2)); + } + + return Buffer.from(bytes); + }; + + it("binary literal helper should work", function() { + assert.deepEqual(binaryLiteral("11110000"), Buffer.from([0xf0])); + assert.deepEqual( + binaryLiteral("11110000 10100101"), + Buffer.from([0xf0, 0xa5]) + ); + }); + + it("should encode 1-byte-length bit field sequence", function() { + var parser = new Parser() + .bit1("a") + .bit2("b") + .bit4("c") + .bit1("d"); + + var buf = binaryLiteral("1 10 1010 0"); + var decoded = parser.parse(buf); + assert.deepEqual(decoded, { + a: 1, + b: 2, + c: 10, + d: 0 + }); + + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buf); + + // Endianess will change nothing you still specify bits for left to right + parser = new Parser() + .endianess("little") + .bit1("a") + .bit2("b") + .bit4("c") + .bit1("d"); + + encoded = parser.encode({ + a: 1, + b: 2, + c: 10, + d: 0 + }); + assert.deepEqual(encoded, buf); + }); + it("should parse 2-byte-length bit field sequence", function() { + var parser = new Parser() + .bit3("a") + .bit9("b") + .bit4("c"); + + var buf = binaryLiteral("101 111000111 0111"); + var decoded = parser.parse(buf); + assert.deepEqual(decoded, { + a: 5, + b: 455, + c: 7 + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buf); + }); + it("should parse 4-byte-length bit field sequence", function() { + var parser = new Parser() + .bit1("a") + .bit24("b") + .bit4("c") + .bit2("d") + .bit1("e"); + var buf = binaryLiteral("1 101010101010101010101010 1111 01 1"); + var decoded = parser.parse(buf); + assert.deepEqual(decoded, { + a: 1, + b: 11184810, + c: 15, + d: 1, + e: 1 + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buf); + }); + it("should parse nested bit fields", function() { + var parser = new Parser().bit1("a").nest("x", { + type: new Parser() + .bit2("b") + .bit4("c") + .bit1("d") + }); + + var buf = binaryLiteral("1 10 1010 0"); + var decoded = parser.parse(buf); + assert.deepEqual(decoded, { + a: 1, + x: { + b: 2, + c: 10, + d: 0 + } + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buf); + }); + }); + + describe("String encoder", function() { + it("should encode ASCII encoded string", function() { + var text = "hello, world"; + var buffer = Buffer.from(text, "ascii"); + var parser = Parser.start().string("msg", { + length: buffer.length, + encoding: "ascii" + }); + + var decoded = parser.parse(buffer); + assert.equal(decoded.msg, text); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should encode UTF8 encoded string", function() { + var text = "こんにちは、せかい。"; + var buffer = Buffer.from(text, "utf8"); + var parser = Parser.start().string("msg", { + length: buffer.length, + encoding: "utf8" + }); + + var decoded = parser.parse(buffer); + assert.equal(decoded.msg, text); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should encode HEX encoded string", function() { + var text = "cafebabe"; + var buffer = Buffer.from(text, "hex"); + var parser = Parser.start().string("msg", { + length: buffer.length, + encoding: "hex" + }); + + var decoded = parser.parse(buffer); + assert.equal(decoded.msg, text); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should encode variable length string", function() { + var buffer = Buffer.from("0c68656c6c6f2c20776f726c64", "hex"); + var parser = Parser.start() + .uint8("length") + .string("msg", { length: "length", encoding: "utf8" }); + + var decoded = parser.parse(buffer); + assert.equal(decoded.msg, "hello, world"); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should encode zero terminated string", function() { + var buffer = Buffer.from("68656c6c6f2c20776f726c6400", "hex"); + var parser = Parser.start().string("msg", { + zeroTerminated: true, + encoding: "ascii" + }); + + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { msg: "hello, world" }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should encode zero terminated fixed-length string", function() { + // In that case parsing and encoding are not the exact oposite + var buffer = Buffer.from("abc\u0000defghij\u0000"); + var parser = Parser.start() + .string("a", { length: 5, zeroTerminated: true }) + .string("b", { length: 5, zeroTerminated: true }) + .string("c", { length: 5, zeroTerminated: true }); + + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + a: "abc", + b: "defgh", + c: "ij" + }); + + var encoded = parser.encode({ + a: "abc", + b: "defghzzzzzzz", + c: "ij" + }); + assert.deepEqual( + encoded, + Buffer.from("abc \u0000defgh\u0000ij \u0000") + ); + }); + it("should strip trailing null characters", function() { + var buffer = Buffer.from("746573740000", "hex"); + var parser1 = Parser.start().string("str", { + length: 6, + stripNull: false + }); + var parser2 = Parser.start().string("str", { + length: 6, + stripNull: true + }); + + var decoded1 = parser1.parse(buffer); + assert.equal(decoded1.str, "test\u0000\u0000"); + var encoded1 = parser1.encode(decoded1); + assert.deepEqual(encoded1, buffer); + + var decoded2 = parser2.parse(buffer); + assert.equal(decoded2.str, "test"); + // In this case (stripNull = true) parsing and encoding are not the exact oposite + var encoded2 = parser2.encode(decoded2); + assert.notDeepEqual(encoded2, buffer); + assert.deepEqual(encoded2, Buffer.from("test ")); + }); + it("should encode string with zero-bytes internally", function() { + var buffer = Buffer.from("abc\u0000defghij\u0000"); + var parser = Parser.start().string("a", { greedy: true }); + + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + a: "abc\u0000defghij\u0000" + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + }); + + describe("Buffer encoder", function() { + it("should encode buffer", function() { + var parser = new Parser().uint8("len").buffer("raw", { + length: "len" + }); + + var buf = Buffer.from("deadbeefdeadbeef", "hex"); + var result = parser.parse( + Buffer.concat([Buffer.from([8]), buf, Buffer.from("garbage at end")]) + ); + + assert.deepEqual(result, { + len: 8, + raw: buf + }); + + var encoded = parser.encode(result); + assert.deepEqual(encoded, Buffer.concat([Buffer.from([8]), buf])); + }); + }); +}); diff --git a/test/zz_composite_encoder.js b/test/zz_composite_encoder.js new file mode 100644 index 00000000..fba23606 --- /dev/null +++ b/test/zz_composite_encoder.js @@ -0,0 +1,1002 @@ +var assert = require("assert"); +var util = require("util"); +var Parser = require("../lib/binary_parser").Parser; + +describe("Composite encoder", function() { + describe("Array encoder", function() { + it("should encode array of primitive types", function() { + var parser = Parser.start() + .uint8("length") + .array("message", { + length: "length", + type: "uint8" + }); + + var buffer = Buffer.from([12, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + length: 12, + message: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12] + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should encode array of primitive types with lengthInBytes", function() { + var parser = Parser.start() + .uint8("length") + .array("message", { + lengthInBytes: "length", + type: "uint8" + }); + + var buffer = Buffer.from([12, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + length: 12, + message: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12] + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should encode array of primitive types with lengthInBytes as a maximum but not minimum", function() { + var parser = Parser.start() + .uint8("length") + .array("message", { + lengthInBytes: "length", + type: "uint8" + }); + var encoded = parser.encode({ + length: 5, + message: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] // Extra items in array than encoding limit + }); + assert.deepEqual(encoded, Buffer.from([5, 1, 2, 3, 4, 5])); + encoded = parser.encode({ + length: 5, + message: [1, 2, 3] // Less items in array than encoding limit + }); + assert.deepEqual(encoded, Buffer.from([5, 1, 2, 3])); + }); + it("should encode array of user defined types", function() { + var elementParser = new Parser().uint8("key").int16le("value"); + + var parser = Parser.start() + .uint16le("length") + .array("message", { + length: "length", + type: elementParser + }); + + var buffer = Buffer.from([ + 0x02, + 0x00, + 0xca, + 0xd2, + 0x04, + 0xbe, + 0xd3, + 0x04 + ]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + length: 0x02, + message: [{ key: 0xca, value: 1234 }, { key: 0xbe, value: 1235 }] + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should encode array of user defined types with lengthInBytes", function() { + var elementParser = new Parser().uint8("key").int16le("value"); + + var parser = Parser.start() + .uint16le("length") + .array("message", { + lengthInBytes: "length", + type: elementParser + }); + + var buffer = Buffer.from([ + 0x06, + 0x00, + 0xca, + 0xd2, + 0x04, + 0xbe, + 0xd3, + 0x04 + ]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + length: 0x06, + message: [{ key: 0xca, value: 1234 }, { key: 0xbe, value: 1235 }] + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should encode array of user defined types with length function", function() { + var elementParser = new Parser().uint8("key").int16le("value"); + + var parser = Parser.start() + .uint16le("length") + .array("message", { + length: function() { + return this.length; + }, + type: elementParser + }); + + var buffer = Buffer.from([ + 0x02, + 0x00, + 0xca, + 0xd2, + 0x04, + 0xbe, + 0xd3, + 0x04 + ]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + length: 0x02, + message: [{ key: 0xca, value: 1234 }, { key: 0xbe, value: 1235 }] + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should encode array of arrays", function() { + var rowParser = Parser.start() + .uint8("length") + .array("cols", { + length: "length", + type: "int32le" + }); + + var parser = Parser.start() + .uint8("length") + .array("rows", { + length: "length", + type: rowParser + }); + + var buffer = Buffer.alloc(1 + 10 * (1 + 5 * 4)); + var i, j; + + iterator = 0; + buffer.writeUInt8(10, iterator); + iterator += 1; + for (i = 0; i < 10; i++) { + buffer.writeUInt8(5, iterator); + iterator += 1; + for (j = 0; j < 5; j++) { + buffer.writeInt32LE(i * j, iterator); + iterator += 4; + } + } + + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + length: 10, + rows: [ + { length: 5, cols: [0, 0, 0, 0, 0] }, + { length: 5, cols: [0, 1, 2, 3, 4] }, + { length: 5, cols: [0, 2, 4, 6, 8] }, + { length: 5, cols: [0, 3, 6, 9, 12] }, + { length: 5, cols: [0, 4, 8, 12, 16] }, + { length: 5, cols: [0, 5, 10, 15, 20] }, + { length: 5, cols: [0, 6, 12, 18, 24] }, + { length: 5, cols: [0, 7, 14, 21, 28] }, + { length: 5, cols: [0, 8, 16, 24, 32] }, + { length: 5, cols: [0, 9, 18, 27, 36] } + ] + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should encode until function returns true when readUntil is function", function() { + var parser = Parser.start().array("data", { + readUntil: function(item, buf) { + return item === 0; + }, + type: "uint8" + }); + + var buffer = Buffer.from([ + 0xff, + 0xff, + 0xff, + 0x01, + 0x00, + 0xff, + 0xff, + 0xff, + 0xff, + 0xff + ]); + assert.deepEqual(parser.parse(buffer), { + data: [0xff, 0xff, 0xff, 0x01, 0x00] + }); + var encoded = parser.encode({ + ignore1: [0x00, 0x00], + data: [0xff, 0xff, 0xff, 0x01, 0x00, 0xff, 0xff, 0x00, 0xff], + ignore2: [0x01, 0x00, 0xff] + }); + assert.deepEqual(encoded, Buffer.from([0xff, 0xff, 0xff, 0x01, 0x00])); + }); + it("should not support associative arrays", function() { + var parser = Parser.start() + .int8("numlumps") + .array("lumps", { + type: Parser.start() + .int32le("filepos") + .int32le("size") + .string("name", { length: 8, encoding: "ascii" }), + length: "numlumps", + key: "name" + }); + + assert.throws(function() { + parser.encode({ + numlumps: 2, + lumps: { + AAAAAAAA: { + filepos: 1234, + size: 5678, + name: "AAAAAAAA" + }, + bbbbbbbb: { + filepos: 5678, + size: 1234, + name: "bbbbbbbb" + } + } + }); + }, /Encoding associative array not supported/); + }); + it("should use encoder to transform encoded array", function() { + var parser = Parser.start().array("data", { + type: "uint8", + length: 4, + formatter: function(arr) { + return arr.join("."); + }, + encoder: function(str) { + return str.split("."); + } + }); + + var buffer = Buffer.from([0x0a, 0x0a, 0x01, 0x6e]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + data: "10.10.1.110" + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should be able to go into recursion", function() { + var parser = Parser.start() + .namely("self") + .uint8("length") + .array("data", { + type: "self", + length: "length" + }); + + var buffer = Buffer.from([1, 1, 1, 0]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + length: 1, + data: [ + { + length: 1, + data: [ + { + length: 1, + data: [{ length: 0, data: [] }] + } + ] + } + ] + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should be able to go into even deeper recursion", function() { + var parser = Parser.start() + .namely("self") + .uint8("length") + .array("data", { + type: "self", + length: "length" + }); + + // 2 + // / \ + // 3 1 + // / | \ \ + // 1 0 2 0 + // / / \ + // 0 1 0 + // / + // 0 + + var buffer = Buffer.from([ + 2, + /* 0 */ 3, + /* 0 */ 1, + /* 0 */ 0, + /* 1 */ 0, + /* 2 */ 2, + /* 0 */ 1, + /* 0 */ 0, + /* 1 */ 0, + /* 1 */ 1, + /* 0 */ 0 + ]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + length: 2, + data: [ + { + length: 3, + data: [ + { length: 1, data: [{ length: 0, data: [] }] }, + { length: 0, data: [] }, + { + length: 2, + data: [ + { length: 1, data: [{ length: 0, data: [] }] }, + { length: 0, data: [] } + ] + } + ] + }, + { + length: 1, + data: [{ length: 0, data: [] }] + } + ] + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + + it("should allow parent parser attributes as choice key", function() { + var ChildParser = Parser.start().choice("data", { + tag: function(vars) { + return vars.version; + }, + choices: { + 1: Parser.start().uint8("v1"), + 2: Parser.start().uint16("v2") + } + }); + + var ParentParser = Parser.start() + .uint8("version") + .nest("child", { type: ChildParser }); + + var buffer = Buffer.from([0x1, 0x2]); + var decoded = ParentParser.parse(buffer); + assert.deepEqual(decoded, { + version: 1, + child: { data: { v1: 2 } } + }); + var encoded = ParentParser.encode(decoded); + assert.deepEqual(encoded, buffer); + + buffer = Buffer.from([0x2, 0x3, 0x4]); + decoded = ParentParser.parse(buffer); + assert.deepEqual(decoded, { + version: 2, + child: { data: { v2: 0x0304 } } + }); + encoded = ParentParser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + }); + + describe("Choice encoder", function() { + it("should encode choices of primitive types", function() { + var parser = Parser.start() + .uint8("tag1") + .choice("data1", { + tag: "tag1", + choices: { + 0: "int32le", + 1: "int16le" + } + }) + .uint8("tag2") + .choice("data2", { + tag: "tag2", + choices: { + 0: "int32le", + 1: "int16le" + } + }); + + var buffer = Buffer.from([0x0, 0x4e, 0x61, 0xbc, 0x00, 0x01, 0xd2, 0x04]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + tag1: 0, + data1: 12345678, + tag2: 1, + data2: 1234 + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should encode default choice", function() { + var parser = Parser.start() + .uint8("tag") + .choice("data", { + tag: "tag", + choices: { + 0: "int32le", + 1: "int16le" + }, + defaultChoice: "uint8" + }) + .int32le("test"); + + buffer = Buffer.from([0x03, 0xff, 0x2f, 0xcb, 0x04, 0x0]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + tag: 3, + data: 0xff, + test: 314159 + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should parse choices of user defied types", function() { + var parser = Parser.start() + .uint8("tag") + .choice("data", { + tag: "tag", + choices: { + 1: Parser.start() + .uint8("length") + .string("message", { length: "length" }), + 3: Parser.start().int32le("number") + } + }); + + var buffer = Buffer.from([ + 0x1, + 0xc, + 0x68, + 0x65, + 0x6c, + 0x6c, + 0x6f, + 0x2c, + 0x20, + 0x77, + 0x6f, + 0x72, + 0x6c, + 0x64 + ]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + tag: 1, + data: { + length: 12, + message: "hello, world" + } + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + buffer = Buffer.from([0x03, 0x4e, 0x61, 0xbc, 0x00]); + decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + tag: 3, + data: { + number: 12345678 + } + }); + encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should be able to go into recursion", function() { + var stop = Parser.start(); + + var parser = Parser.start() + .namely("self") + .uint8("type") + .choice("data", { + tag: "type", + choices: { + 0: stop, + 1: "self" + } + }); + + var buffer = Buffer.from([1, 1, 1, 0]); + var decoded = parser.parse(buffer); + assert.deepEqual(parser.parse(buffer), { + type: 1, + data: { + type: 1, + data: { + type: 1, + data: { type: 0, data: {} } + } + } + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should be able to go into recursion with simple nesting", function() { + var stop = Parser.start(); + + var parser = Parser.start() + .namely("self") + .uint8("type") + .choice("data", { + tag: "type", + choices: { + 0: stop, + 1: "self", + 2: Parser.start() + .nest("left", { type: "self" }) + .nest("right", { type: stop }) + } + }); + + var buffer = Buffer.from([2, /* left */ 1, 1, 0 /* right */]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + type: 2, + data: { + left: { + type: 1, + data: { + type: 1, + data: { + type: 0, + data: {} + } + } + }, + right: {} + } + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should be able to refer to other parsers by name", function() { + var parser = Parser.start().namely("self"); + + var stop = Parser.start().namely("stop"); + + var twoCells = Parser.start() + .namely("twoCells") + .nest("left", { type: "self" }) + .nest("right", { type: "stop" }); + + parser.uint8("type").choice("data", { + tag: "type", + choices: { + 0: "stop", + 1: "self", + 2: "twoCells" + } + }); + + var buffer = Buffer.from([2, /* left */ 1, 1, 0 /* right */]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + type: 2, + data: { + left: { + type: 1, + data: { type: 1, data: { type: 0, data: {} } } + }, + right: {} + } + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should be able to refer to other parsers both directly and by name", function() { + var parser = Parser.start().namely("self"); + + var stop = Parser.start(); + + var twoCells = Parser.start() + .nest("left", { type: "self" }) + .nest("right", { type: stop }); + + parser.uint8("type").choice("data", { + tag: "type", + choices: { + 0: stop, + 1: "self", + 2: twoCells + } + }); + + var buffer = Buffer.from([2, /* left */ 1, 1, 0 /* right */]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + type: 2, + data: { + left: { + type: 1, + data: { type: 1, data: { type: 0, data: {} } } + }, + right: {} + } + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should be able to go into recursion with complex nesting", function() { + var stop = Parser.start(); + + var parser = Parser.start() + .namely("self") + .uint8("type") + .choice("data", { + tag: "type", + choices: { + 0: stop, + 1: "self", + 2: Parser.start() + .nest("left", { type: "self" }) + .nest("right", { type: "self" }), + 3: Parser.start() + .nest("one", { type: "self" }) + .nest("two", { type: "self" }) + .nest("three", { type: "self" }) + } + }); + + // 2 + // / \ + // 3 1 + // / | \ \ + // 1 0 2 0 + // / / \ + // 0 1 0 + // / + // 0 + + var buffer = Buffer.from([ + 2, + /* left -> */ 3, + /* one -> */ 1, + /* -> */ 0, + /* two -> */ 0, + /* three -> */ 2, + /* left -> */ 1, + /* -> */ 0, + /* right -> */ 0, + /* right -> */ 1, + /* -> */ 0 + ]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + type: 2, + data: { + left: { + type: 3, + data: { + one: { type: 1, data: { type: 0, data: {} } }, + two: { type: 0, data: {} }, + three: { + type: 2, + data: { + left: { type: 1, data: { type: 0, data: {} } }, + right: { type: 0, data: {} } + } + } + } + }, + right: { + type: 1, + data: { type: 0, data: {} } + } + } + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should be able to 'flatten' choices when using null varName", function() { + var parser = Parser.start() + .uint8("tag") + .choice(null, { + tag: "tag", + choices: { + 1: Parser.start() + .uint8("length") + .string("message", { length: "length" }), + 3: Parser.start().int32le("number") + } + }); + + var buffer = Buffer.from([ + 0x1, + 0xc, + 0x68, + 0x65, + 0x6c, + 0x6c, + 0x6f, + 0x2c, + 0x20, + 0x77, + 0x6f, + 0x72, + 0x6c, + 0x64 + ]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + tag: 1, + length: 12, + message: "hello, world" + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + buffer = Buffer.from([0x03, 0x4e, 0x61, 0xbc, 0x00]); + decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + tag: 3, + number: 12345678 + }); + encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should be able to 'flatten' choices when omitting varName paramater", function() { + var parser = Parser.start() + .uint8("tag") + .choice({ + tag: "tag", + choices: { + 1: Parser.start() + .uint8("length") + .string("message", { length: "length" }), + 3: Parser.start().int32le("number") + } + }); + + var buffer = Buffer.from([ + 0x1, + 0xc, + 0x68, + 0x65, + 0x6c, + 0x6c, + 0x6f, + 0x2c, + 0x20, + 0x77, + 0x6f, + 0x72, + 0x6c, + 0x64 + ]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + tag: 1, + length: 12, + message: "hello, world" + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + buffer = Buffer.from([0x03, 0x4e, 0x61, 0xbc, 0x00]); + decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + tag: 3, + number: 12345678 + }); + encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + it("should be able to use function as the choice selector", function() { + var parser = Parser.start() + .string("selector", { length: 4 }) + .choice(null, { + tag: function() { + return parseInt(this.selector, 2); // string base 2 to integer decimal + }, + choices: { + 2: Parser.start() + .uint8("length") + .string("message", { length: "length" }), + 7: Parser.start().int32le("number") + } + }); + + var buffer = Buffer.from([ + 48, + 48, + 49, + 48, + 0xc, + 0x68, + 0x65, + 0x6c, + 0x6c, + 0x6f, + 0x2c, + 0x20, + 0x77, + 0x6f, + 0x72, + 0x6c, + 0x64 + ]); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + selector: "0010", // -> choice 2 + length: 12, + message: "hello, world" + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + buffer = Buffer.from([48, 49, 49, 49, 0x4e, 0x61, 0xbc, 0x00]); + decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + selector: "0111", // -> choice 7 + number: 12345678 + }); + encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + }); + + describe("Nest parser", function() { + it("should encode nested parsers", function() { + var nameParser = new Parser() + .string("firstName", { + zeroTerminated: true + }) + .string("lastName", { + zeroTerminated: true + }); + var infoParser = new Parser().uint8("age"); + var personParser = new Parser() + .nest("name", { + type: nameParser + }) + .nest("info", { + type: infoParser + }); + + var buffer = Buffer.concat([ + Buffer.from("John\0Doe\0"), + Buffer.from([0x20]) + ]); + var person = personParser.parse(buffer); + assert.deepEqual(person, { + name: { + firstName: "John", + lastName: "Doe" + }, + info: { + age: 0x20 + } + }); + var encoded = personParser.encode(person); + assert.deepEqual(encoded, buffer); + }); + + it("should format parsed nested parser", function() { + var nameParser = new Parser() + .string("firstName", { + zeroTerminated: true + }) + .string("lastName", { + zeroTerminated: true + }); + var personParser = new Parser().nest("name", { + type: nameParser, + formatter: function(name) { + return name.firstName + " " + name.lastName; + }, + encoder: function(name) { + // Reverse of aboce formatter + var names = name.split(" "); + return { firstName: names[0], lastName: names[1] }; + } + }); + + var buffer = Buffer.from("John\0Doe\0"); + var person = personParser.parse(buffer); + assert.deepEqual(person, { + name: "John Doe" + }); + var encoded = personParser.encode(person); + assert.deepEqual(encoded, buffer); + }); + + it("should 'flatten' output when using null varName", function() { + var parser = new Parser() + .string("s1", { zeroTerminated: true }) + .nest(null, { + type: new Parser().string("s2", { zeroTerminated: true }) + }); + + var buf = Buffer.from("foo\0bar\0"); + var decoded = parser.parse(buf); + assert.deepEqual(decoded, { s1: "foo", s2: "bar" }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buf); + }); + + it("should 'flatten' output when omitting varName", function() { + var parser = new Parser().string("s1", { zeroTerminated: true }).nest({ + type: new Parser().string("s2", { zeroTerminated: true }) + }); + + var buf = Buffer.from("foo\0bar\0"); + var decoded = parser.parse(buf); + assert.deepEqual(decoded, { s1: "foo", s2: "bar" }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buf); + }); + }); + + describe("Buffer encoder", function() { + //this is a test for testing a fix of a bug, that removed the last byte of the + //buffer parser + it("should return a buffer with same size", function() { + var bufferParser = new Parser().buffer("buf", { + readUntil: "eof", + formatter: function(buffer) { + return buffer; + } + }); + + var buffer = Buffer.from("John\0Doe\0"); + var decoded = bufferParser.parse(buffer); + assert.deepEqual(decoded, { buf: buffer }); + var encoded = bufferParser.encode(decoded); + assert.deepEqual(encoded, buffer); + }); + }); + + describe("Constructors", function() { + it("should create a custom object type", function() { + function Person() { + this.name = ""; + } + Person.prototype.toString = function() { + return "[object Person]"; + }; + var parser = Parser.start() + .create(Person) + .string("name", { + zeroTerminated: true + }); + + var buffer = Buffer.from("John Doe\0"); + var person = parser.parse(buffer); + assert.ok(person instanceof Person); + assert.equal(person.name, "John Doe"); + var encoded = parser.encode(person); + assert.deepEqual(encoded, buffer); + }); + }); + + describe("encode other fields after bit", function() { + it("Encode uint8", function() { + var buffer = Buffer.from([0, 1, 0, 4]); + for (var i = 17; i <= 24; i++) { + var parser = Parser.start() + ["bit" + i]("a") + .uint8("b"); + var decoded = parser.parse(buffer); + assert.deepEqual(decoded, { + a: 1 << (i - 16), + b: 4 + }); + var encoded = parser.encode(decoded); + assert.deepEqual(encoded, buffer); + } + }); + }); +});