Repository navigation
Fix Lua 5.2 library contracts and bytecode interoperability - #395
Merged
Merged
Conversation
Invoke xpcall handlers before unwinding failing frames, preserve one handler result, and keep handler failures protected. Store the handler and its result in a stack slot so successful calls do not allocate extra state. Preserve cancellation and nested protected-call boundaries. Correct require caching and loader arguments, searchpath defaults and errors, rawset results, print's global tostring lookup, numeric assert messages, integer argument truncation, and multi-format file iterators. Implement load readers and string.dump with Lua 5.2 bytecode headers and representation-aware string serialization. Share numeric scanning across memory and stream readers to preserve trailing input and signed hex values. Add regression coverage and document bytecode and binary I/O compatibility.
Enable calls.lua and exercise load readers in files.lua directly. Run the binary dump and file-loading sections in byte mode, retaining UTF-16 text I/O adaptations. Replace hard-coded line patches with source section markers.
Drive suspension manually and return from a non-inlined execution helper before collecting. Check release once while keeping the completed coroutine and Lua state alive, without scheduler yields or GC polling.
Serialize Lua's constant tags independently of LuaValueType, and include the terminating byte in every string length, including empty strings. This lets native Lua load string constants without interpreting them as null strings. Add wire-format regressions and live NuLua Lua 5.2 interoperability checks to the CI verification command. Exchange dumps in both directions across the interpreter/compiler workloads, captured closures, varargs and binary strings.
Keep an outer xpcall handler pending while a C# CallAsync caller can catch the exception. Let Lua call sites handle errors that propagate back into Lua, and avoid dispatching handlers from RunAsync for host exceptions. Keep module searchers on the internal Lua call path so error handlers can still inspect failing Lua frames and locals. Check load(reader) modes after the first piece and reject Lua yields across reader calls. Use a nesting counter shared with error handlers, restore it on errors and cancellation, and retain support for asynchronous host reads. Keep source buffering and the synchronous compiler unchanged. Resolve print's tostring through the global metatable when the raw global is absent, while preserving its common direct lookup and per-call snapshot. Add regression coverage for synchronous and suspended host calls, nested reader yield boundaries, cancellation cleanup, first-piece mode rejection, searcher frame inspection, and suspended global tostring lookup. Known limitation: accepted reader input is still buffered to EOF before parsing. Early syntax errors and complete binary chunks do not stop reader callbacks as early as native Lua 5.2; a later reader error can take precedence. Validation: Release solution build; 2,861 runtime tests on each of .NET 8 and .NET 10; 73 source-generator tests; formatting; native interpreter and bytecode interoperability verification. Paired .NET 10 benchmarks show no additional allocations and ordinary reader, print, and public call timings within 1.5% of the preceding commit.
Keep enclosing xpcall handlers pending while C# callers can catch Lua exceptions from RunAsync, ExecuteAsync, and DoStringAsync. Share the runner with internal library calls, which retain the active handler so it can inspect failing frames before unwinding. Keep the ordinary public runner direct when no outer handler is installed. Reject Lua yields from print callbacks and module-loading callbacks while preserving asynchronous host awaits. Restore the nesting counter on errors and cancellation, and leave already-cached module returns outside the slow loading path. Honor package.loaded reads and writes through its metatable, including the default true result for loaders that return nil. Honor package and preload field lookups as well. Capture the package table in require and the Lua file searcher through existing CSharpClosure upvalues so global reassignment and library reopening do not change saved functions or their debug upvalues. Require a mask when installing a debug hook, and ignore unused mask/count arguments when disabling one. Add regression coverage for all five issues in both string modes, including suspended host calls, failing-frame inspection, package replacement and reopening, direct searcher calls, metatable callbacks, yield restoration, and cancellation cleanup. Validation: Release solution build across all targets; 2,925 runtime tests on each of .NET 8 and .NET 10 plus focused direct-searcher regressions; 73 source-generator tests; formatting; native Lua 5.2 behavior comparisons and interpreter/bytecode interoperability. Paired .NET 10 benchmarks add no steady-state allocations. Ordinary print, reader, and public call paths remain near baseline, with no cached-require slowdown in the final comparison. A synthetic repeatedly uncached preload loop is about 8% slower, and protected RunAsync reentry adds about 12 ns per call for preserving the enclosing host exception boundary.
Keep an enclosing xpcall handler pending while C# callers can catch errors from public indexing, assignment, arithmetic, comparison, and concatenation APIs. Apply the host boundary in the shared callback helpers, leaving the primitive fast paths and VM instruction dispatch unchanged. Keep internal module table access and default sort comparisons on the Lua call path. Dispatch active handlers before shared callback helpers remove failing managed frames, including concatenation callbacks. Cancellation still escapes without invoking a Lua handler. Add synchronous and suspended regressions for every affected public operation, caught and propagated failures, and handler frame inspection through module caches, print, sorting, gsub, and concatenation. Validation: Release build across all targets, full .NET 8/.NET 10 test suite, native Lua 5.2 comparisons, formatting, and native interoperability checks. Paired benchmarks add no steady-state allocations. Primitive operations, numeric loops, cached require, and sorting show no slowdown; host setter callbacks cost approximately 3-4 ns more per call.
Decode an omitted bytecode source as Lua 5.2's unknown source marker while preserving an actual empty source name. Treat absent line information as unknown in error messages and tracebacks instead of inserting an internal Trace back error diagnostic. Remove console output for missing line data. Add a genuine Lua 5.2.4 luac -s fixture covering pcall error values, xpcall tracebacks, source metadata, redumping, and the distinction between omitted and empty source names in both string modes. Validation: Release build across all targets, focused and full .NET 8/.NET 10 tests, native Lua 5.2 comparisons using the same compiler-produced fixture, native bytecode interoperability checks, and formatting.
Report CSharpClosure upvalue counts through debug.getinfo and provide stable slot identities through debug.upvalueid. Key identities by the backing upvalue array in a ConditionalWeakTable so equal values in different slots remain distinct and retained IDs do not keep captured values alive. Add regressions for metadata, repeated identity queries, distinct slots and closures, setupvalue preserving identity, standard package closures, and collection of discarded managed captures while an ID remains retained. Validation: 3,023 runtime tests on each of .NET 8 and .NET 10, 73 generator tests, Release build across all targets, native Lua 5.2 behavior comparisons, native interoperability checks, and formatting.
Keep table-library length metamethods inside the active Lua error boundary so xpcall handlers can inspect and modify failing frames. Require actual functions as xpcall error handlers rather than dispatching __call. Reject Lua yields across the remaining standard-library callback boundaries while preserving C# awaits and restoring state after cancellation. Scope the sort guard around the sort operation rather than each comparison. Share debug upvalue-index validation and reject managed closures in upvaluejoin with Lua 5.2 argument errors. Retain weak upvalue identities. Add regressions for all affected callbacks, table length error frames, handler validation and invalid upvalue requests. Validation: Release build (all targets), 3,123 runtime tests per net8.0 and net10.0, 73 generator tests, formatting, native Lua 5.2.4 repro comparisons, and bytecode interoperability verification. Paired benchmarks show unchanged allocations and timing changes within 2.2% across the measured workloads.
Register _G, package and coroutine in package.loaded when their libraries are opened so require returns the existing library tables. Preserve supplied messages when formatting saved coroutine tracebacks and tracebacks with no active frames. Use the normal gethook metadata result for disabled hooks so callers receive nil, an empty mask and zero count. Reject Lua yields from Lua-installed debug hooks, including callbacks that disable their own hooks. Capture the hook policy before invocation and restore the existing non-yieldable counter on every exit. Keep C# awaits and host-installed hook yields supported without adding instruction-loop checks or callback wrappers. Add regressions for library registration and reopening, coroutine traceback messages, disabled hook metadata, all Lua hook events, and host async/yield compatibility. Validation: Release build across all targets; 3,157 runtime tests on each of net8.0 and net10.0; 73 generator tests; final targeted hook tests on both runtimes; formatting; native Lua 5.2.4 comparisons in both string modes. Paired hook benchmarks show unchanged allocations and timing changes from -1.8% to +3.3% across the measured workloads.
Reject non-string/non-number filenames and closed handles in io.input and io.output before opening files, flushing output or changing the default stream. Preserve numeric filename coercion and open-handle selection. Add regression coverage in both string modes using a recording filesystem and output stream, avoiding real filesystem side effects for invalid input. Validation: regression tests and full runtime suite on net8.0 and net10.0; Release build across all targets; formatting; native Lua 5.2.4 comparisons.
Create temporary files with unique names, CreateNew and DeleteOnClose so io.tmpfile does not leave files behind after close or process exit. Avoid creating a separate persistent placeholder file. Add a deterministic deletion-on-close regression and remove ineffective cleanup code from the existing temporary-file round-trip test. Validation: I/O tests and full runtime suite on net8.0 and net10.0; Release build across all targets; formatting; isolated cleanup repro.
Detect a UTF-8 BOM only at the start of text input, including short reads and buffered EOF probes. Keep embedded BOM characters as ordinary data. Decode counted UTF-16 reads with enough room for a surrogate pair, retaining one pending character across subsequent reads and format changes. Flush the decoder at EOF and reset it after successful seeks. Position queries still synchronize the underlying stream before writes while preserving pending text. Leave failed seeks untouched. Separate byte reads from UTF-16 decoding and run BOM initialization only when needed. Share buffer refills and keep additional decoder state to a single pending character. Add regressions for buffer boundaries, short reads, supplementary characters, EOF replacement fallback, format changes, seek state and read-seek-write sequences in both string modes. Validation: Release build across all targets; 3,252 runtime tests per net8.0 and net10.0; 73 generator tests; formatting; native Lua 5.2.4 comparisons and StreamReader comparisons. Paired benchmarks show unchanged allocations, byte reads within about 1%, UTF-16 single-character reads 45-47% faster, and UTF-16 bulk reads about 3-4% slower across the measured workloads.
Distinguish omitted io.close arguments from explicit nil, reject closed file line iterators when they are created, and return access-denied failures from io.open. Reuse the existing open helper's error path. Add regression coverage in UTF-16 and byte-string modes.
Keep incomplete UTF-8 input in the byte buffer so format changes, raw reads, EOF probes, and seeks share the same unread-byte accounting. Flush truncated input at EOF and keep the single-character ASCII path allocation-free apart from the returned string. Observe read cancellation at entry and before buffer fills, without adding checks to character or numeric-scanning loops. Add deterministic regressions for format changes, seek positions, EOF, and cancellation. Validation: Release build for all targets; 3,302 runtime tests on each of .NET 8 and .NET 10; 73 source-generator tests; formatting and native Lua 5.2 comparisons. Stream benchmark allocations are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lua 5.2 scripts could fail when loading reader-produced chunks, exchanging bytecode with native Lua, or handling errors through callbacks. Buffered UTF-8 reads could also lose characters when switching formats or querying the file position. This change fixes these contracts and adds regression coverage in UTF-16 and byte-string modes.
Changes
loadandstring.dump; correct the Lua 5.2 bytecode layout, constant tags, empty strings, string-mode handling, and stripped debug metadata.xpcallhandlers before unwinding failing frames, preserve host exception boundaries across metamethods, and enforce Lua 5.2 callback yield restrictions while retaining C# async support.io.close(nil)from an omitted argument, reject closed-file line iterators immediately, return access-denied open failures, and delete temporary files on close.Bytecode compatibility change
Regenerate older Lua-CSharp bytecode from source. Previously generated chunks used an incorrect function-header field order and string constant tag. Serialization now follows the C Lua 5.2 layout; older chunks using the previous format are incompatible.
Prototype.StringModepreserves the representation when dumping.Prototype.FromBytecode(bytes, name, stringMode)selects it when loading without a state;LuaState.Loaduses the state's mode.string.dumpreturns raw bytes in both string modes and serializes the prototype without captured upvalue values. Save it through byte-mode Lua streams orLuaByteString.ToBytesin C#. UTF-16 streams encode text as UTF-8 and cannot preserve arbitrary binary strings. The compatibility guidance is documented in both READMEs.Performance against main
Compared
3fbaaea(main) with0f650c3on Windows x64 / .NET 10.0.14, using isolated assembly contexts in one process, one CPU affinity, tiered compilation disabled, and nine alternating 200 ms samples after warmup. Results below are median paired changes in time per operation; each operation reads the full test input.Measured allocations are unchanged across all 14 stream workloads. These comparisons use main as the baseline, rather than an intermediate branch commit.
Validation
TestCategory!=ExpectedFailurefilter.git diff --checkpassed.