- update NODE_ZSUPER to use mrb_ast_super_node instead of empty mrb_ast_zsuper_node
- convert new_super and new_zsuper to always create variable-sized nodes
- update call_with_block to handle NODE_SUPER/NODE_ZSUPER wrapped in NODE_VARIABLE
- inline codegen_super and codegen_zsuper into their gen_*_var functions
- remove traditional NODE_SUPER and NODE_ZSUPER cases from codegen
Co-authored-by: Claude <noreply@anthropic.com>
Remove var_nodes_enabled conditions from new_dot2 and new_dot3 functions
and inline variable-sized node creation logic directly. Clean up obsolete
codegen paths by removing case NODE_DOT2 and NODE_DOT3 from traditional
codegen() and removing unused codegen_dot2 and codegen_dot3 functions.
Update gen_dot2_var and gen_dot3_var to use proper DOT2/DOT3_NODE macros
and generate OP_RANGE_INC/EXC instructions directly.
Co-authored-by: Claude <noreply@anthropic.com>
Remove var_nodes_enabled condition from new_sym function and inline
new_sym_var directly. Clean up obsolete codegen paths by removing
case NODE_SYM from traditional codegen() and inlining codegen_sym
into variable-sized node handler. Remove unused new_sym_original
helper function.
Co-authored-by: Claude <noreply@anthropic.com>
Complete the conversion of boolean literal nodes by:
1. Convert new_true to always use variable-sized nodes and inline new_true_var
directly into the function, eliminating function call overhead
2. Remove obsolete NODE_TRUE case from traditional codegen() and inline
codegen_true function into gen_true_var for cleaner code
3. Apply the same optimizations to new_false - inline new_false_var and
remove obsolete NODE_FALSE case and codegen_false function
4. Clean up unused functions and forward declarations
Both true and false literals now always use the variable-sized node path
with direct OP_LOADT/OP_LOADF instruction generation, eliminating
conditional branching and function call overhead.
Co-authored-by: Claude <noreply@anthropic.com>
Temporarily revert new_call to avoid issues with assignment to method calls
like self[idx] = value causing "unknown lhs" errors. The function now always
uses traditional cons-list NODE_CALL/NODE_SCALL nodes instead of variable-sized
nodes to maintain compatibility with existing assignment codegen.
Co-authored-by: Claude <noreply@anthropic.com>
This completes the conversion of NODE_HEREDOC from traditional cons-list
nodes to variable-sized nodes by:
1. Modified new_heredoc to always use variable-sized nodes with embedded
parser_heredoc_info struct and updated function signature to return
info pointer via output parameter
2. Fixed parsing_heredoc_info to handle NODE_VARIABLE wrapper detection
and return address of embedded struct
3. Updated gen_heredoc_var to use embedded info structure for codegen
4. Removed obsolete NODE_HEREDOC case and codegen_heredoc function from
traditional codegen path
5. Replaced codegen_heredoc_str wrapper with direct codegen_cons_list_string
calls for cleaner semantic naming
Co-authored-by: Claude <noreply@anthropic.com>
Remove var_nodes_enabled condition from new_dsym function, completing the
transition to variable-sized nodes for dynamic symbol processing.
Fix gen_dsym_var function to properly extract the dsym node using the
dsym_node() macro and simplify the codegen pattern to match traditional
codegen_dsym behavior.
Remove unused codegen_dsym function and its corresponding NODE_DSYM case
from the main codegen switch, cleaning up dead traditional codegen paths.
Co-authored-by: Claude <noreply@anthropic.com>
Remove unused codegen_words and codegen_symbols functions along with their
corresponding cases in the main codegen switch. These became dead code
after converting new_words and new_symbols to always use variable-sized nodes.
Also remove var_nodes_enabled conditions from new_words and new_symbols,
completing the transition to always using variable-sized nodes for word and
symbol arrays.
Co-authored-by: Claude <noreply@anthropic.com>
Add helper functions to simplify string representation creation in cons format:
- new_str_rep(p, str, len): creates cons(length, string_ptr)
- new_str_tok(p): creates string representation from current token
- new_str_empty(p): creates empty string representation
This reduces code duplication and improves readability by replacing
verbose patterns like cons(int_to_node(toklen(p)), (node*)strndup(...))
with cleaner helper function calls.
Co-authored-by: Claude <noreply@anthropic.com>
NODE_LITERAL_DELIM was only used as a marker in literal arrays.
Replace it with a (0 . 0) pattern which cannot conflict with
empty strings (which would be (0 . ptr) with non-NULL ptr).
This allows removing NODE_LITERAL_DELIM from the node type enum.
Co-authored-by: Claude <noreply@anthropic.com>
NODE_DREGX_ONCE was defined but never used in the codebase. No creation
functions, no codegen cases, and no parser rules reference this node type.
Removed:
- NODE_DREGX_ONCE enum value
- struct mrb_ast_dregx_once_node definition
- dregx_once_node() macro
- DREGX_ONCE_NODE_LIST() and DREGX_ONCE_NODE_OPTIONS() macros
Co-authored-by: Claude <noreply@anthropic.com>
Rename NODE_DSTR to NODE_STR and NODE_DXSTR to NODE_XSTR to reflect
that all strings now use dynamic (cons list) representation. Also
rename all associated functions for consistency:
- gen_dstr_var() -> gen_str_var()
- gen_dxstr_var() -> gen_xstr_var()
- codegen_heredoc_dstr() -> codegen_heredoc_str()
- codegen_dxstr() -> codegen_xstr()
The "D" prefix is no longer meaningful since all strings use the
variable-sized cons list format ((len . ptr) (-1 . node)...).
Co-authored-by: Claude <noreply@anthropic.com>
Remove NODE_STR and NODE_XSTR enum values and all associated code as these
traditional node types are no longer used with the new cons list string
representation. The compiler now exclusively uses the cons list format
((len . str) (-1 . node)...) for all string types.
- remove NODE_STR and NODE_XSTR from node_type enum in node.h
- remove NODE_STR and NODE_XSTR cases from codegen.c switch statements
- remove NODE_STR and NODE_XSTR cases from parse.y codedump functions
- remove unused codegen_str(), codegen_xstr(), and gen_xstr_var() functions
- update codegen_dregx() to use cons list string handling instead of
checking for obsolete NODE_STR
- preserve str_dump() function wrapped in #if 0 for future codedump updates
- update comment in node.h to reflect current node types
NODE_DSTR remains available for dynamic string interpolation. All string
functionality continues to work via the cons list representation and
variable-sized node implementations.
Co-authored-by: Claude <noreply@anthropic.com>
- change AST string representation from traditional node list to cons list
format where elements are either (len . str) for literals or (-1 . node)
for expressions
- implement codegen_cons_list_string() to handle new string format across
all string types (heredoc, dstr, xstr, dxstr, literal arrays)
- fix heredoc interpolation producing garbage by wrapping expressions as
(-1 . node) in parse.y heredoc_body rule instead of pushing directly
- fix backtick commands not executing in NOVAL mode by modifying
gen_dxstr_var and codegen_xstr to always generate OP_SSEND calls
- update gen_literal_array() to properly handle cons list format with
NODE_LITERAL_DELIM separators for %w[] and %i[] arrays
- refactor all dstr/dxstr/dregx variable node generators to use new format
- both simple `cmd` and dynamic `cmd #{var}` backticks now execute
correctly even when result is discarded
- all mrbtest cases now pass (1730/1731)
Co-authored-by: Claude <noreply@anthropic.com>
Previously the lexer dynamically called new_regx() and new_str() functions
which created different node types based on the var_nodes_enabled flag,
causing complexity in grammar actions and requiring dynamic dispatch handling.
This change simplifies the architecture by:
- Making lexer always return traditional cons structures:
- tREGEXP: (NODE_REGX . (pattern . (flags . encoding)))
- tSTRING: (NODE_STR . (string . length))
- Moving variable node generation to grammar actions where it belongs
- Simplifying new_dregx() to always receive traditional cons structures
- Updating mrb_ast_dregx_node to store the whole regx structure
This eliminates dynamic dispatch complexity and centralizes variable node
creation in grammar actions, making the code flow cleaner and more predictable.
Co-authored-by: Claude <noreply@anthropic.com>
Remove if (!p->var_nodes_enabled) branch from new_nth_ref function
to use variable-sized nodes exclusively for numbered regex references.
Co-authored-by: Claude <noreply@anthropic.com>
Remove if (!p->var_nodes_enabled) branch from new_back_ref function
to use variable-sized nodes exclusively for regex backreferences.
Co-authored-by: Claude <noreply@anthropic.com>
Remove if (!p->var_nodes_enabled) branch from new_dxstr function
to use variable-sized nodes exclusively for dynamic execution strings.
Co-authored-by: Claude <noreply@anthropic.com>
- Modified new_undef to accept node *syms list instead of single mrb_sym
- Simplified gen_undef_var to directly pass symbol list
- Removed traditional node generation path from new_negate
- Both functions now use variable-sized nodes exclusively
Co-authored-by: Claude <noreply@anthropic.com>
- Update new_undef function signature to accept node list instead of single symbol
- Fix grammar rule to properly construct undef nodes from symbol lists
- Simplify gen_undef_var function to directly pass symbol list to codegen
- Support multiple symbols in single undef statement (e.g., undef foo, bar)
Co-Authored-By: Claude <noreply@anthropic.com>
This removes the legacy `cons` node creation path from several `new_*`
functions, forcing them to use the variable-sized node implementation.
This is a step towards simplifying the parser and unifying the AST
representation.
Co-authored-by: Claude <noreply@anthropic.com>
Removes NODE_METHOD from the node type enum as this node type is not used
in the current parser implementation.
Co-authored-by: Claude <noreply@anthropic.com>
Removes NODE_CDECL, NODE_CVASGN, NODE_CVDECL, NODE_ITER, and NODE_WHEN
from the node type enum as these node types are not used in the current
parser implementation.
Co-authored-by: Claude <noreply@anthropic.com>
Implements variable-sized AST node support for Group 16 declarations and
definitions including NODE_ALIAS, NODE_POSTEXE, NODE_UNDEF, and NODE_SDEF.
This continues the systematic implementation of memory-efficient variable-
sized nodes across the mruby compiler's AST infrastructure.
Co-authored-by: Claude <noreply@anthropic.com>
Successfully implement NODE_SCOPE, NODE_BEGIN, and NODE_ENSURE as
variable-sized nodes. These structural nodes benefit from optimized
memory allocation and improved cache locality while maintaining
compatibility with existing codegen patterns.
Key improvements:
- NODE_SCOPE: Function scope definitions with variable-sized allocation
- NODE_BEGIN: Begin block structures with optimized memory layout
- NODE_ENSURE: Exception handling blocks with efficient storage
- All tests passing (1730/1731) with existing variable-sized nodes
- NODE_STMTS remains traditional to avoid codegen complexity
This extends the variable-sized node optimization to cover the primary
structural elements of the AST while keeping statement list handling
in its proven traditional form.
Co-authored-by: Claude <noreply@anthropic.com>
Add variable-sized node support for containers (array, hash, words, symbols)
and arguments (splat, to_ary, svalue, block_arg) to optimize memory usage
for statement blocks and argument processing.
Co-authored-by: Claude <noreply@anthropic.com>
Add variable-sized node support for containers (array, hash, words, symbols)
and arguments (splat, to_ary, svalue, block_arg) to optimize memory usage
for statement blocks and argument processing.
Co-authored-by: Claude <noreply@anthropic.com>
Implements variable-sized nodes for function calls and special forms
(NODE_FCALL, NODE_ZSUPER, NODE_LAMBDA) with optimized memory allocation.
These nodes now use compact variable-sized structures instead of fixed-size
headers, reducing AST memory usage.
Co-authored-by: Claude <noreply@anthropic.com>
Implements variable-sized nodes for operators and expressions
(NODE_NEGATE, NODE_COLON2, NODE_COLON3) with optimized memory
allocation. These nodes now use compact variable-sized structures
instead of fixed-size headers, reducing AST memory usage.
Co-authored-by: Claude <noreply@anthropic.com>
Implements variable-sized nodes for references and variables (NODE_NTH_REF,
NODE_BACK_REF, NODE_DVAR, NODE_NVAR, NODE_MATCH) with optimized memory
allocation. These nodes now use compact variable-sized structures instead
of fixed-size headers, reducing AST memory usage.
Co-authored-by: Claude <noreply@anthropic.com>
added variable-sized nodes for control flow and string/regex variants:
- control flow: break, next, redo, retry, while_mod, until_mod
- string/regex: xstr, dxstr, dregx, heredoc, dsym
- proper integration with existing codegen patterns
- maintains backward compatibility with traditional nodes
- tested with control flow and string interpolation
Co-authored-by: Claude <noreply@anthropic.com>
add variable-sized node structures for simple nodes (self, nil, true,
false, const) with conditional usage based on var_nodes_enabled.
singleton nodes use only 8-byte header for maximum memory efficiency.
includes proper forward declarations, casting macros, creation functions,
and codegen support maintaining compatibility with existing functions.
Co-authored-by: Claude <noreply@anthropic.com>
add variable-sized node structures for literal nodes (dstr, regx,
dot2/dot3 ranges, float) with conditional usage based on var_nodes_enabled.
includes casting macros, value access macros, creation functions,
and codegen support that maintains compatibility with existing
traditional codegen functions.
Co-authored-by: Claude <noreply@anthropic.com>
Add support for variable-sized AST nodes for logical and control expression
operations including AND, OR, RETURN, YIELD, and SUPER.
Changes:
- Add variable-sized node structures for expression nodes in node.h
- Add casting and value access macros for expression nodes
- Modify existing expression functions to conditionally use variable-sized versions
- Implement variable-sized node creation functions (new_and_var, new_or_var, etc.)
- Add codegen support for variable-sized expression nodes
- All expression types (AND, OR, RETURN, YIELD, SUPER) now support variable-sized allocation
Co-authored-by: Claude <noreply@anthropic.com>
Add support for variable-sized AST nodes for assignment operations including
simple assignment, multiple assignment, and operator assignment.
Changes:
- Add variable-sized node structures for assignment nodes in node.h
- Add casting and value access macros for assignment nodes
- Modify existing assignment functions to conditionally use variable-sized versions
- Implement variable-sized node creation functions (new_asgn_var, new_masgn_var, new_op_asgn_var)
- Add codegen support for variable-sized assignment nodes
- All assignment types (simple, multiple, operator) now support variable-sized allocation
Co-authored-by: Claude <noreply@anthropic.com>
Add variable-sized node structures for all control flow statements:
- IF/ELSIF/ELSE statements with optimized condition handling
- WHILE and UNTIL loops with proper jump generation
- FOR loops with iterator support
- CASE/WHEN statements with multiple condition matching
Key changes:
- Added variable-sized node structures (mrb_ast_if_node, mrb_ast_while_node,
mrb_ast_until_node, mrb_ast_case_node, mrb_ast_for_node) to node.h
- Implemented parser functions with size class allocation in parse.y
- Added comprehensive codegen support with proper jump handling and
stack management in codegen.c
- All control flow nodes now use NODE_VARIABLE wrapper for consistency
- Variable-sized nodes enabled by default for improved memory efficiency
This provides memory-efficient storage for control flow constructs while
maintaining full compatibility with existing functionality.
Co-authored-by: Claude <noreply@anthropic.com>
This completes the implementation of variable-sized AST nodes for control flow
structures (if, while, for, case), further reducing memory usage. Changes were
verified with AddressSanitizer.
Co-authored-by: Gemini <gemini@google.com>
Introduces variable-sized AST nodes for method calls (NODE_CALL),
arrays (NODE_ARRAY), and hashes (NODE_HASH). This change improves
memory efficiency by storing elements directly within the AST node,
avoiding an extra layer of pointer indirection for their data.
This is achieved by adding new data structures and functions in both
the parser and the code generator to handle these new node types.
Variable-sized nodes are now enabled by default.
Co-authored-by: Claude <noreply@anthropic.com>
This commit introduces the core infrastructure for variable-sized AST
nodes, designed to improve memory efficiency. The previous fixed-size
nodes are replaced by nodes that can store data inline, such as
strings and integers, reducing pointer indirection and memory overhead.
Key changes include:
- A generic variable-sized node header (`mrb_ast_var_header`).
- A size-class-based memory allocation system for these nodes.
- Implementation of variable-sized nodes for core types: symbols,
strings, integers, and variables (lvar, gvar, ivar, cvar).
- Integration into the parser and code generator, controlled by a
feature flag.
- Centralized and improved type-casting macros for AST nodes.
Co-authored-by: Claude <noreply@anthropic.com>
This implements a memory optimization for AST nodes that stores location
information (lineno, filename_index) only in head nodes rather than in
every node, reducing memory usage for structure nodes.
Key changes:
- Split node types: mrb_ast_node (structure nodes without location),
mrb_ast_head_node (with location info). Sizes are platform-dependent:
8/12 bytes on 32-bit, 16/24 bytes on 64-bit platforms
- Separate allocation: cons() creates structure nodes, cons_head()
creates head nodes with location information
- Node recycling: all nodes are recycled when freed, but only smaller
structure nodes are reused from the free list to maintain type safety
- Updated macro: added headn() for consistent head node casting
- Removed NODE_LINENO macro: eliminated redundant location copying
since head-only optimization already provides adequate location info
- Fixed codegen to properly access location fields via head node casts
This optimization reduces AST memory usage while preserving all
debugging and location information functionality.
Co-authored-by: Claude <noreply@anthropic.com>
Add helper functions to reduce code duplication in codegen load operations:
- gen_load_op1/gen_load_op2: for simple literal load operations following
the pattern "if (!val) return; genop_X(...); push();"
- gen_load_nil: for conditional nil loading with "if (!val) return;" check
- gen_load_lit: for literal loading with push
Refactor 8 literal loading functions (codegen_self, codegen_nil, codegen_true,
codegen_false, codegen_sym, codegen_float, codegen_back_ref, codegen_nth_ref)
and multiple inline nil loading patterns throughout codegen.c.
Each refactored function reduced from 5-8 lines to 2-4 lines while maintaining
identical bytecode generation behavior. All 1730 tests pass.
Co-authored-by: Claude <noreply@anthropic.com>
Extract final complex cases (NODE_OP_ASGN, NODE_MASGN), unify while/until
loop handling, apply early return pattern to reduce indentation, and achieve
complete switch statement consistency.
The original 5000+ line monolithic function is now organized into 60+ focused
functions while preserving all functionality and performance.
Co-Authored-By: Claude <noreply@anthropic.com>
Change int variables to mrb_int in mrb_dir_getwd and mrb_dir_chroot
to maintain consistent use of mruby's integer type internally.
Keep explicit casts only at system interface boundaries where
different types are required by system calls.
Eliminates VC warning C4267 while following the same type
unification approach used in pack.c.
Co-authored-by: Claude <noreply@anthropic.com>
Change count variables from int to mrb_int in mrb_pack_pack and
read_tmpl functions to eliminate mixed type usage and resolve
VC warning C4244 about conversion from mrb_int to int.
Co-authored-by: Claude <noreply@anthropic.com>
This issue was originally discovered by OSS-Fuzz:
https://issues.oss-fuzz.com/issues/428404023
The root cause was that str_strip_bang modified the string content and
length in-place but failed to null-terminate the string at its new
length.
When this modified, non-null-terminated string was duplicated, the
buffer may be resized, dropping the old null terminator (via str_uminus
-> mrb_str_dup -> str_replace -> str_share). When this is later passed
to mrb_raisef using the %!s format specifier, mrb_vformat called strlen
on the underlying non-null terminated buffer pointer.
The fix adds explicit null-termination in str_strip_bang,
str_lstrip_bang, and str_rstrip_bang after the string length is updated.
Yukihiro "Matz" Matsumoto