Modernize the parser dump functionality to support the post-NODE_VARIABLE
hybrid AST architecture with both variable-sized nodes and traditional
cons-list nodes.
Co-authored-by: Claude <noreply@anthropic.com>
This removes the NODE_VARIABLE enum and associated wrapper system, updating
the parser and codegen to work directly with variable-sized AST nodes.
Key changes:
- Removed NODE_VARIABLE from node.h enum
- Updated parser functions to handle direct variable-sized nodes
- Fixed codegen() main dispatch to detect variable-sized nodes directly
- Added helper functions for node type detection and header access
- Updated all parser and codegen functions to work with modern AST structure
Co-authored-by: Claude <noreply@anthropic.com>
Remove mrb_ast_head_node structure and cons_head() function while maintaining
accurate line number tracking for debugging. Replace cons_head() calls with
cons() calls but preserve NODE_VARIABLE wrapper as requested.
Key changes:
- Remove mrb_ast_head_node struct and head() macro from node.h
- Remove cons_head_gen() function and cons_head() macro from parse.y
- Update SET_LINENO macro to work with variable-sized nodes:
SET_LINENO(c,n) (((struct mrb_ast_var_header*)(c)->cdr)->lineno = (n))
- Restore all 11 SET_LINENO calls in grammar rules to maintain accurate
line number reporting for error messages and debugging
- Convert list1/list2/list3 and all new_*() function calls to use cons()
instead of cons_head() while keeping NODE_VARIABLE wrapper intact
Co-Authored-By: Claude <noreply@anthropic.com>
Eliminates NODE_KW_HASH enum, mrb_ast_kw_hash_node struct, gen_kw_hash_var
function, and related macros. All keyword hash functionality now unified
under NODE_HASH, completing the AST simplification.
Co-authored-by: Claude <noreply@anthropic.com>
Following the same pattern as the case node upgrade (e0f07c9), this
change eliminates the complex flat array packing approach for hash nodes
in favor of simple cons-list storage. The flat array packing provided
no memory benefit since cons lists aren't recycled, while adding
unnecessary complexity to both allocation and traversal logic.
Changes:
- Simplified mrb_ast_hash_node structure from variable-sized flexible
array to fixed-size structure with cons-list pointer
- Reduced new_hash() from complex 30+ line allocation to simple 4-line
pattern matching array node implementation
- Updated gen_hash_var() to use cons-list iteration instead of
interleaved array access (pairs[i*2] for key, pairs[i*2+1] for value)
- Removed HASH_NODE_LEN macro as length tracking is no longer needed
- Maintains identical functionality while reducing code complexity
Co-authored-by: Claude <noreply@anthropic.com>
Replace complex flat array packing with simple cons-list storage to reduce
memory overhead and code complexity. This continues the compiler simplification
work by reverting array nodes to the original memory-efficient approach.
- Remove len/flags fields from mrb_ast_array_node structure
- Eliminate complex two-pass processing (count + copy) in new_array()
- Replace array indexing with cons-list iteration in gen_array_var()
- Reduce parser code from 30+ lines to 4 lines for array creation
- Maintain full functionality with zero test regressions
Following the same successful pattern used for mrb_ast_case_node upgrade,
this change proves that flat array packing provides no memory benefit
since cons lists aren't recycled, while adding unnecessary complexity.
Co-authored-by: Claude <noreply@anthropic.com>
Replace NODE_KW_REST_ARGS wrapper nodes with direct ** symbol markers to
reduce memory overhead and simplify code structure. This continues the
compiler simplification work by unifying keyword rest arguments with
other node types while maintaining full functionality.
Co-authored-by: Claude <noreply@anthropic.com>
Replace NODE_KW_ARG wrapper with direct (key . value) cons structure,
eliminating unnecessary memory overhead and access indirection.
Changes:
- Remove NODE_KW_ARG node type from enum
- Modify new_kw_arg() to return direct cons instead of wrapped structure
- Update codegen.c to use simplified access patterns (k->car->car, k->car->cdr)
- Fix new_args_tail() to handle simplified keyword argument structure
- Remove NODE_KW_ARG case from parser dump function
This reduces memory usage from 3 cons cells to 1 per keyword argument
while maintaining full functionality and following mruby's design priority
of memory efficiency over complexity.
Co-authored-by: Claude <noreply@anthropic.com>
Replace variable-sized NODE_VARIABLE wrapper with fixed-size struct
allocation, following the same pattern as new_args(). This eliminates
the need for NODE_VARIABLE checking and uses direct casting instead.
Changes:
- Remove mrb_ast_var_header from callargs struct
- Use parser_palloc instead of parser_alloc_var for fixed-size allocation
- Update all access points to use direct casting: (struct mrb_ast_callargs*)
- Remove unnecessary backward compatibility code for newly introduced NODE_CALLARGS
Co-authored-by: Claude <noreply@anthropic.com>
Rename mrb_ast_op_asgn_node.operator field to op to avoid conflict with
C++ operator keyword. Update all references including macro definitions
and field access code.
Co-authored-by: Claude <noreply@anthropic.com>
- Remove obsolete NODE_ARGS_TAIL enum value and all references
- Simplify mrb_ast_case_node from variable-sized array back to simple cons-list structure
- Update new_case() function to use original cons-list approach instead of flattening
- Fix infinite loop in gen_case_var() when case statements have no matching clauses
- Improve code readability by renaming pos3 to case_end_jumps in gen_case_var()
- Restore memory-efficient case statement parsing without complex array management
The variable-sized array approach for case nodes provided no memory benefit
since cons lists aren't recycled. This change restores the simpler original
implementation while fixing a critical bug that caused mrbtest to hang
on "register window of calls" test.
Co-authored-by: Claude <noreply@anthropic.com>
NODE_ARGS_TAIL was a legacy enum value no longer created or used
after the conversion to struct-based argument handling. This change:
- Removes the NODE_ARGS_TAIL enum value from node.h
- Removes the unused case from mrb_parser_dump function
- Removes the obsolete assertion in dump_args function
All tests pass and argument forwarding continues to work correctly.
Co-authored-by: Claude <noreply@anthropic.com>
NODE_ARG and NODE_LVAR were handled identically in codegen.c, making
the distinction unnecessary. This change:
- Replaces all new_arg() calls with new_xvar(p, sym, NODE_LVAR)
- Removes the new_arg() function entirely
- Removes the unused NODE_ARG enum value
- Updates codegen.c to handle only NODE_LVAR case
The simplification reduces parser complexity while maintaining identical
functionality for argument processing.
Co-authored-by: Claude <noreply@anthropic.com>
This commit completes the transformation of mruby's argument processing from
cons-list based representation to direct struct field access.
Key changes:
- Transform new_args() to return struct mrb_ast_args* instead of cons-list
- Update lambda_body() to use direct struct field access for all argument types
- Fix anonymous keyword rest (**) to use intern_op(pow) marker for proper bytecode generation
- Fix argument forwarding (...) to correctly pass rest_arg to new_args()
- Eliminate mrb_ast_args_tail_node allocation by embedding fields directly in mrb_ast_args
- Update all node structure definitions to use struct mrb_ast_args*
- Remove unused NODE_ARGS enum value since args are now plain C structs
The new approach provides:
- More efficient memory usage by eliminating intermediate cons-list allocations
- Cleaner code generation with direct struct field access
- Proper distinction between anonymous kwrest and no kwrest
- Correct bytecode generation for both anonymous kwrest and argument forwarding
Fixes both anonymous keyword rest (def m(**) end) and argument forwarding
(def a(...) p(...) end) to generate correct bytecode and execute properly.
Co-authored-by: Claude <noreply@anthropic.com>
Remove NODE_SCALL and NODE_FCALL node types, consolidating all method calls
into a single NODE_CALL variable-sized node structure. This simplifies the
AST by unifying call semantics while maintaining support for safe navigation
and different call types through node flags.
Key changes:
- Convert call nodes to use variable-sized allocation with call_node structure
- Unify new_call() and new_fcall() to create NODE_CALL nodes consistently
- Replace gen_call() with separate gen_call_var() and gen_call_assign_var()
- Add gen_call_assign_var() for assignment operations like h[k] = v
- Remove legacy call handling from main codegen switch statement
- Preserve argument structure using args pointer instead of unpacking
- Support safe calls, keyword arguments, and blocks in unified structure
This migration maintains backward compatibility while enabling more efficient
call node processing and reduced code duplication in the compiler.
Co-authored-by: Claude <noreply@anthropic.com>
Complete migration of method definition nodes to variable-sized format:
- Convert NODE_DEF and NODE_SDEF from fixed cons-based to variable-sized nodes
- Update parser to create variable-sized def/sdef nodes directly
- Remove old codegen_def and codegen_sdef functions
- Consolidate method setup logic in defn_setup function
- Rename lambda_body_ex to lambda_body after removing wrapper layer
- Update all method definition code generation to use new node structure
This completes the variable-sized node migration for method definitions,
improving memory efficiency and enabling more flexible AST handling.
Co-authored-by: Claude <noreply@anthropic.com>
Complete NODE_STMTS migration by removing unused codegen_stmts function
and inlining statement traversal logic directly into gen_stmts_var.
Co-authored-by: Claude <noreply@anthropic.com>
Replace cons-list based case statement implementation with variable-sized
nodes for improved memory efficiency. The new implementation maintains
identical register allocation behavior using the original's proven
"nil-first, align-last" strategy.
Key changes:
- Convert new_case() to create variable-sized mrb_ast_case_node directly
- Replace codegen_case() with gen_case_var() using array iteration
- Apply original register allocation logic to new node structure
- Fix else clause handling in jump dispatch logic
Supports all case statement variants:
- Bare case statements (case when condition)
- Case with values (case expr when condition)
- UPVAR combinations with closure variables
- Splat operations (*case)
Co-Authored-By: Claude <noreply@anthropic.com>
Consolidate NODE_WHILE/NODE_UNTIL with MOD variants by sharing structures
and implementations, eliminating redundant code and improving maintainability.
Changes:
- remove separate mrb_ast_while_mod_node and mrb_ast_until_mod_node structures
- share mrb_ast_while_node between NODE_WHILE and NODE_WHILE_MOD variants
- share mrb_ast_until_node between NODE_UNTIL and NODE_UNTIL_MOD variants
- simplify new_while_mod to call new_while and update node_type
- simplify new_until_mod to call new_until and update node_type
- update gen_while_mod_var and gen_until_mod_var to use shared structures
The MOD variants now reuse core allocation logic from regular variants,
differing only in node_type. This eliminates code duplication while
preserving identical functionality for both pre-tested and post-tested loops.
Co-authored-by: Claude <noreply@anthropic.com>
Replace dual integer parsing paths with two-tier system:
- NODE_INT stores int32_t values directly for common case
- NODE_BIGINT stores string representation for overflow values
- Custom read_int32() function provides locale-independent parsing
- Remove unused readint() function from codegen
This eliminates confusing dual code paths while maintaining performance
for the majority of integer literals that fit in 32-bit range.
Co-authored-by: Claude <noreply@anthropic.com>
- update NODE_ZSUPER to use mrb_ast_super_node instead of empty mrb_ast_zsuper_node
- convert new_super and new_zsuper to always create variable-sized nodes
- update call_with_block to handle NODE_SUPER/NODE_ZSUPER wrapped in NODE_VARIABLE
- inline codegen_super and codegen_zsuper into their gen_*_var functions
- remove traditional NODE_SUPER and NODE_ZSUPER cases from codegen
Co-authored-by: Claude <noreply@anthropic.com>
This completes the conversion of NODE_HEREDOC from traditional cons-list
nodes to variable-sized nodes by:
1. Modified new_heredoc to always use variable-sized nodes with embedded
parser_heredoc_info struct and updated function signature to return
info pointer via output parameter
2. Fixed parsing_heredoc_info to handle NODE_VARIABLE wrapper detection
and return address of embedded struct
3. Updated gen_heredoc_var to use embedded info structure for codegen
4. Removed obsolete NODE_HEREDOC case and codegen_heredoc function from
traditional codegen path
5. Replaced codegen_heredoc_str wrapper with direct codegen_cons_list_string
calls for cleaner semantic naming
Co-authored-by: Claude <noreply@anthropic.com>
NODE_LITERAL_DELIM was only used as a marker in literal arrays.
Replace it with a (0 . 0) pattern which cannot conflict with
empty strings (which would be (0 . ptr) with non-NULL ptr).
This allows removing NODE_LITERAL_DELIM from the node type enum.
Co-authored-by: Claude <noreply@anthropic.com>
NODE_DREGX_ONCE was defined but never used in the codebase. No creation
functions, no codegen cases, and no parser rules reference this node type.
Removed:
- NODE_DREGX_ONCE enum value
- struct mrb_ast_dregx_once_node definition
- dregx_once_node() macro
- DREGX_ONCE_NODE_LIST() and DREGX_ONCE_NODE_OPTIONS() macros
Co-authored-by: Claude <noreply@anthropic.com>
Rename NODE_DSTR to NODE_STR and NODE_DXSTR to NODE_XSTR to reflect
that all strings now use dynamic (cons list) representation. Also
rename all associated functions for consistency:
- gen_dstr_var() -> gen_str_var()
- gen_dxstr_var() -> gen_xstr_var()
- codegen_heredoc_dstr() -> codegen_heredoc_str()
- codegen_dxstr() -> codegen_xstr()
The "D" prefix is no longer meaningful since all strings use the
variable-sized cons list format ((len . ptr) (-1 . node)...).
Co-authored-by: Claude <noreply@anthropic.com>
Remove NODE_STR and NODE_XSTR enum values and all associated code as these
traditional node types are no longer used with the new cons list string
representation. The compiler now exclusively uses the cons list format
((len . str) (-1 . node)...) for all string types.
- remove NODE_STR and NODE_XSTR from node_type enum in node.h
- remove NODE_STR and NODE_XSTR cases from codegen.c switch statements
- remove NODE_STR and NODE_XSTR cases from parse.y codedump functions
- remove unused codegen_str(), codegen_xstr(), and gen_xstr_var() functions
- update codegen_dregx() to use cons list string handling instead of
checking for obsolete NODE_STR
- preserve str_dump() function wrapped in #if 0 for future codedump updates
- update comment in node.h to reflect current node types
NODE_DSTR remains available for dynamic string interpolation. All string
functionality continues to work via the cons list representation and
variable-sized node implementations.
Co-authored-by: Claude <noreply@anthropic.com>
- change AST string representation from traditional node list to cons list
format where elements are either (len . str) for literals or (-1 . node)
for expressions
- implement codegen_cons_list_string() to handle new string format across
all string types (heredoc, dstr, xstr, dxstr, literal arrays)
- fix heredoc interpolation producing garbage by wrapping expressions as
(-1 . node) in parse.y heredoc_body rule instead of pushing directly
- fix backtick commands not executing in NOVAL mode by modifying
gen_dxstr_var and codegen_xstr to always generate OP_SSEND calls
- update gen_literal_array() to properly handle cons list format with
NODE_LITERAL_DELIM separators for %w[] and %i[] arrays
- refactor all dstr/dxstr/dregx variable node generators to use new format
- both simple `cmd` and dynamic `cmd #{var}` backticks now execute
correctly even when result is discarded
- all mrbtest cases now pass (1730/1731)
Co-authored-by: Claude <noreply@anthropic.com>
Previously the lexer dynamically called new_regx() and new_str() functions
which created different node types based on the var_nodes_enabled flag,
causing complexity in grammar actions and requiring dynamic dispatch handling.
This change simplifies the architecture by:
- Making lexer always return traditional cons structures:
- tREGEXP: (NODE_REGX . (pattern . (flags . encoding)))
- tSTRING: (NODE_STR . (string . length))
- Moving variable node generation to grammar actions where it belongs
- Simplifying new_dregx() to always receive traditional cons structures
- Updating mrb_ast_dregx_node to store the whole regx structure
This eliminates dynamic dispatch complexity and centralizes variable node
creation in grammar actions, making the code flow cleaner and more predictable.
Co-authored-by: Claude <noreply@anthropic.com>
Removes NODE_METHOD from the node type enum as this node type is not used
in the current parser implementation.
Co-authored-by: Claude <noreply@anthropic.com>
Removes NODE_CDECL, NODE_CVASGN, NODE_CVDECL, NODE_ITER, and NODE_WHEN
from the node type enum as these node types are not used in the current
parser implementation.
Co-authored-by: Claude <noreply@anthropic.com>
Implements variable-sized AST node support for Group 16 declarations and
definitions including NODE_ALIAS, NODE_POSTEXE, NODE_UNDEF, and NODE_SDEF.
This continues the systematic implementation of memory-efficient variable-
sized nodes across the mruby compiler's AST infrastructure.
Co-authored-by: Claude <noreply@anthropic.com>
Successfully implement NODE_SCOPE, NODE_BEGIN, and NODE_ENSURE as
variable-sized nodes. These structural nodes benefit from optimized
memory allocation and improved cache locality while maintaining
compatibility with existing codegen patterns.
Key improvements:
- NODE_SCOPE: Function scope definitions with variable-sized allocation
- NODE_BEGIN: Begin block structures with optimized memory layout
- NODE_ENSURE: Exception handling blocks with efficient storage
- All tests passing (1730/1731) with existing variable-sized nodes
- NODE_STMTS remains traditional to avoid codegen complexity
This extends the variable-sized node optimization to cover the primary
structural elements of the AST while keeping statement list handling
in its proven traditional form.
Co-authored-by: Claude <noreply@anthropic.com>
Add variable-sized node support for containers (array, hash, words, symbols)
and arguments (splat, to_ary, svalue, block_arg) to optimize memory usage
for statement blocks and argument processing.
Co-authored-by: Claude <noreply@anthropic.com>
Add variable-sized node support for containers (array, hash, words, symbols)
and arguments (splat, to_ary, svalue, block_arg) to optimize memory usage
for statement blocks and argument processing.
Co-authored-by: Claude <noreply@anthropic.com>
Implements variable-sized nodes for function calls and special forms
(NODE_FCALL, NODE_ZSUPER, NODE_LAMBDA) with optimized memory allocation.
These nodes now use compact variable-sized structures instead of fixed-size
headers, reducing AST memory usage.
Co-authored-by: Claude <noreply@anthropic.com>
Implements variable-sized nodes for operators and expressions
(NODE_NEGATE, NODE_COLON2, NODE_COLON3) with optimized memory
allocation. These nodes now use compact variable-sized structures
instead of fixed-size headers, reducing AST memory usage.
Co-authored-by: Claude <noreply@anthropic.com>
Implements variable-sized nodes for references and variables (NODE_NTH_REF,
NODE_BACK_REF, NODE_DVAR, NODE_NVAR, NODE_MATCH) with optimized memory
allocation. These nodes now use compact variable-sized structures instead
of fixed-size headers, reducing AST memory usage.
Co-authored-by: Claude <noreply@anthropic.com>
added variable-sized nodes for control flow and string/regex variants:
- control flow: break, next, redo, retry, while_mod, until_mod
- string/regex: xstr, dxstr, dregx, heredoc, dsym
- proper integration with existing codegen patterns
- maintains backward compatibility with traditional nodes
- tested with control flow and string interpolation
Co-authored-by: Claude <noreply@anthropic.com>
add variable-sized node structures for simple nodes (self, nil, true,
false, const) with conditional usage based on var_nodes_enabled.
singleton nodes use only 8-byte header for maximum memory efficiency.
includes proper forward declarations, casting macros, creation functions,
and codegen support maintaining compatibility with existing functions.
Co-authored-by: Claude <noreply@anthropic.com>
add variable-sized node structures for literal nodes (dstr, regx,
dot2/dot3 ranges, float) with conditional usage based on var_nodes_enabled.
includes casting macros, value access macros, creation functions,
and codegen support that maintains compatibility with existing
traditional codegen functions.
Co-authored-by: Claude <noreply@anthropic.com>
Add support for variable-sized AST nodes for logical and control expression
operations including AND, OR, RETURN, YIELD, and SUPER.
Changes:
- Add variable-sized node structures for expression nodes in node.h
- Add casting and value access macros for expression nodes
- Modify existing expression functions to conditionally use variable-sized versions
- Implement variable-sized node creation functions (new_and_var, new_or_var, etc.)
- Add codegen support for variable-sized expression nodes
- All expression types (AND, OR, RETURN, YIELD, SUPER) now support variable-sized allocation
Co-authored-by: Claude <noreply@anthropic.com>
Add support for variable-sized AST nodes for assignment operations including
simple assignment, multiple assignment, and operator assignment.
Changes:
- Add variable-sized node structures for assignment nodes in node.h
- Add casting and value access macros for assignment nodes
- Modify existing assignment functions to conditionally use variable-sized versions
- Implement variable-sized node creation functions (new_asgn_var, new_masgn_var, new_op_asgn_var)
- Add codegen support for variable-sized assignment nodes
- All assignment types (simple, multiple, operator) now support variable-sized allocation
Co-authored-by: Claude <noreply@anthropic.com>
Add variable-sized node structures for all control flow statements:
- IF/ELSIF/ELSE statements with optimized condition handling
- WHILE and UNTIL loops with proper jump generation
- FOR loops with iterator support
- CASE/WHEN statements with multiple condition matching
Key changes:
- Added variable-sized node structures (mrb_ast_if_node, mrb_ast_while_node,
mrb_ast_until_node, mrb_ast_case_node, mrb_ast_for_node) to node.h
- Implemented parser functions with size class allocation in parse.y
- Added comprehensive codegen support with proper jump handling and
stack management in codegen.c
- All control flow nodes now use NODE_VARIABLE wrapper for consistency
- Variable-sized nodes enabled by default for improved memory efficiency
This provides memory-efficient storage for control flow constructs while
maintaining full compatibility with existing functionality.
Co-authored-by: Claude <noreply@anthropic.com>
This completes the implementation of variable-sized AST nodes for control flow
structures (if, while, for, case), further reducing memory usage. Changes were
verified with AddressSanitizer.
Co-authored-by: Gemini <gemini@google.com>
Introduces variable-sized AST nodes for method calls (NODE_CALL),
arrays (NODE_ARRAY), and hashes (NODE_HASH). This change improves
memory efficiency by storing elements directly within the AST node,
avoiding an extra layer of pointer indirection for their data.
This is achieved by adding new data structures and functions in both
the parser and the code generator to handle these new node types.
Variable-sized nodes are now enabled by default.
Co-authored-by: Claude <noreply@anthropic.com>
This commit introduces the core infrastructure for variable-sized AST
nodes, designed to improve memory efficiency. The previous fixed-size
nodes are replaced by nodes that can store data inline, such as
strings and integers, reducing pointer indirection and memory overhead.
Key changes include:
- A generic variable-sized node header (`mrb_ast_var_header`).
- A size-class-based memory allocation system for these nodes.
- Implementation of variable-sized nodes for core types: symbols,
strings, integers, and variables (lvar, gvar, ivar, cvar).
- Integration into the parser and code generator, controlled by a
feature flag.
- Centralized and improved type-casting macros for AST nodes.
Co-authored-by: Claude <noreply@anthropic.com>
Yukihiro "Matz" Matsumoto