mruby-mruby/mrbgems/mruby-task/README.md

# mruby-task

mruby-task is an mrbgem that provides cooperative multitasking with preemptive
scheduling for mruby. It enables concurrent execution of multiple tasks within
a single mruby VM instance using a priority-based scheduler with tick-based
time slicing.

## Purpose

The primary purpose of `mruby-task` is to enable mruby applications to:

- Execute multiple tasks concurrently within a single VM.
- Schedule tasks based on priority (0-255, where 0 is highest priority).
- Provide cooperative yielding with `Task.pass`.
- Support preemptive scheduling via timer-based interrupts.
- Synchronize tasks using `sleep`, `join` and `Task::Queue`.
- Suspend and resume tasks programmatically.
- Coordinate producers and consumers via `Task::Queue` without polling.

## Architecture

### Task Scheduler

The scheduler uses four priority-sorted queues:

- **DORMANT**: Tasks that have not started or have finished execution.
- **READY**: Tasks ready to run, ordered by priority.
- **WAITING**: Tasks waiting for sleep timeout or join completion.
- **SUSPENDED**: Tasks manually suspended via `#suspend`.

### Tick-Based Preemption

A platform-specific timer generates periodic ticks (default: 4ms). Each task
receives a timeslice (default: 3 ticks = 12ms) before being preempted. The
scheduler automatically switches to the next ready task when:

- A task's timeslice expires.
- A task calls `sleep`, `Task.pass`, or `join`.
- A task finishes execution.

## Functionality

### Creating Tasks

Tasks are created with `Task.new` and begin execution immediately:

```ruby
# Create a task with default priority (128)
task = Task.new do
  puts "Hello from task!"
  sleep 1
  puts "Task resumed"
end

# Create a named task with custom priority
task = Task.new(name: "worker", priority: 64) do
  loop do
    process_data
    Task.pass  # Yield to other tasks
  end
end

# Start the scheduler (blocks until all tasks complete or idle)
Task.run
```

### Task Class Methods

- **`Task.new(name: nil, priority: 128) { block }`**: Creates and starts a new task. Lower priority values run first (0 is highest priority). The `name` parameter must be a String if provided. The `priority` must be an Integer between 0-255.

  ```ruby
  task = Task.new(name: "background", priority: 200) do
    # Task code here
  end
  ```

- **`Task.current`**: Returns the currently executing task.

  ```ruby
  current = Task.current
  puts "Running: #{current.name}"
  ```

- **`Task.list`**: Returns an array of all tasks (including dormant tasks).

  ```ruby
  Task.list.each do |task|
    puts "#{task.name}: #{task.status}"
  end
  ```

- **`Task.pass`**: Cooperatively yields execution to other ready tasks.

  ```ruby
  loop do
    do_work
    Task.pass  # Let other tasks run
  end
  ```

- **`Task.get(name)`**: Finds a task by name. Returns `nil` if not found.

  ```ruby
  worker = Task.get("worker")
  worker.suspend if worker
  ```

- **`Task.stat`**: Returns a hash containing scheduler statistics:
  - `:tick` (Integer): Current tick count
  - `:wakeup_tick` (Integer): Next scheduled wakeup tick
  - `:dormant`, `:ready`, `:waiting`, `:suspended`: Each is a hash with:
    - `:count` (Integer): Number of tasks in this queue
    - `:tasks` (Array): Array of task objects in this queue

  ```ruby
  stats = Task.stat
  puts "Tick: #{stats[:tick]}"
  puts "Ready tasks: #{stats[:ready][:count]}"
  stats[:ready][:tasks].each { |t| puts t.name }
  ```

- **`Task.run`**: Starts the scheduler main loop. Blocks until no tasks remain ready or waiting.

  ```ruby
  Task.new { do_async_work }
  Task.run  # Run scheduler until tasks complete
  ```

- **`Task.tick`**: Returns the current tick count in milliseconds. This is the elapsed time since the scheduler started, measured in tick units.

  ```ruby
  start_tick = Task.tick
  do_work
  elapsed = Task.tick - start_tick
  puts "Work took #{elapsed} ms"
  ```

### Task Instance Methods

- **`#status`**: Returns the task status as a symbol (`:DORMANT`, `:READY`, `:RUNNING`, `:WAITING`, `:SUSPENDED`).

  ```ruby
  puts task.status  # => :READY
  ```

- **`#name`** / **`#name=`**: Get or set the task name. Returns `"(noname)"` for unnamed tasks. Note: `name=` accepts any value, but `Task.new` requires a String.

  ```ruby
  task.name = "worker-1"
  puts task.name  # => "worker-1"

  task = Task.new { }
  puts task.name  # => "(noname)"
  ```

- **`#priority`** / **`#priority=`**: Get or set the task priority (0-255). Changing priority requeues the task.

  ```ruby
  task.priority = 100  # Lower priority
  ```

- **`#suspend`**: Suspends the task, moving it to the SUSPENDED queue. The task will not run until `#resume` is called.

  ```ruby
  task.suspend
  # Later...
  task.resume
  ```

- **`#resume`**: Resumes a suspended task, moving it to the READY queue.

  ```ruby
  task.resume
  ```

- **`#terminate`**: Terminates the task immediately, moving it to DORMANT state.

  ```ruby
  task.terminate
  ```

- **`#join`**: Blocks the current task until the target task completes.

  ```ruby
  worker = Task.new { do_long_operation }
  worker.join  # Wait for completion
  puts "Worker finished"
  ```

### Task::Error

`Task::Error` is the base error class for queue-related errors. It inherits
from `StandardError`.

```ruby
begin
  q = Task::Queue.new
  q.close
  q.push(1)   # raises Task::Error: queue closed
rescue Task::Error => e
  puts e.message  # => "queue closed"
end
```

Errors raised by `Task::Queue`:

| Situation                          | Error class   | Message          |
| ---------------------------------- | ------------- | ---------------- |
| `push` on a closed queue           | `Task::Error` | `"queue closed"` |
| `pop(true)` on an empty open queue | `Task::Error` | `"queue empty"`  |

Internal consistency errors (programming errors, not queue logic) still use
`RuntimeError`:

| Situation                                               | Error class    |
| ------------------------------------------------------- | -------------- |
| Blocking `pop` called from root context                 | `RuntimeError` |
| Blocking `pop` called from inside a C function boundary | `RuntimeError` |

### Task::Queue

`Task::Queue` is a thread-safe FIFO queue for inter-task communication, analogous to CRuby's `Queue`. A task that calls `pop` on an empty queue is automatically moved to the WAITING state and rescheduled when another task pushes an item.
No polling or explicit sleep is required.

#### Creating a Queue

```ruby
q = Task::Queue.new
```

#### Pushing Items

```ruby
q.push(item)   # add to the back of the queue; raises Task::Error if closed
q << item      # alias for push
q.enq(item)    # alias for push
```

#### Popping Items

```ruby
item = q.pop           # block until an item is available
item = q.pop(true)     # non-blocking: raises Task::Error if empty
item = q.deq           # alias for pop
item = q.shift         # alias for pop
```

Behavior when the queue is **closed**:

- `pop` on a non-empty closed queue returns the remaining items normally.
- `pop` on an empty closed queue returns `nil` immediately (no blocking).

#### Inspecting the Queue

```ruby
q.size        # => Integer: number of items currently in the queue
q.length      # alias for size
q.empty?      # => true if the queue has no items
q.num_waiting # => Integer: number of tasks currently blocked in pop
```

#### Clearing and Closing

```ruby
q.clear       # remove all items; returns self
q.close       # close the queue; returns self
q.closed?     # => true if the queue has been closed
```

After `close`:

- `push` raises `Task::Error`.
- Tasks blocked in `pop` are woken and receive `nil` for an empty queue.
- `pop` on remaining items still returns them normally; returns `nil` when empty.

#### Producer/Consumer Example

```ruby
q = Task::Queue.new

Task.new(name: "producer") do
  10.times do |i|
    q.push(i)
    sleep 0.1
  end
  q.close
end

Task.new(name: "consumer") do
  loop do
    item = q.pop   # blocks until an item arrives or the queue closes
    break if item.nil?
    puts "got #{item}"
  end
end

Task.run
```

### Kernel Methods (Sleep)

The task scheduler provides task-aware sleep methods that cooperatively yield
to other tasks:

- **`sleep(seconds)`**: Sleeps for the specified duration. Accepts integers or floats (when `MRB_NO_FLOAT` is not defined).

  ```ruby
  sleep 1      # Sleep for 1 second
  sleep 0.5    # Sleep for 500ms (with float support)
  sleep        # Sleep indefinitely (no arguments)
  ```

- **`usleep(microseconds)`**: Sleeps for the specified number of microseconds.

  ```ruby
  usleep 500000  # Sleep for 500ms
  usleep 1000    # Sleep for 1ms
  ```

- **`sleep_ms(milliseconds)`**: Sleeps for the specified number of milliseconds.

  ```ruby
  sleep_ms 100  # Sleep for 100ms
  ```

**Note**: These methods override `mruby-sleep` when both gems are present. They
provide task-aware cooperative sleep when called from within a task, or
blocking sleep when called outside task context.

## Configuration

### Build Configuration

Enable the task scheduler by including the gem in your build config:

```ruby
MRuby::Build.new do |conf|
  # ... other configuration ...

  conf.gem :core => 'mruby-task'

  # ... other gems ...
end
```

This automatically defines `MRB_USE_TASK_SCHEDULER`.

### Timing Configuration

Timing parameters can be configured via C defines:

```c
#define MRB_TICK_UNIT 4              // Tick period in milliseconds (default: 4ms)
#define MRB_TIMESLICE_TICK_COUNT 3   // Ticks per timeslice (default: 3)
```

Default timeslice: `MRB_TICK_UNIT * MRB_TIMESLICE_TICK_COUNT = 12ms`

### Stack Configuration

Task stack and call info sizes:

```c
#define TASK_STACK_INIT_SIZE 64   // Initial stack entries (default: 64)
#define TASK_CI_INIT_SIZE 8       // Initial callinfo entries (default: 8)
```

These grow automatically as needed, similar to Fiber.

## Platform Requirements

### HAL (Hardware Abstraction Layer)

The task scheduler uses a Hardware Abstraction Layer (HAL) to support
different platforms. Platform-specific timer and interrupt handling
lives in `mruby-task/ports/<port_name>/task_hal.c`, and the active
port is selected at build configuration time.

#### Built-in Ports

**`ports/posix/`** - For POSIX systems (Linux, macOS, BSD, Unix)

- Uses `SIGALRM` and `setitimer()` for the timer
- Uses `sigprocmask()` for interrupt protection
- Uses `SA_RESTART` to prevent `EINTR` on system calls
- Supports multiple VMs per process
- **WASM/Emscripten support**: When compiled with Emscripten
  (`__EMSCRIPTEN__` defined), the SIGALRM timer is automatically
  disabled. JavaScript handles tick calls via `setInterval`,
  preventing double-increment of the tick counter

**`ports/win/`** - For Windows

- Uses the multimedia timer API (`timeSetEvent`/`timeKillEvent`)
- Uses `CRITICAL_SECTION` for interrupt protection
- Supports multiple VMs per process

#### Port Selection

The build system auto-detects `:posix` on Linux/macOS/BSD and `:win`
on Windows. For explicit control (cross-compilation, Cosmopolitan,
etc.) set `conf.ports` in your build configuration:

```ruby
MRuby::Build.new do |conf|
  conf.ports :posix   # or :win, or your own port name
  conf.gem core: 'mruby-task'
end
```

When `conf.ports` is set, the corresponding `ports/<name>/`
directory in every gem is compiled in; directories for other port
names are skipped. This is how `build_config/cosmopolitan.rb` reuses
the POSIX HAL on Cosmopolitan.

**Multi-VM support:**

- Both built-in ports support multiple `mrb_state` instances
- A single system timer ticks all registered VMs
- Maximum VMs: configurable via `MRB_TASK_MAX_VMS` (default: 8)

#### Adding a New HAL

To support a new platform (an RTOS, a UI runloop like GLib or Cocoa,
a bare-metal target), add a new directory
`mruby-task/ports/<name>/task_hal.c` and contribute it upstream. The
HAL must implement the six functions declared in
`mruby-task/include/task_hal.h`:

```c
/**
 * Initialize timer and register VM.
 * Called during gem initialization. Must set up a periodic timer
 * that calls mrb_tick(mrb) every MRB_TICK_UNIT milliseconds.
 */
void mrb_hal_task_init(mrb_state *mrb);

/**
 * Cleanup timer and unregister VM.
 * Called during gem finalization.
 */
void mrb_hal_task_final(mrb_state *mrb);

/**
 * Enable timer interrupts (exit critical section).
 * Must be reentrant for nested calls.
 */
void mrb_task_enable_irq(void);

/**
 * Disable timer interrupts (enter critical section).
 * Must be reentrant for nested calls.
 */
void mrb_task_disable_irq(void);

/**
 * Put CPU in low-power/idle mode.
 * Called when no tasks are ready but some are waiting; should sleep
 * roughly MRB_TICK_UNIT milliseconds and allow the timer to fire.
 */
void mrb_hal_task_idle_cpu(mrb_state *mrb);

/**
 * Sleep for the given number of microseconds of wall-clock time.
 * Must allow timer interrupts/callbacks during the sleep and should
 * complete the full duration even if interrupted.
 */
void mrb_hal_task_sleep_us(mrb_state *mrb, mrb_int usec);
```

Users selecting your port set `conf.ports :<name>`; the built-in
POSIX and Windows ports are then skipped, so there is no symbol
clash. See `mruby-task/ports/posix/task_hal.c` and
`mruby-task/ports/win/task_hal.c` for reference implementations.

## C API

The task scheduler provides a C API for integrating with C code and embedding environments. All exported functions are marked with `MRB_API` for external linkage.

### Core Scheduler API

```c
/* Tick handler - called by timer interrupt */
MRB_API void mrb_tick(mrb_state *mrb);

/* Main scheduler loop - blocks until all tasks complete */
MRB_API mrb_value mrb_task_run(mrb_state *mrb);

/* Single-step task execution for event loop integration */
MRB_API mrb_value mrb_task_run_once(mrb_state *mrb);
```

**`mrb_task_run_once()`** executes one ready task and returns. This is designed for WASM/JavaScript event loop integration where the scheduler should yield control back to the browser between task executions.

### Task Creation API

```c
/* Create a task from a proc */
MRB_API mrb_value mrb_create_task(mrb_state *mrb, struct RProc *proc,
                                  mrb_value name, mrb_value priority,
                                  mrb_value top_self);
```

Creates a new task from a `RProc` object. The `name` should be a String or `mrb_nil_value()`, `priority` should be an Integer (0-255) or `mrb_nil_value()` for default priority (128), and `top_self` sets the task's self object (or `mrb_nil_value()` to use default).

### Task Control API

```c
/* Suspend a task - prevents it from running until resumed */
MRB_API void mrb_suspend_task(mrb_state *mrb, mrb_value task);

/* Resume a suspended task - moves it back to ready/waiting queue */
MRB_API void mrb_resume_task(mrb_state *mrb, mrb_value task);

/* Terminate a task immediately - moves to dormant state */
MRB_API void mrb_terminate_task(mrb_state *mrb, mrb_value task);

/* Stop a task - marks as stopped without moving to dormant */
MRB_API mrb_bool mrb_stop_task(mrb_state *mrb, mrb_value task);

/* Get task result value */
MRB_API mrb_value mrb_task_value(mrb_state *mrb, mrb_value task);

/* Get task status symbol */
MRB_API mrb_value mrb_task_status(mrb_state *mrb, mrb_value task);
```

**Note**: These functions raise `E_RUNTIME_ERROR` if called during synchronous execution (when `scheduler_lock > 0`).

### Synchronous Execution API

```c
/* Execute a proc synchronously without context switching */
MRB_API mrb_value mrb_execute_proc_synchronously(mrb_state *mrb,
                                                  mrb_value proc,
                                                  mrb_int argc,
                                                  const mrb_value *argv);
```

This function creates a temporary task, executes it to completion, and returns the result. During execution, the scheduler is locked (`scheduler_lock++`), preventing any asynchronous task operations. This is designed for **picoruby-wasm** to execute Ruby code synchronously from JavaScript without triggering task switches.

**Key characteristics**:

- Blocks until the proc completes execution
- No context switching occurs during execution
- Other tasks cannot be created, suspended, or resumed while locked
- Temporary task is automatically freed after execution
- If the proc raises an exception, it's returned as the result

### Task Context Management API

```c
/* Initialize task context with a new proc */
MRB_API void mrb_task_init_context(mrb_state *mrb, mrb_value task,
                                   struct RProc *proc);

/* Reset task context to initial state */
MRB_API void mrb_task_reset_context(mrb_state *mrb, mrb_value task);

/* Set proc for task (without full reinitialization) */
MRB_API void mrb_task_proc_set(mrb_state *mrb, mrb_value task,
                               struct RProc *proc);
```

These functions are designed for **picoruby-sandbox** to reuse task objects for multiple executions without reallocating memory. `mrb_task_init_context()` fully reinitializes the context, while `mrb_task_proc_set()` only updates the proc pointer.

### Example: Event Loop Integration (WASM)

```c
/* JavaScript calls this function periodically via setInterval */
void js_tick_callback(void) {
  mrb_tick(mrb);
}

/* Main loop - called from JavaScript event loop */
mrb_value js_run_task_once(void) {
  return mrb_task_run_once(mrb);
}

/* Execute Ruby code synchronously from JavaScript */
mrb_value js_eval_sync(const char *code) {
  struct RProc *proc = mrb_generate_code(mrb, code);
  return mrb_execute_proc_synchronously(mrb, mrb_obj_value(proc), 0, NULL);
}
```

### Example: Task Creation from C

```c
/* Create a background task */
static mrb_value my_background_proc(mrb_state *mrb, mrb_value self) {
  /* Task code here */
  return mrb_nil_value();
}

void create_background_task(mrb_state *mrb) {
  struct RProc *proc = mrb_proc_new_cfunc(mrb, my_background_proc);
  mrb_value name = mrb_str_new_cstr(mrb, "background");
  mrb_value priority = mrb_fixnum_value(128);
  mrb_value task = mrb_create_task(mrb, proc, name, priority, mrb_nil_value());
}
```

## Examples

### Basic Multitasking

```ruby
Task.new(name: "task1") do
  3.times do |i|
    puts "Task 1: #{i}"
    sleep 0.1
  end
end

Task.new(name: "task2") do
  3.times do |i|
    puts "Task 2: #{i}"
    sleep 0.1
  end
end

Task.run  # Run until both tasks complete
```

### Priority Scheduling

```ruby
# High priority task (runs first)
Task.new(priority: 0) do
  puts "High priority"
  sleep 0.1
end

# Low priority task (runs after high priority yields)
Task.new(priority: 255) do
  puts "Low priority"
end

Task.run
```

### Cooperative Yielding

```ruby
Task.new(name: "cooperative") do
  loop do
    do_some_work
    Task.pass  # Yield to other tasks
    break if done?
  end
end

Task.new(name: "other") do
  do_other_work
end

Task.run
```

### Task Synchronization

```ruby
worker = Task.new(name: "worker") do
  puts "Working..."
  sleep 1
  puts "Work done"
  42  # Return value
end

Task.new(name: "main") do
  puts "Waiting for worker..."
  worker.join
  puts "Worker completed!"
end

Task.run
```

### Task Control

```ruby
task = Task.new do
  loop do
    puts "Running..."
    sleep 0.5
  end
end

# From another task or after Task.run returns:
task.suspend     # Pause execution
sleep 1
task.resume      # Resume execution
sleep 1
task.terminate   # Stop permanently
```

### Producer/Consumer with Task::Queue

Using `Task::Queue` eliminates polling. The consumer blocks on `pop` and wakes
automatically when the producer pushes an item.

```ruby
q = Task::Queue.new
results = []

Task.new(name: "producer") do
  ["a", "b", "c"].each do |v|
    q.push(v)
    sleep 0.1
  end
  q.close
end

Task.new(name: "consumer") do
  loop do
    item = q.pop  # blocks here until an item is available or the queue closes
    break if item.nil?
    results << item
  end
end

Task.run
puts results.inspect  # => ["a", "b", "c"]
```

Multiple producers and consumers work naturally:

```ruby
q = Task::Queue.new

3.times { |i| Task.new { q.push(i) } }

received = []
3.times { Task.new { received << q.pop } }

Task.run
puts received.sort.inspect  # => [0, 1, 2]
```

## Limitations and Compatibility

### Relationship with Fiber

Tasks and Fibers both use `mrb_context` but are **not compatible**:

- Tasks are scheduled automatically by the preemptive scheduler.
- Fibers require explicit `Fiber.yield` and `resume` calls.
- Do not mix Tasks and Fibers in the same application.

### Relationship with mruby-sleep

When `mruby-task` is enabled:

- The `sleep`, `usleep`, and `sleep_ms` methods are task-aware.
- Inside a task, they cooperatively yield to the scheduler.
- Outside a task (or when scheduler is idle), they block.
- `mruby-sleep` should be excluded from your build when using `mruby-task`.

### Thread Safety

The task scheduler is **not thread-safe**. All tasks run in a single OS thread.
For multi-core concurrency, use OS threads with separate mruby VMs per thread.

### Exceptions

Uncaught exceptions in a task will terminate that task but not affect other
tasks. The exception is not propagated to the scheduler.

### GC Integration

Task contexts are registered with the garbage collector. Tasks and their
stacks/callinfo are properly marked and freed.

## Testing

The gem includes tests that verify:

- Task creation and execution
- Priority scheduling
- Sleep and wakeup
- Join synchronization
- Suspend and resume
- Task.pass cooperative yielding
- Task::Queue push/pop FIFO order
- Task::Queue blocking pop and wakeup on push
- Task::Queue close semantics
- Task::Queue num_waiting count

Run tests with:

```bash
rake CONFIG=host-debug test:lib
```

## Implementation Details

### Task States

Each task can be in one of five states:

- `DORMANT (0x00)`: Not started or finished
- `READY (0x02)`: Ready to run
- `RUNNING (0x03)`: Currently executing
- `WAITING (0x04)`: Waiting (sleep, join, queue)
- `SUSPENDED (0x08)`: Manually suspended

### Wait Reasons

When a task is in WAITING state, the reason indicates why:

- `NONE (0x00)`: No specific reason
- `SLEEP (0x01)`: Sleeping for time
- `MUTEX (0x02)`: Waiting for mutex (reserved, not yet implemented)
- `JOIN (0x04)`: Waiting for another task
- `QUEUE (0x08)`: Waiting for an item to be pushed to a `Task::Queue`

### Scheduler Algorithm

1. Get the highest-priority task from the READY queue
2. Set task status to RUNNING and switch context (`mrb->c = &task->c`)
3. Execute task via `mrb_vm_exec()` until:
   - Task yields (sleep, pass, join)
   - Timeslice expires (preemption)
   - Task completes or terminates
4. Handle completion (wake joined tasks, move to DORMANT)
5. Run incremental GC if needed
6. Requeue task to READY (if still running) or appropriate queue
7. Repeat from step 1

### Scheduler Lock

The scheduler includes a lock counter (`mrb->task.scheduler_lock`) that prevents asynchronous task operations during synchronous execution:

- When `scheduler_lock > 0`, asynchronous APIs (`mrb_create_task`, `mrb_suspend_task`, `mrb_resume_task`) raise `E_RUNTIME_ERROR`
- This ensures that synchronous execution (via `mrb_execute_proc_synchronously`) completes without interference
- The lock is incremented when entering synchronous execution and decremented when exiting
- Maximum lock depth is 254 to prevent overflow

## Future Enhancements

Planned features not yet implemented:

- **Mutex support**: Thread-safe synchronization primitives
- **Task::SizedQueue**: Bounded queue with backpressure (push blocks when full)
- **Task.raise**: Throw exceptions to other tasks
- **Task#value**: Retrieve task return value (like Thread#value)
- **Per-task timeslice configuration**

## License

MIT License (same as mruby)

## See Also

- `mruby-fiber`: Cooperative fibers with manual control
- `mruby-sleep`: Blocking sleep (superseded by mruby-task)