Await fsm checkpoint by yusufyian · Pull Request #6765

Await fsm checkpoint by yusufyian · Pull Request #6765 · RustPython/RustPython

and others added 30 commits

December 18, 2025 17:13

- Implemented `checkpoint_stack` and `restore_stack` methods in the Frame struct for managing stack checkpoints.
- Added `run_script_resume` method in VirtualMachine to resume execution from a checkpoint.
- Introduced `resume_path` option in Settings to specify the checkpoint file path.
- Updated command-line arguments to support `--resume` for checkpoint execution.
- Registered the new `rustpython_checkpoint` module in the standard library.

- Introduced `maybe_checkpoint_request` function to handle checkpoint requests during execution.
- Updated `checkpoint` function to set a checkpoint request with the expected last instruction index.
- Enhanced `save_checkpoint` to validate the stack state and added a new `save_checkpoint_from_exec` function for saving checkpoints from execution context.
- Modified global state to include a mutex for checkpoint requests.
- Updated demo and README to reflect changes in checkpoint handling and serialization requirements.

- Added multiple print statements to display the `imhere` variable at various stages of execution.
- Updated the `imhere` variable to include new messages for better tracking of execution flow.

- Changed the demo user from "alice" to "Tony" for better context.
- Added a section in the README for running automated tests to verify checkpoint resume functionality.

- Added demo.dot and demo.png to .gitignore to prevent tracking of demo-related files.

- Added references for Cowboy PVM Task Plan to .gitignore to prevent tracking.
- Updated import statement in demo.py to include type ignore for rustpython_checkpoint, ensuring compatibility with type checkers.

…r improved clarity and context. Changed user references in demo script and enhanced comments for better understanding of checkpoint phases.

…r improved clarity and context. Changed user references in demo script and enhanced comments for better understanding of checkpoint phases.

…tpython' to 'pvm'

- Modified README instructions and examples to use 'pvm' for running the demo and tests.
- Updated the test script to resolve the binary path to 'pvm' instead of 'rustpython'.
- Enhanced clarity in README regarding the prototype's limitations.

…process

- Updated the demo script to enhance the presentation of phases in the checkpoint/resume process.
- Reorganized print statements to provide clearer context and summaries of the runs and costs.
- Improved comments to better explain the purpose of each phase and the checkpointing mechanism.
- Ensured that the script maintains a consistent format for logging and output.

- Modified demo.py to simulate a trading day pipeline, including order loading, risk checks, and trade settlement.
- Enhanced print statements for clarity on order processing, risk flags, and final report generation.
- Updated README to describe the new financial trading scenario and its phases, ensuring users understand the demo's functionality.

…README for clarity

- Expanded demo.py to include new features for simulating trading scenarios, such as enhanced order processing and risk management.
- Improved print statements for better visibility into the trading workflow and outcomes.
- Revised README to provide clearer instructions and context for the updated trading scenario functionality.

- Added `pvm-host` crate for Host API definitions and error types.
- Introduced `pvm-runtime` crate for PVM runtime encapsulation, including new features and modules.
- Updated Cargo.lock to reflect the addition of `pvm-host` and `pvm-runtime` dependencies.

- Modified `.cspell.json` and `.cspell.dict` files to enhance spell checking capabilities.
- Updated `Cargo.toml` and `Cargo.lock` to reflect dependency changes.
- Enhanced `whats_left.py` for better argument handling.
- Refined CI workflows in `.github` for improved automation.
- Added new test cases and updated existing ones across multiple modules to ensure comprehensive coverage.
- Cleaned up and organized various source files for better readability and maintainability.

- Added a new reference file to `.gitignore` for better source management.
- Updated `frame.rs` and `checkpoint.rs` to allow dead code, improving future development flexibility.
- Refined condition checks in `checkpoint.rs` and `frame.rs` to use `PopTop` for better instruction handling.
- Improved warning logging in `core.rs` to check for VM presence before logging errors.
- Adjusted `compile.rs` to access configuration settings more clearly.
- Made `checkpoint_request` field in `PyGlobalState` private for encapsulation.
- Introduced a new utility function in `thread.rs` to check VM stack status.
- Cleaned up `interpreter.rs` by removing unused imports for better readability.
- Updated `lib.rs` to access resume path settings through the new configuration structure.

- Added a new reference file to `.gitignore` for improved source management.
- Introduced a helper function in `demo_en.py` to format section titles after resuming from checkpoints, enhancing output clarity.
- Updated the binary snapshot file `demo_en.rpsnap` to reflect changes in the demo script.

…entries

- Added a wildcard entry to `.gitignore` to ignore all reference files for better source management.
- Removed specific entries from `.gitignore` to streamline the ignore list.

…design

- Deleted `PVM_CHAIN_INTEGRATION_CN.md`, `PVM_Continuation_Design_CN.md`, and `PVM_Feasibility_and_Roadmap_CN.md` as they are no longer needed for the current project direction.
- This cleanup helps streamline the repository and focus on active development areas.

- Deleted `demo_en.py`, `demo_en.rpsnap`, and associated README files as they are no longer relevant to the current project direction.
- This cleanup streamlines the repository and focuses on active development areas, removing outdated examples and documentation.

- Deleted `demo.rpsnap` as it is no longer relevant to the current project direction.
- This cleanup continues to streamline the repository by removing outdated demo artifacts.

- Deleted `test_checkpoint.py` as it is no longer relevant to the current project direction.
- This cleanup continues to streamline the repository by removing outdated test artifacts.

- Updated comments in `state_store.py` to provide clear English descriptions of the functions' purposes, enhancing code readability and maintainability.

- Added a new entry to `.gitignore` for the `demo.rpsnap` file to improve source management.
- Refactored `checkpoint.rs` to streamline checkpoint saving and loading processes, including the introduction of `save_checkpoint_bytes_from_exec` for better data handling.
- Updated `save_checkpoint_from_exec` to accept a `code` parameter, enhancing flexibility in checkpoint management.
- Marked several functions as `#[allow(dead_code)]` to facilitate future development without immediate usage requirements.

- Added `PositionEncoding` to the `source_location` method in `SourceCode` for improved text encoding support.
- Updated the public use statement to include `PositionEncoding` from `ruff_source_file`.

- Updated `save_checkpoint_from_exec` to ensure data is written correctly by passing a reference to `data`.
- Enhanced `SnapshotWriter` and `SnapshotReader` to support additional object types and improve serialization/deserialization processes.
- Introduced caching mechanisms for type attributes and instance data to optimize performance during snapshot operations.
- Improved error handling and added cycle detection during object restoration to prevent infinite loops.
- Refactored global resolution logic in `SnapshotReader` for better handling of function globals.
- Cleaned up demo script by removing unnecessary imports and comments for clarity.

- Changed the version string format to include "PVM 0.0.2" for clarity in version reporting.
- Maintained existing formatting conventions to ensure consistency with previous outputs.

- Introduced a new section in `Cargo.toml` for PVM versioning, setting the current version to "0.0.2".
- Updated `build.rs` to read the PVM version from the environment or `Cargo.toml`, enhancing build configuration.
- Modified `version.rs` to include the PVM version in the output format for better clarity in version reporting.

- Added new methods `get_stack` and `push_stack_value` to the `Frame` struct for improved stack handling during checkpoints.
- Refactored checkpoint saving functions to support multiple frames, allowing for more complex execution states to be captured.
- Updated `save_checkpoint_with_lasti_and_stack` to handle both the instruction pointer and stack state for the innermost frame.
- Enhanced serialization of checkpoint data to include frame states and their respective stacks, improving the robustness of the checkpointing mechanism.
- Introduced a new demo script to showcase the updated checkpoint and resume functionality in a complex actor model scenario.

- Added a new entry to `.gitignore` for `actor_complex_demo.rpsnap` to improve source management.
- Introduced methods `get_stack` and `push_stack_value` in the `Frame` struct to support multi-frame checkpointing.
- Refactored checkpoint saving functions to handle both instruction pointer and stack state for improved robustness.
- Updated demo script `actor_complex_demo.py` to showcase the enhanced checkpoint and resume functionality in a complex actor model scenario.

- Introduced new `ObjTag` variants for `Enumerate`, `Zip`, `Map`, and `Filter` to enhance object type recognition during snapshot operations.
- Updated `ObjectPayload` to include corresponding payload structures for these new object types.
- Implemented serialization and deserialization logic for `Enumerate`, `Zip`, `Map`, and `Filter` to ensure proper restoration of these objects from snapshots.
- Enhanced `classify_obj` function to identify these new iterator types based on their class names.
- Improved error handling and validation during the snapshot process for these new object types, ensuring robustness in state management.

- Made `Block`, `BlockType`, and `UnwindReason` structs public for better accessibility.
- Introduced methods `push_block` and `get_blocks` in the `Frame` struct to manage block stack during checkpoints.
- Updated checkpoint saving functions to include block states and prepared locals, improving the robustness of the checkpointing mechanism.
- Enhanced serialization and deserialization processes to handle block states, ensuring accurate restoration of control flow during execution.
- Refactored demo script to reflect changes in checkpoint handling and improve clarity.

- Introduced new ObjTag variants for ListIterator, RangeIterator, and Range to enhance object type recognition during snapshot operations.
- Updated ObjectPayload to include corresponding payload structures for ListIterator, RangeIterator, and Range.
- Implemented serialization and deserialization logic for ListIterator and RangeIterator, ensuring proper restoration of these objects from snapshots.
- Enhanced classify_obj function to identify new iterator types based on their class names.
- Updated demo script to reflect changes in loop handling and checkpointing, improving clarity and functionality.

- Added `comprehensive_demo.rpsnap` to the `.gitignore` file to improve source management and prevent unnecessary tracking of demo snapshots.

- Introduced `ExecutionOptions` struct to encapsulate execution parameters for better configurability.
- Enhanced error handling in the `execute_tx` and `run_source` functions to map exceptions to `HostError` types.
- Updated the `host_error` function to include error code and name attributes for improved debugging.
- Added new methods for `ExecutionOptions` to facilitate setting parameters fluently.
- Refactored demo scripts to utilize the new execution options, improving clarity and functionality.

- Introduced `DeterminismOptions` to the `ExecutionOptions` struct for improved execution control.
- Added new error types: `DeterministicValidationError`, `NonDeterministicError`, and `OutOfGasError` to enhance error reporting.
- Updated `execute_tx_with_options` to handle determinism settings and associated error handling.
- Enhanced demo scripts to showcase determinism features and provide clearer usage instructions.
- Refactored exception mapping to include determinism-related errors for better debugging and user feedback.

- Added `trace_imports`, `trace_allow_all`, and `trace_path` options to `DeterminismOptions` for improved import tracking.
- Implemented import tracing functionality in the PVM runtime, allowing for detailed logging of module imports.
- Updated `Cargo.toml` to include a new dependency on `pvm-runtime` and incremented the PVM version to "0.1.3".
- Enhanced demo scripts to support new tracing features and provide clearer usage instructions for import tracing.
- Refactored the `determinism_check.py` script to include options for generating and printing suggested import whitelists.

- Included three new demo scripts: `escrow_marketplace_demo.py`, `batch_payroll_demo.py`, and `staking_rewards_demo.py` to showcase various business scenarios.
- Updated the README to provide usage instructions for running the new demos with determinism checks.
- Enhanced clarity in the documentation regarding input options and expected outputs for the demos.

…e at the end of the file for better readability and adherence to coding standards.

- Added support for continuation modes (FSM and checkpoint) in the PVM runtime, allowing for more flexible execution control.
- Introduced new methods for sending messages, scheduling timers, and canceling timers in the `FsHost` struct.
- Updated `HostContext` to include additional fields for actor address, message ID, and nonce.
- Enhanced the `ExecutionOptions` struct to accommodate continuation options and updated related functions for better configurability.
- Refactored demo scripts to showcase new continuation features and provide clearer usage instructions.
- Improved error handling and state management during checkpoint operations, ensuring robust execution flow.