[Feature Request]: Add timeout to RunInference operations
What would you like to happen?
We should add the ability to add timeouts to RunInference
Full design here - https://docs.google.com/document/d/19ves6iv-m_6DFmePJZqYpLm-bCooPu6wQ-Ti6kAl2Jo/edit?usp=sharing
As part of this, we should complete the following subtasks:
- Add callback support to
with_exception_handling(Add callback to with_exception_handling #32136) - Allow multi_process_shared objects to be garbage collected via a single call (Add hard delete function to multi_process_shared #32238)
- Add ability to garbage collect RunInference models from callback (Add timeout to runinference #32237)
- Add timeout parameter to
with_exception_handlingfunction in RunInference (Add timeout to runinference #32237) - Move garbage collection to finish_bundle (Move RunInference GC to finalize_bundle #32281)
Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
Issue Components
- Component: Python SDK
- Component: Java SDK
- Component: Go SDK
- Component: Typescript SDK
- Component: IO connector
- Component: Beam YAML
- Component: Beam examples
- Component: Beam playground
- Component: Beam katas
- Component: Website
- Component: Infrastructure
- Component: Spark Runner
- Component: Flink Runner
- Component: Samza Runner
- Component: Twister2 Runner
- Component: Hazelcast Jet Runner
- Component: Google Cloud Dataflow Runner