You can elect to set up a schedule to run the batch processing on a recurring basis.
First create a yaml called “scheduled_job.yaml” that defines the schedule and entry point for the job in the same directory as the python script you uploaded. By using “.” as the working directory Anyscale will automatically use the Workspaces working directory and configs / dependencies for the job.
name: batch-example
description: Run every 10 minutes
entrypoint: python script.py
working_dir: .
schedule:
cron_expression: "*/10 * * * *"
timezone: local
Then from the terminal run:
anyscale schedule create scheduled_job.yaml