We have developed the brambleterminal package to enable you to integrate spots into your cluster. Its functionality is based on R’s parallel package, but under the hood it rewires quite a lot:

  • dynamic load balancing which is robust against the termination of nodes, handles nodes of varying capacity and latency, and keeps computational overhead low
  • lets you define start-up commands to configure nodes when they become available, such as spot nodes whose request is fulfilled when the computation is already running
  • documentation of intermediate results, so nothing is lost when your computation fails for any reason
  • progress bar for the computation

The brambleterminal package is in still beta phase and we are working on a number of improvements. You are free to use it and we are happy about any feedback (touch.base@bramblecloud.com). But we give no warranty and the disclaimer in our Terms of Use applies.