Support for Continuously Running Jobs

Problem

Some recently developed apps run jobs very often, e.g., every 10 secs or 1 min. Recent examples are mParticle segment sync app and sfmc campaign sync app. Current implementation of jobs is not really designed for such use case as every time a job is fired a heavy bootstrapping is performed (k8s job created, Anduin started, secrets loaded, etc.). This causes performance issues and not optimal utilization of resources.

Scope

• For frequently running jobs, pre-load relevant resources and keep them available between job executions

• This can be either an internal change or having an impact on the semantics of job execution, e.g. different definition of `prepare’ method, state management, etc.

Impact

The aim of this initiative is to improve performance and scalability and open OCP to more use cases, which should increase the number of apps in App Directory.