Processes mostly consists of steps and business logic. If any of them are more dynamic, it becomes very difficult to meet the ROI targets. Instead of completing the last 20% of the process with 80% effort, would it not be possible to already benefit from the 80% result with 20% effort.
Filter out the 20% errors for later automation or manual processing. Profit now and learn about the caveats as you move along.With 20% of the effort an effectiveness of 80% can be achieved, this can only be possible with a flexible software solution like Yarado.
The 4 Bucket Principle, create the 4 buckets within the task directory
|Todo||the list of records that need to be processed|
|Doing||the current record that is being processed|
|Done||records without errors|
|Failed||records with identified errors or process hang|
- Build a process that is Restartable, set a timeout on the process to monitor that any record processed should be completed within a time limit.
- Catch errors at the highest level, copy current record doing to failed and restart the process.
An Example there are 7 records that need to be processed. The principle is the same for 700.000
On Start Processing, record 1 is removed from TODO and Placed into DOING
Once processed without error it is copied to DONE and initiate the next record.
This record will have an error and therefore is moved to FAILED, the next record is initiated.
Now the process or the application is stuck (External factor / Timeout) and the whole process needs to be restarted.
On restart there is a record in Doing, therefore this record failed and will be moved to failed, and the next row is initiated.
- At the end of the process all records in FAILED, can be moved to TODO once the issue is fixed or additional Business logic is added making the process more Robust or complete on every bulk run.
- Any Batch of records can be processed without the process is stopped at the first error
- Automatically restart the process until all records are processed
- Segmentation in the example is per record but any desire segmentation is applicable e.g. per Department / Geo Location / Per 10 records etc.