fix(GODT-3124): Handling of sync child jobs

Improve the handling of sync child jobs to ensure it behaves correctly
in all scenarios.

The sync service now uses a isolated context to avoid all the pipeline
stages shutting down before all the sync tasks have had the opportunity
to run their course.

The job waiter now immediately starts with a counter of 1 and waits
until all the child and the parent job finish before considering the
work to be finished.

Finally, we also handle the case where a sync job can't be queued
because the calling context has been cancelled.
This commit is contained in:
Leander Beernaert
2023-11-29 13:52:12 +01:00
parent 9449177553
commit 7a1c7e8743
15 changed files with 100 additions and 78 deletions

View File

@ -210,7 +210,11 @@ func (t *Handler) run(ctx context.Context,
stageContext.metadataFetched = syncStatus.NumSyncedMessages
stageContext.totalMessageCount = syncStatus.TotalMessageCount
t.regulator.Sync(ctx, stageContext)
if err := t.regulator.Sync(ctx, stageContext); err != nil {
stageContext.onError(err)
_ = stageContext.waitAndClose(ctx)
return fmt.Errorf("failed to start sync job: %w", err)
}
// Wait on reply
if err := stageContext.waitAndClose(ctx); err != nil {