Improve the handling of sync child jobs to ensure it behaves correctly
in all scenarios.
The sync service now uses a isolated context to avoid all the pipeline
stages shutting down before all the sync tasks have had the opportunity
to run their course.
The job waiter now immediately starts with a counter of 1 and waits
until all the child and the parent job finish before considering the
work to be finished.
Finally, we also handle the case where a sync job can't be queued
because the calling context has been cancelled.
Ensure that the heartbeat background task is stopped before we close
the users as it accesses data within these instances.
Additionally, we also make sure that when telemetry is disabled, we stop
the background task.
Finally, `HeartbeatManager` now specifies what the desired interval is
so we can better configure the test cases.
* Ensure IMAP service sync cancel request waits until the sync has
completely cancelled rather than just signaling. It's possible that
due the context reset on `group.Cancel` that something may have not
have been bookmarked correctly in subsequent sync restarts.
* Handle connection lost/restored events in the services. Removes the
need to lock bridge users. Which could conflict with other ongoing
lock operations. Additionally, it ensure that if one service is
blocked it doesn't block the entire bridge.
* Revise access to bridge user locks.
It should not be possible to reach this state on purpose. But due to
scheduling and synchronization variances, it is possible in theory that
a UserDeathEvent can occur at the same time as the bridge is closing,
causing a call to `User.Close()` to be executed 2 times.
To avoid this in the future we just clear the map after all the channels
have been closed.
Add a dedicated go-routine whose sole responsibility is to manage the
life time of the IMAP and SMTP servers and their listeners.
The current implementation behaves the same way as the previous state.
The new behavior will be implemented in a follow MR.
We don't want to start processing events until those events have
somewhere to be sent to.
Also, to be safe, ensure remove and re-add the gluon user while
clearing its sync status. This shouldn't be necessary.
fix(GODT-2327): Only start processing events once sync is finished
fix(GODT-2327): avoid windows delete all deadlock
fix(GODT-2327): Clear update channels whenever clearing sync status
fix(GODT-2327): Properly cancel event stream when handling refresh
fix(GODT-2327): Remove unnecessary sync abort call
fix(GODT-2327): Fix lint issue
fix(GODT-2327): Don't retry with abortable context because it's canceled
fix(GODT-2327): Loop to retry until sync has complete
fix(GODT-2327): Better sleep (with context)