Improve the handling of sync child jobs to ensure it behaves correctly
in all scenarios.
The sync service now uses a isolated context to avoid all the pipeline
stages shutting down before all the sync tasks have had the opportunity
to run their course.
The job waiter now immediately starts with a counter of 1 and waits
until all the child and the parent job finish before considering the
work to be finished.
Finally, we also handle the case where a sync job can't be queued
because the calling context has been cancelled.
Fix tracking of child jobs. The build stage splits the incoming work
even further, but this was not reflected in the wait group counter. This
also fixes an issue where the cache was cleared to late.
Add more debug info for analysis.
Refactor sync state interface in order to have persistent sync rate.
Implementation of the new sync service that interleaves syncing jobs for
all active users.
It also includes improvements to the message downloader. The download
will now auto rate limit the parallel workers based on the server
responses.
Additionally each of the stages is now tested in isolation to ensure the
behavior matches the expectations.
Finally, this patch does not replace the existing IMAP sync. A follow up
patch is necessary to integrate the IMAP bits into the interfaces
required by these changes.
