GODT-1356 GODT-1302: Cache on disk concurency and API retries

- GODT-1302: Change maximum resty retries from 0 to 30
- GODT-1302: Make sure we are closing GetAttachmen io.ReadCloser on error
- GODT-1356: Do not use attachmentPool - it was useless anyway
- GODT-1356: Increase cache watcher limit to 10min
- GODT-1356: Start cache watcher right after start (do not wait first 10 min)
- GODT-1356: Limit number of buildJobs (memory allocation) in BuildAndCacheMessage
- Other: Pass context from job options (message builder) to fetcher (both message and attachments)
- Other: BuildJob contains same function as returned buildDone (proper map locking)

pkg/pmapi/manager.go

@@ -66,6 +66,13 @@ func newManager(cfg Config) *manager {
 	m.rc.OnError(m.handleRequestFailure)
 
 	// Configure retry mechanism.
+	//
+	// SetRetryCount(30): The most probable value of Retry-After is 1s (max
+	// 10s). Retrying up to 30 times will most probably cause a delay of
+	// 30s. The worst case scenario is 5min which is OK for background
+	// requests. We shuold use context to control a foreground requests
+	// which should be finish or fail sooner.
+	m.rc.SetRetryCount(30)
 	m.rc.SetRetryMaxWaitTime(time.Minute)
 	m.rc.SetRetryAfter(catchRetryAfter)
 	m.rc.AddRetryCondition(shouldRetry)