Commit Graph

34 Commits

Author SHA1 Message Date
e464e11ab9 GODT-2224: Refactor bridge sync to use less memory
Updates go-proton-api and Gluon to includes memory reduction changes and
modify the sync process to take into account how much memory is used
during the sync stage.

The sync process now has an extra stage which first download the message
metada to ensure that we only download up to `syncMaxDownloadRequesMem`
messages or 250 messages total. This allows for scaling the download
request automatically to accommodate many small or few very large
messages.

The IDs are then sent to a download go-routine which downloads the
message and its attachments. The result is then forwarded to another
go-routine which builds the actual message. This stage tries to ensure
that we don't use more than `syncMaxMessageBuildingMem` to build these
messages.

Finally the result is sent to a last go-routine which applies the
changes to Gluon and waits for them to be completed.

The new process is currently limited to 2GB. Dynamic scaling will be
implemented in a follow up. For systems with less than 2GB of memory we
limit the values to a set of values that is known to work.
2023-01-30 15:05:43 +01:00
cf32b84257 GODT-2305: Detect missing gluon DB 2023-01-25 14:55:13 +01:00
03c8c323bc GODT-2251: Store gluon cache in user cache rather than user data 2023-01-16 16:27:41 +01:00
0580842ad2 GODT-2229: Own the full path for gluon and do not change Database path. 2023-01-12 13:23:09 +00:00
715efaa087 Revert "GODT-2229: Allow changing cache folder to a non-empty folder."
This reverts commit b19e16e4b8.
2023-01-11 10:19:38 +01:00
b19e16e4b8 GODT-2229: Allow changing cache folder to a non-empty folder. 2023-01-10 16:40:52 +00:00
bb07138fb0 GODT-2236: add log entry when SMTP / IMAP serve method fails. 2023-01-04 16:45:34 +01:00
6d0a128111 Other: Update copyright year 2023-01-02 11:09:11 +01:00
1c374b59d3 GODT-2160: Prevent double closing of bridge if restart fails
Set imapServer instance to nil once the server is no longer running to
prevent multiple calls to close on shutdown.
2022-11-29 08:38:21 +01:00
e601245f01 Other(chore): Bump major version to v3 2022-11-23 16:08:27 +01:00
294d1edfee GODT-2120: Encrypt gluon store with gzip 2022-11-21 17:37:56 +01:00
1f0312573a GODT-1846: remove restart cues, implement restart-less behaviour.
Other: fixed case issue in SSL member function names.
Other: removed 'restart' mention in SMTP and IMAP SSL settings.
GODT-1846: modified gRPC server to introduce ConnectionMode settings.
GODT-1846: implemented connection mode handling in bridge-gui.
GODT-1846: implemented error reporting in bridge-gui for connection mode.
Other: gathered all IMAP/SMTP server settings.
GODT-1846: wired IMAP/SMTP port change errors.
Other: Renamed some error events and signals.
Other: Fixed crash in IMAP restart when not started.
Other: dismiss port error notifications before changing ports.
Other: misc. fixes.
2022-11-21 09:08:52 +00:00
7d84ab37f6 GODT-2100: Load users in parallel at startup 2022-11-19 12:13:33 +01:00
6bdcdf7fd2 GODT-2033: Only set user agent from IMAP ID if not empty 2022-11-18 19:55:23 +01:00
e60bbaa60f Other: Add more user-level logs 2022-11-18 13:30:45 +00:00
ae87d7b236 GODT-1913: pass reporter to gluon, limit restarts, add crash handlers. 2022-11-16 13:48:31 +01:00
2023df3ef8 Other: Log mailbox message counts at startup 2022-11-16 13:48:31 +01:00
c548ba85fe Other: Add more extensive logging 2022-11-16 13:48:30 +01:00
bc7912e8fb GODT-2022: Fix change between address modes 2022-11-16 13:48:30 +01:00
a553ced979 Other: go mod tidy/goimports/gofumpt after rebase on devel 2022-11-16 13:48:30 +01:00
2bda47fcad Other(refactor): Less unwieldy user type in Bridge
Instead of the annoying safe.Map type, we just use a normal go map
and mutex pair, and use safe.Lock/safe.RLock as a helper function
2022-11-16 13:48:30 +01:00
afc5307a23 Other: Don't close IMAP/SMTP listeners if they could not be created 2022-11-16 13:48:30 +01:00
6fbf6d90dc Other: Fix IMAP/SMTP/Login leaks/race conditions
Depending on the timing of bridge closure, it was possible for the 
IMAP/SMTP servers to not have started serving yet. By grouping this in
a cancelable goroutine group (*xsync.Group), we mitigate this issue.

Further, depending on internet disconnection timing during user login,
it was possible for a user to be improperly logged in. This change 
fixes this and adds test coverage for it.

Lastly, depending on timing, certain background tasks (updates check,
connectivity ping) could be improperly started or closed. This change
groups them in the *xsync.Group as well to be closed properly.
2022-11-16 13:48:30 +01:00
7c62312220 Other: Fix all linter errors 2022-11-16 12:26:09 +01:00
e10cd2a3ed GODT-1901: Allow to set IMAP SSL from UI 2022-11-16 12:26:09 +01:00
14a578f319 Other: Linter fixes after bumping linter version 2022-11-16 12:26:09 +01:00
245f2afeac GODT-1816: Fix variable case conflict 2022-11-16 12:26:09 +01:00
f01c70e506 GODT-1816: Connect Gluon Logs to bridge Logs
Ensure the IMAP commands and SMTP commands are logged to trace channels
with an entry so they are recognizable as before.
2022-11-16 12:26:09 +01:00
509a767e50 GODT-1657: More stable sync, with some tests 2022-11-16 12:26:09 +01:00
e7526f2e78 GODT-1650: Log IMAP errors 2022-11-16 12:26:09 +01:00
9d69a2e565 GODT-1657: Stable sync (still needs more tests) 2022-11-16 12:26:09 +01:00
2cb739027b GODT-1650: Use trace logger for gluon 2022-11-16 12:26:09 +01:00
9670e29d9f GODT-1815: Start with missing gluon files 2022-11-16 12:26:08 +01:00
39433fe707 GODT-1779: Remove go-imap 2022-11-16 12:26:08 +01:00