Commit Graph

252 Commits

Author SHA1 Message Date
45ec6b6e74 feat(GODT-2289): UIDValidity as Timestamp
Update UIDValidity to be timestamp with the number of seconds since
the 1st of February 2023. This avoids the problem where we lose the
last UIDValidity value due to the vault being missing/corrupted/deleted.
2023-02-01 14:04:45 +01:00
e382687168 fix(GODT-2318): Remove gluon DB if label sync was incomplete 2023-01-31 12:40:42 +00:00
c0aacb7d62 GODT-2224: Allow the user to specify max sync memory usage in Vault 2023-01-31 09:40:22 +01:00
e464e11ab9 GODT-2224: Refactor bridge sync to use less memory
Updates go-proton-api and Gluon to includes memory reduction changes and
modify the sync process to take into account how much memory is used
during the sync stage.

The sync process now has an extra stage which first download the message
metada to ensure that we only download up to `syncMaxDownloadRequesMem`
messages or 250 messages total. This allows for scaling the download
request automatically to accommodate many small or few very large
messages.

The IDs are then sent to a download go-routine which downloads the
message and its attachments. The result is then forwarded to another
go-routine which builds the actual message. This stage tries to ensure
that we don't use more than `syncMaxMessageBuildingMem` to build these
messages.

Finally the result is sent to a last go-routine which applies the
changes to Gluon and waits for them to be completed.

The new process is currently limited to 2GB. Dynamic scaling will be
implemented in a follow up. For systems with less than 2GB of memory we
limit the values to a set of values that is known to work.
2023-01-30 15:05:43 +01:00
4aa1091f62 GODT-2210: fix splash screen always showing on CentOS and Ubuntu. 2023-01-27 14:10:14 +01:00
cf32b84257 GODT-2305: Detect missing gluon DB 2023-01-25 14:55:13 +01:00
e7dea0a77f GODT-2223(test): Fix array access off-by-one in test code 2023-01-25 13:05:05 +01:00
160489a771 GODT-2223: Testing event processing scenarios. 2023-01-25 13:05:04 +01:00
996c6826b9 GODT-2223: Handle gracefully multiple create events with same id. 2023-01-25 13:04:04 +01:00
24c68f100e GODT-2210: v3.0 splash screen.
Other: new splash screen content.
2023-01-25 09:42:00 +00:00
f1404cd3ee GODT-1556: If no references, use the in-reply-to header as ParentID. 2023-01-24 15:36:23 +01:00
ee4da8a89c GODT-2291: Change gluon store default location from Cache to Data. 2023-01-24 13:35:45 +00:00
94703bcf37 GODT-2223: Treat label/message deletion errors as non-critical 2023-01-23 21:25:15 +01:00
b9c3fa9401 GODT-2266: Add test for sent message flags 2023-01-20 13:04:01 +00:00
d26a8319b6 Other: Better user load logs 2023-01-19 17:47:45 +01:00
c8925cd270 GODT-2223(test): Assert that bad request errors lead to user logout 2023-01-18 14:10:23 +00:00
e35f3b6056 GODT-2223: Fix handling of create/update label events 2023-01-18 14:10:23 +00:00
849c8bee78 GODT-2223: Handle bad events by logging user out 2023-01-18 14:10:23 +00:00
7119c566ef Other: Bump GPA to v0.3.0 2023-01-17 14:10:07 +00:00
a377384553 Other: added user's primary email address to the vault. 2023-01-17 11:27:54 +01:00
03c8c323bc GODT-2251: Store gluon cache in user cache rather than user data 2023-01-16 16:27:41 +01:00
0580842ad2 GODT-2229: Own the full path for gluon and do not change Database path. 2023-01-12 13:23:09 +00:00
715efaa087 Revert "GODT-2229: Allow changing cache folder to a non-empty folder."
This reverts commit b19e16e4b8.
2023-01-11 10:19:38 +01:00
b19e16e4b8 GODT-2229: Allow changing cache folder to a non-empty folder. 2023-01-10 16:40:52 +00:00
bb07138fb0 GODT-2236: add log entry when SMTP / IMAP serve method fails. 2023-01-04 16:45:34 +01:00
3743e45566 GODT-2221: Set DOH off by default. 2023-01-04 12:08:06 +00:00
6d0a128111 Other: Update copyright year 2023-01-02 11:09:11 +01:00
4cba009ac8 GODT-2188: Do not fail append with invalid mime-type
Requires gluon update where the fix was applied.

Disable TestBridge_Sync_BadMessage as it is no longer valid with the
latest Gluon fixes. Traked as GODT-2215.
2022-12-19 15:24:35 +01:00
e4b81063cb GODT-2213: Don't unnecessarily enable/disable autostart 2022-12-19 08:29:57 +00:00
4b3d4690e8 GODT-2196: Do not generate message updates for unknown labels
During sync a user may continue to perform operations on the server it
is possible we run into a message which has a labelID we are not aware
of. To counter this we issue `CreateMessage` updates with
`IgnoreUnknownMailboxIDs` set to true. Eventually, after sync the state
will resolve itself with events.
2022-12-15 09:37:22 +01:00
792317e945 Other: Prevent double login 2022-12-14 10:15:40 +01:00
c39108043b Merge branch 'release/perth_narrows' into devel (3.0.7) 2022-12-13 19:37:12 +01:00
55ee6a9d13 Other: default UIDVALIDITY 2022-12-13 16:16:54 +01:00
1aca2cde71 GODT-2181(test): Refactor integration test setup a bit 2022-12-13 10:28:59 +01:00
ebeec056cd Other(test): Add test that we skip and report bad messages during sync 2022-12-07 12:10:02 +01:00
6c30a04ac0 Merge branch 'release/perth_narrows' into devel 2022-12-07 09:26:59 +01:00
58d04f9693 GODT-2187: Skip messages during sync that fail to build/parse 2022-12-06 14:07:13 +00:00
04881b9b78 GODT-2178: Bump go-proton-api to fix drafts 2022-12-05 15:14:30 +01:00
990b8cda96 GODT-2180: Allow login with FIDO2
The API docs didn't specify what the "integer" meant. Turns out it's a
bitfield; we can't compare with equality.
2022-12-05 14:22:38 +01:00
4003e0a2ab GODT-2042: fix setup guide not always showing on first login. 2022-12-02 11:36:31 +01:00
6ac8a4c0bc GODT-2160: Ensure we can safely move cache file
It's currently impossible to wait until all SQLite write finish to disk.
This is not guaranteed when closing the ent DB client.

The existing code to move the cache handles the case where the
new location is on a new drive. However, due to the above issue this can
now lead to database corruption.

To avoid database corruption we now use the `os.Rename` function and
prevent moving the cache between drives until a better solution can be
implemented.
2022-11-29 16:33:53 +01:00
0827d81617 Other: Bump gluon version to drop non-UTF-8 commands 2022-11-29 16:20:16 +01:00
e71e56f7fe Other: Ensure context is string in sentry reports 2022-11-29 14:58:29 +00:00
9f64e8a6fa Other: Wipe vault properly on factory reset
Deleting the file isn't enough because it's still held in memory
and is written back to disk on the next write (SetLastVersion during
bridge teardown).
2022-11-29 09:59:30 +01:00
1c374b59d3 GODT-2160: Prevent double closing of bridge if restart fails
Set imapServer instance to nil once the server is no longer running to
prevent multiple calls to close on shutdown.
2022-11-29 08:38:21 +01:00
ae7ae2886f GODT-2041: Crash after factory reset
I forgot to remove the user from the users map during factory reset.
This meant the (deleted) would attempt to be closed during teardown.
2022-11-28 19:58:10 +01:00
e601245f01 Other(chore): Bump major version to v3 2022-11-23 16:08:27 +01:00
ad1fb47b0d Other: Switch from liteapi to go-proton-api 2022-11-23 15:17:56 +01:00
57e9310510 Other: Use API call rather than server-specific method in test code 2022-11-22 16:01:01 +01:00
b81fa5ed39 GODT-2139: Validate key pass during login 2022-11-22 14:51:31 +00:00