397 Commits

Author SHA1 Message Date
ee4a8939d5 fix(GODT-2419): Use connector.ErrOperationNotAllowed
Return this error when we detect operations that we know are not allowed
so that gluon does not report them to sentry.

Includes Gluon update for the connector error
(https://github.com/ProtonMail/gluon/pull/309)
2023-03-01 10:44:18 +01:00
9e4310712c fix(GODT-2419): Use connector.ErrOperationNotAllowed
Return this error when we detect operations that we know are not allowed
so that gluon does not report them to sentry.

Includes Gluon update for the connector error
(https://github.com/ProtonMail/gluon/pull/309)
2023-02-28 17:02:35 +01:00
d6acb0fb19 fix(GODT-2418): Ensure child folders are updated when parent is 2023-02-28 14:25:11 +01:00
2db5a04e7a fix(GODT-2417): Gluon update to address ticket issues
* GODT-2417: Fixes connector requests for recovered messages.
* GODT-2416: Allow message updates to work if the literal is missing.
2023-02-28 13:41:15 +01:00
2b09796d1c fix(GODT-2399): Fix immediate message deletion during updates
Fix was applied in Gluon.
2023-02-27 15:09:11 +01:00
5ba07db7e3 chore: Bump Gluon for GODT-2399, GODT-2400 and GODT-2414
fix(GODT-2399): Defer updated message deletion
fix(GODT-2400): Allow state updates to be applied if command fails
fix(GODT-2414): Multiple deletion bug in WriteControlledStore
2023-02-27 14:53:37 +01:00
00256fafe8 fix(GODT-2394): Bump Gluon for golang.org/x/text DoS risk
This also fixes a harmless error related to multiple deletions in the store.
2023-02-23 13:09:06 +01:00
5007d451c2 feat(GODT-2385): Gluon cache fallback
Update Gluon to have access to the cache fallback reader.

Provide fallback reader to handle old cache file format.

Remove the old logic to erase all cache files on start as the fallback
option renders this irrelevant.
2023-02-22 12:00:23 +01:00
8a5c8eaf6e chore: Use gluon temp/hotfix-perth-narrows branch 2023-02-21 16:07:27 +01:00
a75a72a2b9 chore: Bump Gluon version for Go rfc5322 date parser 2023-02-21 15:46:24 +01:00
038eb6d243 feat(GODT-2366): Handle failed message updates as creates
This handles the following case:
- event says message was created
- we try to fetch the message but API says the doesn’t exist yet — we skip applying the “message created” update
- event then says message was updated at some point in the future
- we try to handle it but fail because we don’t have the message — we should treat it as a creation
2023-02-21 15:20:05 +01:00
2bd8f6938a chore: Bump gluon to create missing messages during MessageUpdated 2023-02-21 12:53:05 +01:00
889f3286b5 chore: Bump gluon to use new address parser 2023-02-21 12:15:37 +01:00
9dfdd07f7a fix(GODT-2381): Unset draft flag on sent messages 2023-02-20 15:37:36 +01:00
6a9f6a173a feat(GODT-2201): Bump Gluon to use pure Go IMAP parser 2023-02-17 14:25:16 +00:00
cac0cf35f6 feat(GODT-2361): Bump GPA to use simple encrypter 2023-02-17 14:06:15 +01:00
249658c05b chore: merge branch release/perth_narrows to devel 2023-02-15 14:39:28 +01:00
08dab2d115 feat(GODT-1264): constraint on Scheduled mailbox in connector + Integration tests. 2023-02-15 07:37:09 +00:00
0823d393ed feat(GODT-1264): creation and visibility of the 'Scheduled' system label.
feat(GODT-1264): typo in error message
feat(GODT-1264): fix split mode broken by previous commit.
2023-02-10 15:24:31 +01:00
8b9265ad96 feat(GODT-2283): Limit max import size to 30MB (bump GPA to v0.4.0) 2023-02-09 16:35:08 +01:00
afa95d4799 fix(GODT-2351): Bump GPA to automatically retry on net.OpError 2023-02-09 12:36:44 +01:00
2fa7c97f39 fix(GODT-2351): Bump GPA to better handle net.OpError 2023-02-09 12:04:39 +01:00
cdff2ef792 fix(GODT-2351): Bump GPA to properly handle net.OpError and add tests 2023-02-08 15:16:28 +00:00
a79fce907e test: Bump GPA to fix flaky test TestBridge_User_BadMessage_NoBadEvent
It was necessary to return an actual proton.APIError json object
in order for our detection to work properly in one of the tests.
2023-02-07 18:26:13 +01:00
43100d11bf fix(GODT-2323): Fix Expunge not issued for move
When moving between system labels the expunge commands were not being
issued.
2023-02-07 15:09:23 +01:00
2f75131710 fix(GODT-2340): improve logging 2023-02-07 14:31:06 +01:00
d7cac8a8f0 fix(GODT-2327): avoid windows delete all deadlock 2023-02-07 14:31:06 +01:00
f3c5e300cd fix(GODT-2327): Delay event processing until gluon user exists
We don't want to start processing events until those events have
somewhere to be sent to.

Also, to be safe, ensure remove and re-add the gluon user while
clearing its sync status. This shouldn't be necessary.

fix(GODT-2327): Only start processing events once sync is finished

fix(GODT-2327): avoid windows delete all deadlock

fix(GODT-2327): Clear update channels whenever clearing sync status

fix(GODT-2327): Properly cancel event stream when handling refresh

fix(GODT-2327): Remove unnecessary sync abort call

fix(GODT-2327): Fix lint issue

fix(GODT-2327): Don't retry with abortable context because it's canceled

fix(GODT-2327): Loop to retry until sync has complete

fix(GODT-2327): Better sleep (with context)
2023-02-07 13:41:16 +01:00
367c505444 fix(GODT-1804): Use cherry-picked mail settings in GPA 2023-02-06 15:57:24 +00:00
3bd39b3ea5 fix(GODT-1804): Only promote content headers if non-empty
When attaching public key, we take the root mime part, create a new root,
and put the old root alongside an additional public key mime part.
But when moving the root, we would copy all content headers, even empty ones.
So we’d be left with Content-Disposition: "" which would fail to parse.
2023-02-06 15:57:24 +00:00
ad65bdde9d fix(GODT-2326): Fix potential Win32 API deadlock
Update gluon so that the store implementation uses `os.Remove` instead
of `os.RemoveAll`. The latter has an issue where it can deadlock on
windows. See https://github.com/golang/go/issues/36375 for more details.
2023-02-06 16:30:03 +01:00
d82b71de89 fix(GODT-1804): Only promote content headers if non-empty
When attaching public key, we take the root mime part, create a new root,
and put the old root alongside an additional public key mime part.
But when moving the root, we would copy all content headers, even empty ones.
So we’d be left with Content-Disposition: "" which would fail to parse.
2023-02-06 13:18:08 +00:00
c3d5a0b8f8 feat(GODT-2295): notifications for IMAP login when signed out. 2023-02-03 11:18:59 +01:00
a955dcbaa9 fix(GODT-2323): Fix Expunge not issued for move
When moving between system labels the expunge commands were not being
issued.
2023-02-02 07:21:01 +01:00
45ec6b6e74 feat(GODT-2289): UIDValidity as Timestamp
Update UIDValidity to be timestamp with the number of seconds since
the 1st of February 2023. This avoids the problem where we lose the
last UIDValidity value due to the vault being missing/corrupted/deleted.
2023-02-01 14:04:45 +01:00
c8065c8092 GODT-2312: used space is properly updated. 2023-01-31 07:22:03 +00:00
e464e11ab9 GODT-2224: Refactor bridge sync to use less memory
Updates go-proton-api and Gluon to includes memory reduction changes and
modify the sync process to take into account how much memory is used
during the sync stage.

The sync process now has an extra stage which first download the message
metada to ensure that we only download up to `syncMaxDownloadRequesMem`
messages or 250 messages total. This allows for scaling the download
request automatically to accommodate many small or few very large
messages.

The IDs are then sent to a download go-routine which downloads the
message and its attachments. The result is then forwarded to another
go-routine which builds the actual message. This stage tries to ensure
that we don't use more than `syncMaxMessageBuildingMem` to build these
messages.

Finally the result is sent to a last go-routine which applies the
changes to Gluon and waits for them to be completed.

The new process is currently limited to 2GB. Dynamic scaling will be
implemented in a follow up. For systems with less than 2GB of memory we
limit the values to a set of values that is known to work.
2023-01-30 15:05:43 +01:00
c86cdf737f GODT-2311: Fix missing headers in re-downloaded Gluon messages 2023-01-27 09:55:02 +01:00
badebbef9f GODT-2297: More significantly improve GPA's paging algorithm 2023-01-26 12:43:30 +01:00
5e072c3282 Other: Slightly improve GPA's paging algorithm 2023-01-26 09:48:19 +01:00
cf32b84257 GODT-2305: Detect missing gluon DB 2023-01-25 14:55:13 +01:00
160489a771 GODT-2223: Testing event processing scenarios. 2023-01-25 13:05:04 +01:00
996c6826b9 GODT-2223: Handle gracefully multiple create events with same id. 2023-01-25 13:04:04 +01:00
ee4da8a89c GODT-2291: Change gluon store default location from Cache to Data. 2023-01-24 13:35:45 +00:00
94703bcf37 GODT-2223: Treat label/message deletion errors as non-critical 2023-01-23 21:25:15 +01:00
c8925cd270 GODT-2223(test): Assert that bad request errors lead to user logout 2023-01-18 14:10:23 +00:00
70f0384cc3 GODT-2165: Reduce UTF8 parsing errors from TLS header input
Updates Gluon to include fix which does not report UTF-8 errors when we
parse a TLS header on a non TLS connection.
2023-01-18 13:06:14 +01:00
7119c566ef Other: Bump GPA to v0.3.0 2023-01-17 14:10:07 +00:00
0e9428aaae GODT-2252: Recover from deleted cached messages
Update to latest Gluon version and implement the new
`Connector.GetMessageLiteral` function.
2023-01-17 14:02:39 +01:00
93c7552a41 GODT-2202: Report update errors from Gluon
For every update sent to gluon wait and check the error code to see if
an error occurred.

Note: Updates can't be inspect on the call site as it can lead to
deadlocks.
2023-01-13 15:54:31 +01:00