Commit Graph

76 Commits

Author SHA1 Message Date
b2830b39e0 fix(GODT-2782): Filter all labels when doing perma delete check
Previously we were not filtering out labels we ignored from the
perma-delete check. The introduction of new system label types could
break this check leading to user never being able to perma-delete
messages.
2023-07-11 15:25:46 +02:00
91de6e001e fix(GODT-2763): Missing Answered flag on Sync and Message Create
Ensure we are using the same flag conversion code for all IMAP updates.
2023-07-06 14:02:05 +02:00
40c48ba804 fix(GODT-2774): Add external context to telemetry tasks
This ensures they get cancelled if the parent context becomes invalid
2023-07-06 13:09:35 +02:00
124231c3c7 feat(GODT-2710): Send config success on IMAP/SMTP connection.. 2023-06-30 09:43:26 +00:00
ab70e85f1c fix(GODT-2653): Only log when err is not nil 2023-06-22 10:12:03 +02:00
51315d5d2b feat(GODT-2653): Log API error details on Message import and send 2023-06-05 13:21:04 +02:00
98031d296e Revert "fix(GODT-2588): Always perma-delete from Drafts/Trash"
This reverts commit f9a0c35daa.
2023-05-12 10:19:56 +02:00
b51d85e768 chore: upgraded golangci-lint v1.52.2 and fixed all issues. 2023-05-05 07:57:22 +02:00
910060a14c fix(GODT-2598): Map Message Size Error to Gluon Error
Prevents messages with invalid size from ending up in the recovery
mailbox.
2023-04-28 11:44:02 +02:00
f9a0c35daa fix(GODT-2588): Always perma-delete from Drafts/Trash 2023-04-25 13:49:05 +02:00
c84919faae chore: merge release/Quebec into devel 2023-04-05 13:55:22 +02:00
ec92c918cd feat(GODT-2500): Add panic handlers everywhere. 2023-04-03 06:38:31 +02:00
02ca6428b5 fix(GODT-2407): Replace invalid email addresses with emtpy for new Drafts 2023-03-17 13:25:42 +01:00
ee4a8939d5 fix(GODT-2419): Use connector.ErrOperationNotAllowed
Return this error when we detect operations that we know are not allowed
so that gluon does not report them to sentry.

Includes Gluon update for the connector error
(https://github.com/ProtonMail/gluon/pull/309)
2023-03-01 10:44:18 +01:00
08dab2d115 feat(GODT-1264): constraint on Scheduled mailbox in connector + Integration tests. 2023-02-15 07:37:09 +00:00
0823d393ed feat(GODT-1264): creation and visibility of the 'Scheduled' system label.
feat(GODT-1264): typo in error message
feat(GODT-1264): fix split mode broken by previous commit.
2023-02-10 15:24:31 +01:00
8b9265ad96 feat(GODT-2283): Limit max import size to 30MB (bump GPA to v0.4.0) 2023-02-09 16:35:08 +01:00
29072f0285 fix(GODT-2343): Only poll after send if sync is complete 2023-02-06 16:39:32 +00:00
34cd611a8b chore: Disable funlen linter 2023-02-06 14:29:13 +00:00
a36dbbf422 fix(GODT-2333): Do not allow modifications to All Mail label
Rather than waiting for API to reply, prevent these operations from
taking place in the first place.
2023-02-03 05:26:42 +00:00
a955dcbaa9 fix(GODT-2323): Fix Expunge not issued for move
When moving between system labels the expunge commands were not being
issued.
2023-02-02 07:21:01 +01:00
45ec6b6e74 feat(GODT-2289): UIDValidity as Timestamp
Update UIDValidity to be timestamp with the number of seconds since
the 1st of February 2023. This avoids the problem where we lose the
last UIDValidity value due to the vault being missing/corrupted/deleted.
2023-02-01 14:04:45 +01:00
e464e11ab9 GODT-2224: Refactor bridge sync to use less memory
Updates go-proton-api and Gluon to includes memory reduction changes and
modify the sync process to take into account how much memory is used
during the sync stage.

The sync process now has an extra stage which first download the message
metada to ensure that we only download up to `syncMaxDownloadRequesMem`
messages or 250 messages total. This allows for scaling the download
request automatically to accommodate many small or few very large
messages.

The IDs are then sent to a download go-routine which downloads the
message and its attachments. The result is then forwarded to another
go-routine which builds the actual message. This stage tries to ensure
that we don't use more than `syncMaxMessageBuildingMem` to build these
messages.

Finally the result is sent to a last go-routine which applies the
changes to Gluon and waits for them to be completed.

The new process is currently limited to 2GB. Dynamic scaling will be
implemented in a follow up. For systems with less than 2GB of memory we
limit the values to a set of values that is known to work.
2023-01-30 15:05:43 +01:00
faf28a6d4e GODT-2223: Fix mutex double lock 2023-01-18 14:10:23 +00:00
59745e6fb6 GODT-2223: Ensure apiLabels map is properly up to date 2023-01-18 14:10:23 +00:00
0e9428aaae GODT-2252: Recover from deleted cached messages
Update to latest Gluon version and implement the new
`Connector.GetMessageLiteral` function.
2023-01-17 14:02:39 +01:00
9390cb64b4 GODT-1817: Restore move related feature tests
Gluon updated to latest dev commit, required for feature. Checks from
move_local_folder.feature are implemented in Gluon.
2023-01-06 10:58:07 +01:00
757e8a02ec GODT-2233: Fix sub folder creation bug
Sub folders with more than 2 levels of depth (e.g.: Folders/first/second)
could not be created since we did not update the known label list we use
to validate the request.
2023-01-02 11:41:49 +01:00
6d0a128111 Other: Update copyright year 2023-01-02 11:09:11 +01:00
04881b9b78 GODT-2178: Bump go-proton-api to fix drafts 2022-12-05 15:14:30 +01:00
8408a5fdc0 GODT-2170: Improving test server behaviour. 2022-12-02 13:27:19 +00:00
828fe0e86e GODT-2170: Update draft event means delete old and create new message. 2022-12-02 13:27:19 +00:00
5c3179df48 GODT-2170: User create draft rounte: first steps. 2022-12-02 13:27:19 +00:00
618cb27ac1 Other: Disable perma-delete for expunge on Spam folder 2022-12-02 13:43:53 +01:00
e601245f01 Other(chore): Bump major version to v3 2022-11-23 16:08:27 +01:00
ad1fb47b0d Other: Switch from liteapi to go-proton-api 2022-11-23 15:17:56 +01:00
d1f140ebcb Other: More descriptive event poll name 2022-11-21 10:20:28 +00:00
721cd9f319 GODT-2002: Wait for API events to be applied after send
When we send a message, the send recorder records the sent message.
When the client then appends an identical message to the sent folder,
the deduplication works and instead returns the message ID of the
existing proton message, rather than creating a new message. Gluon is
expected to notice that it already has this message ID and perform
some deduplication stuff internally.

However, it can happen that gluon doesn't yet have this message ID,
because we haven't yet received the "Message Created" event from the
API. To prevent this, we poll the events after send and wait for all
new events to be applied.

There's still a chance that the event wasn't generated yet on the API
side. Not sure what we can do about this.
2022-11-21 10:20:28 +00:00
9e6cbcb35e GODT-2040: Bump UID validity when clearing sync status 2022-11-20 21:48:22 +01:00
59278913ca GODT-2037: Handle and log API refresh event 2022-11-16 13:48:31 +01:00
098685ec8b GODT-2030: Rework deletion check on expunge
Some messages were not being deleted properly because they were also
present in the All-Sent folder.

The code has now been changed to filter out AllMail, AllDraft and
AllSend. If there are no remaining labels, the message will be deleted
permanently.
2022-11-16 13:48:30 +01:00
6c9293ec14 Other: Use current time if date is missing 2022-11-16 13:48:30 +01:00
1e29a5210f GODT-1954: Draft message support
Add special case handling for draft messages so that if a Draft is
updated via an event it is correctly updated on the IMAP client via a
the new `imap.MessageUpdated event`.

This patch also updates Gluon to the latest version.
2022-11-16 13:48:30 +01:00
c548ba85fe Other: Add more extensive logging 2022-11-16 13:48:30 +01:00
4ded8784fc Other: Allow non-received messages to be imported to INBOX 2022-11-16 13:48:30 +01:00
75b788b793 GODT-1993: Use more efficient filtering for message deletion 2022-11-16 13:48:30 +01:00
62d62474fb Other: Poll events after each IMAP operation
This matches bridge's behaviour (except for it being a non-blocking
wait).

It also cleans up some unused connector methods.
2022-11-16 13:48:30 +01:00
a213b48f93 GODT-2002: Poll after SMTP send
After sending, a client might append to the sent folder over IMAP.
In this case, we perform deduplication and return the message ID of the
sent message. However, if we haven't already processed this message in
gluon, it doesn't work as expected.

This change polls the event stream immediately after send. Note that it
doesn't wait for these events to be processed; that should be done in a
follow-up commit.
2022-11-16 13:48:30 +01:00
8f420d728c GODT-1984: Handle permanent message deletion
Only delete messages when unlabeled from trash/spam if they only exists
in All Mail and (spam or trash).

This patch also ports delete_from_trash.feature and use status rather
than fetch to count messages in a mailboxes.
2022-11-16 13:48:30 +01:00
df818bc2b8 Other: Update UID validity in vault when necessary 2022-11-16 13:48:30 +01:00