Compare commits

...

65 Commits

Author SHA1 Message Date
6230200218 Import-Export app Elbe 1.2.2
Changed
* Improvements to the import from large mbox files with multiple labels
* Not allow to run multiple instances of the app or transfers at the same time
* Better handling and displaying of skipped messages
* Various enhancements of the import process related to parsing
* Cosmetic GUI changes
* Better error handling

Fixed
* Linux font issues - Fedora specific
* App response to the user pausing and canceling import or export
* Upgrade errors
2020-11-27 07:37:08 +01:00
70645c1732 Import-Export Elbe 1.2.1
• Further improvements to address and date parsing
• Better handling and displaying of skipped messages
• Improved error reporting
2020-11-11 14:03:00 +01:00
1055e60d27 Fixing time order in changelog. 2020-11-11 12:02:56 +01:00
e04196f8a0 feat: switch to public go-rfc5322 parser 2020-11-10 09:27:07 +00:00
11a0dec047 Using atomic bool 2020-11-10 07:50:29 +00:00
b9740e1b7d Close connection before deleting labels to prevent panics accessing deleted bucket 2020-11-10 07:50:29 +00:00
f0695eb870 add test gui 2020-11-09 11:58:32 +00:00
a40018cdf9 Percentage available on progress count struct 2020-11-09 11:58:32 +00:00
5b7eabe21a Skipped messages do not change total counts but shows as separate number 2020-11-09 11:58:32 +00:00
d5d60aa11b feat: remove tls upgrade error notification 2020-11-09 10:59:42 +00:00
a62fa132e6 rename build tag 2020-11-06 16:02:30 +01:00
052395f917 test: add benchmarks for rfc5322 address/date parser 2020-11-04 15:00:18 +01:00
9a77650004 Bridge GoldenGate 1.5.0
- Ensured better message flow by refactoring both address and date parsing
- Improved secure connectivity checks
- Better deb packaging
- More robust error handling

- Ensured that conversations are properly threaded
- Fixed Linux font issues (Fedora)
- Better handling of Mime encrypted messages
2020-11-04 12:26:07 +01:00
f1d70361c9 Do not include conversation ID in references 2020-11-04 09:12:16 +00:00
3496599723 feat: custom address/date parser based on rfc5322 abnf 2020-11-03 16:21:06 +01:00
9e0635a6a4 fix: don't check tls fingerprints when checking connectivity 2020-11-02 13:38:39 +00:00
10509621ce Updated go-mbox dependency back to upstream 2020-11-02 10:32:21 +01:00
3727ecdfe5 Show in error counts also lost messages at the end report 2020-10-30 13:58:32 +00:00
ac71d22e86 Waiting for unilateral update during deleting the message 2020-10-30 13:42:04 +00:00
bc81356d53 test: update feature file to use new "seq" command 2020-10-29 13:10:54 +01:00
881cb64beb Release Danube: notes, version bump, change log 2020-10-29 12:57:59 +01:00
1286e57b63 Support Apple Mail MBOX export format 2020-10-29 09:07:37 +01:00
fe5f73d96e Fix crash when IMAP client connects while account is logging in 2020-10-29 07:21:45 +00:00
8f7a8b31a3 Apply 1 suggestion(s) to 1 file(s) 2020-10-28 16:42:57 +00:00
68db35d5d4 Not able to update I-E on mac GODT-794
Added missing signal, corrected the update name, log tweaks.
2020-10-28 16:42:57 +00:00
df17017ced Apply 1 suggestion(s) to 1 file(s) 2020-10-28 10:20:32 +00:00
5c48332b0e change rectangle to column in global settings GODT-677 2020-10-27 10:13:08 +01:00
8985738af5 Merge master into devel 2020-10-23 10:31:08 +02:00
2d8a676dd5 Merge branch 'release/forth' into release/danube 2020-10-22 18:00:33 +02:00
7e0a9f398c I/E Fix printing zero time in error report 2020-10-22 09:12:56 +00:00
9af5769510 Apply 1 suggestion(s) to 1 file(s) 2020-10-22 08:26:35 +00:00
bb46d9a009 README and BUILD info about Import-Export and tags 2020-10-22 10:22:00 +02:00
606b42a6e7 Fix flaky TestFailUnpauseAndStops 2020-10-22 10:04:22 +02:00
d547f5ea22 Changelog 2020-10-21 13:56:55 +02:00
563b4889e3 Update go-imap dependency to get fix for UTF-7 incompatibility 2020-10-21 09:15:42 +00:00
b449beb68c Do not spam sentry with bad ID by integration test 2020-10-21 08:38:54 +00:00
f9d58f4f9c Merge branch 'release/forth' into release/danube 2020-10-21 09:07:27 +02:00
1dfec9902e gofmt fix 2020-10-21 09:04:06 +02:00
79cafee2eb Support quoted printable and filter out some auto-generated Gmail labels 2020-10-21 09:04:06 +02:00
64fbcdc1ca Fix mbox scanning 2020-10-21 09:04:06 +02:00
e4a341af3a Better log message 2020-10-21 09:04:05 +02:00
e0292fe957 Use map instead of list as set 2020-10-21 09:04:05 +02:00
ef85c8df24 Detect Gmail labels from All Mail mbox export 2020-10-21 09:04:05 +02:00
719d369c2a Fix transfer stopping 2020-10-21 06:42:54 +00:00
51b6f95342 Show fatal errors after export is terminated 2020-10-21 06:14:39 +00:00
26fb1fc34d Sanizize mailbox name for exporting 2020-10-21 06:02:02 +00:00
ae1578a5e2 GODT-829 fix apple mail subfolders 2020-10-20 19:09:59 +02:00
cfd8e56277 Do not resume paused transfer progress after dismissing cancel popup 2020-10-19 10:25:52 +02:00
4893931a8d Fix deadlock in integration tests for Import-Export 2020-10-16 10:53:44 +02:00
932928ddc8 Allow to send calendar update multiple times 2020-10-15 13:11:40 +00:00
a33e414f01 Do not mix font awesome icon with regular text to avoid issues on Fedora 2020-10-15 12:48:09 +00:00
43d54c8f4f Clear separation of different message IDs in integration tests 2020-10-14 14:41:39 +02:00
6cbc11a75d Fix update on windows 2020-10-14 11:25:19 +02:00
a21bb130e1 Append duplicate of emails with References 2020-10-14 10:11:49 +02:00
12403785af fix: replace, don't add, transfer encoding when making body 7-bit clean 2020-10-09 13:55:37 +02:00
b4892855d4 Set flags by FLAGS (not using +/-FLAGS) do not change spam state 2020-10-06 08:42:33 +00:00
7ff67f2217 Reverted sending IMAP updates to be not blocking again 2020-10-05 11:33:16 +02:00
4912c27be8 Changelog 2020-10-05 10:51:11 +02:00
288ba11452 test: add test for sending pgp/mime as plaintext 2020-10-01 16:56:38 +02:00
7874183052 fix(GODT-770): handle extraneous end-of-mail 2020-10-01 16:16:15 +02:00
b12873f1df Fix of speed of checking whether message is deleted 2020-10-01 13:42:16 +00:00
dc9851f8ea fix(GODT-749): don't force pgp/inline when sending plaintext 2020-10-01 10:47:39 +02:00
ec73170e9b Use label.Path instead of Name 2020-09-30 09:38:35 +02:00
68616e470c chore: bump crypto version 2020-09-25 15:45:29 +02:00
53cd2ff524 CI artifacts only for a day 2020-09-25 11:29:45 +02:00
131 changed files with 2044 additions and 938 deletions

View File

@ -82,7 +82,9 @@ dependency-updates:
script:
- make build
artifacts:
expire_in: 2 week
# Note: The latest artifacts for refs are locked against deletion, and kept regardless of the expiry time.
# Introduced in GitLab 13.0 behind a disabled feature flag, and made the default behavior in GitLab 13.4.
expire_in: 1 day
build-linux:
extends: .build-base

View File

@ -19,7 +19,6 @@ Otherwise, the sending of crash reports will be disabled.
export MSYSTEM=
```
### Build Bridge
* in project root run
@ -44,6 +43,12 @@ make build-ie
* for `windows`, the binary will have the file extension `.exe` (e.g `proton-bridge.exe`)
* for `darwin`, the application will be created with name of the project directory (e.g `proton-bridge.app`)
### Tags
Note that repository contains both Bridge and Import-Export apps and they are
not released together. Therefore, each app has own tag prefix. Bridge tags
starts with `br-` and Import-Export tags starts with `ie-`. Both tags continue
with semantic versioning `MAJOR.MINOR.PATCH`. An example of full tag is
`br-1.4.4` or `ie-1.1.2` (current versions in October 2020).
## Useful tests, lints and checks
In order to be able to run following commands please install the development dependencies:

View File

@ -1,8 +1,102 @@
# ProtonMail Bridge Changelog
# ProtonMail Bridge and Import-Export app Changelog
Changelog [format](http://keepachangelog.com/en/1.0.0/)
## Unreleased
## [IE 1.2.1, 1.2.2] Elbe
### Added
* GODT-799 Skipped messages do not change total counts but shows as separate number.
## Fixed
* GODT-799 Fix skipping unwanted folders importing from mbox files.
* GODT-769 Close connection before deleting labels to prevent panics accessing deleted bucket.
### Removed
* GODT-766 Remove GUI popup for IMAP TLS error.
## [Bridge 1.5.0] Golden Gate
### Changed
* Updated go-mbox dependency back to upstream.
### Fixed
* GODT-847 Waiting for unilateral update during deleting the message.
* GODT-849 Show in error counts in the end also lost messages.
* GODT-835 Do not include conversation ID in references to show properly conversation threads in clients.
* GODT-685 Improve deb packaging regarding dejavu font
## [IE 1.2.0] Elbe
### Added
* GODT-763 Detect Gmail labels from All Mail mbox export (using X-Gmail-Label header).
* GODT-834 Info about tags in BUILDS.md and link to Import-Export page in README.md.
* GODT-777 Support Apple Mail MBOX export format.
### Fixed
* GODT-677 Windows IE: global import settings not fit in window.
* GODT-794 Congo fails to update to Danube
* GODT-749 Don't force PGP/Inline when sending plaintext messages.
* GODT-764 Fix deadlock in integration tests for Import-Export.
* GODT-662 Do not resume paused transfer progress after dismissing cancel popup.
* GODT-772 Sanitize mailbox names for exporting to follow OS restrictions.
* GODT-771 Show fatal errors after export is terminated.
* GODT-779 Do not propagate updates when progress is stopped.
* GODT-779 Unpause progress during fatal error to properly stop progress.
* GODT-779 Stop ongoing transfer calls sooner (re-check after import request is generated).
* Fix measurement of uploading attachments during transfer.
* GODT-827 Do not spam sentry with bad ID by integration test.
* GODT-700 Fix UTF-7 incompatibility.
* GODT-837 Fix flaky TestFailUnpauseAndStops.
* GODT-782 Don't use TLS pinning when checking connectivity status.
### Changed
* TLS pins conform to official list.
## [Bridge 1.4.5] Forth
### Fixed
* GODT-829 Remove `NoInferior` to display sub-folders in apple mail.
## [Bridge 1.4.4] Forth
### Fixed
* GODT-798 Replace, don't add, transfer encoding when making body 7-bit clean.
* Move/Copy duplicate for emails with References in Outlook
* CSB-247 Cannot update from 1.4.0
## [Bridge 1.4.3] Forth
### Changed
* Reverted sending IMAP updates to be not blocking again.
### Fixed
* GODT-783 Settings flags by FLAGS (not using +/-FLAGS) do not change spam state.
## [Bridge 1.4.2] Forth
### Changed
* GODT-761 Use label.Path instead of Name to partially support subfolders for webapp beta release.
* GODT-765 Improve speed of checking whether message is deleted.
## [IE 1.1.2] Danube (beta 2020-09-xx)
### Fixed
* GODT-770 Better handling of extraneous end-of-mail indicator.
* GODT-776 Fix crash when IMAP client connects while account is logging in.
### Changed
* Bump crypto version to v0.0.0-20200818122824-ed5d25e28db8
* GODT-785 Clear separation of different message IDs in integration tests.
### Changed
* GODT-741 Import-Export shows "Unable to parse time" notice instead of zero time in error report window.
* Bump crypto version to v0.0.0-20200818122824-ed5d25e28db8.
* GODT-374 Allow to send calendar update multiple times.
## [IE 1.1.1] Danube (beta 2020-09-xx) [Bridge 1.4.1] Forth (beta 2020-09-xx)
@ -11,11 +105,15 @@ Changelog [format](http://keepachangelog.com/en/1.0.0/)
* GODT-752 Parsing non-utf8 multipart/alternative message.
* GODT-752 Parsing message with duplicate charset parameter.
## [IE 1.1.0] Danube
### Fixed
* GODT-703 Import-Export showed always at least one total message.
* GODT-738 Fix for mbox files with long lines.
### Fixed
* GODT-732 Do not mix font awesome icon with regular text to avoid issues on Fedora.
## [Bridge 1.4.0] Forth

View File

@ -10,8 +10,8 @@ TARGET_OS?=${GOOS}
.PHONY: build build-ie build-nogui build-ie-nogui check-has-go
# Keep version hardcoded so app build works also without Git repository.
BRIDGE_APP_VERSION?=1.4.0-git
IE_APP_VERSION?=1.1.0-git
BRIDGE_APP_VERSION?=1.5.0-git
IE_APP_VERSION?=1.2.2-git
APP_VERSION:=${BRIDGE_APP_VERSION}
SRC_ICO:=logo.ico
SRC_ICNS:=Bridge.icns
@ -57,7 +57,6 @@ ifeq "${TARGET_CMD}" "Import-Export"
TGZ_TARGET:=ie_${TARGET_OS}_${REVISION}.tgz
endif
build: ${TGZ_TARGET}
build-ie:
TARGET_CMD=Import-Export $(MAKE) build
@ -265,7 +264,6 @@ run-ie-qt:
run-ie-nogui:
TARGET_CMD=Import-Export $(MAKE) run-nogui
clean-frontend-qt:
$(MAKE) -C internal/frontend/qt -f Makefile.local clean
clean-frontend-qt-ie:

View File

@ -1,7 +1,7 @@
# ProtonMail Bridge and Import Export app
Copyright (c) 2020 Proton Technologies AG
This repository holds the ProtonMail Bridge application.
This repository holds the ProtonMail Bridge and the ProtonMail Import-Export applications.
For a detailed build information see [BUILDS](./BUILDS.md).
For licensing information see [COPYING](./COPYING.md).
For contribution policy see [CONTRIBUTING](./CONTRIBUTING.md).
@ -35,6 +35,8 @@ configure transfer rules (match source and target mailboxes, set time
range limits and so on) and hit start. Once the transfer is complete,
check the results.
More details [on the public website](https://protonmail.com/import-export).
## Keychain
You need to have a keychain in order to run the ProtonMail Bridge. On Mac or
Windows, Bridge uses native credential managers. On Linux, use

10
go.mod
View File

@ -18,6 +18,7 @@ require (
github.com/ProtonMail/go-apple-mobileconfig v0.0.0-20160701194735-7ea9927a11f6
github.com/ProtonMail/go-autostart v0.0.0-20181114175602-c5272053443a
github.com/ProtonMail/go-imap-id v0.0.0-20190926060100-f94a56b9ecde
github.com/ProtonMail/go-rfc5322 v0.2.0
github.com/ProtonMail/go-vcard v0.0.0-20180326232728-33aaa0a0c8a5
github.com/ProtonMail/gopenpgp/v2 v2.0.1
github.com/PuerkitoBio/goquery v1.5.1
@ -35,7 +36,7 @@ require (
github.com/emersion/go-imap-quota v0.0.0-20200423100218-dcfd1b7d2b41
github.com/emersion/go-imap-specialuse v0.0.0-20200722111535-598ff00e4075
github.com/emersion/go-imap-unselect v0.0.0-20171113212723-b985794e5f26
github.com/emersion/go-mbox v1.0.0
github.com/emersion/go-mbox v1.0.2
github.com/emersion/go-message v0.12.1-0.20200903165315-e1abe21f389a
github.com/emersion/go-sasl v0.0.0-20200509203442-7bfe0ed36a21
github.com/emersion/go-textwrapper v0.0.0-20160606182133-d0e65e56babe
@ -59,7 +60,7 @@ require (
github.com/nsf/jsondiff v0.0.0-20200515183724-f29ed568f4ce
github.com/olekukonko/tablewriter v0.0.4 // indirect
github.com/pkg/errors v0.9.1
github.com/sirupsen/logrus v1.6.0
github.com/sirupsen/logrus v1.7.0
github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966
github.com/ssor/bom v0.0.0-20170718123548-6386211fdfcf // indirect
github.com/stretchr/testify v1.6.1
@ -74,9 +75,8 @@ require (
replace (
github.com/docker/docker-credential-helpers => github.com/ProtonMail/docker-credential-helpers v1.1.0
github.com/emersion/go-imap => github.com/ProtonMail/go-imap v0.0.0-20200828124548-d04b0dc1f399
github.com/emersion/go-mbox => github.com/ProtonMail/mbox v0.0.0-20200918064939-909a18c9af45
github.com/emersion/go-imap => github.com/ProtonMail/go-imap v0.0.0-20201102134601-418cd74e9474
github.com/emersion/go-smtp => github.com/ProtonMail/go-smtp v0.0.0-20181206232543-8261df20d309
github.com/jameskeane/bcrypt => github.com/ProtonMail/bcrypt v0.0.0-20170924085257-7509ea014998
golang.org/x/crypto => github.com/ProtonMail/crypto v0.0.0-20200416114516-1fa7f403fb9c
golang.org/x/crypto => github.com/ProtonMail/crypto v0.0.0-20200818122824-ed5d25e28db8
)

27
go.sum
View File

@ -1,13 +1,12 @@
github.com/0xAX/notificator v0.0.0-20191016112426-3962a5ea8da1 h1:j9HaafapDbPbGRDku6e/HRs6KBMcKHiWcm1/9Sbxnl4=
github.com/0xAX/notificator v0.0.0-20191016112426-3962a5ea8da1/go.mod h1:NtXa9WwQsukMHZpjNakTTz0LArxvGYdPA9CjIcUSZ6s=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/Masterminds/semver v1.5.0 h1:H65muMkzWKEuNDnfl9d70GUjFniHKHRbFPGBuZ3QEww=
github.com/Masterminds/semver/v3 v3.1.0 h1:Y2lUDsFKVRSYGojLJ1yLxSXdMmMYTYls0rCvoqmMUQk=
github.com/Masterminds/semver/v3 v3.1.0/go.mod h1:VPu/7SZ7ePZ3QOrcuXROw5FAcLl4a0cBrbBpGY/8hQs=
github.com/ProtonMail/bcrypt v0.0.0-20170924085257-7509ea014998 h1:YT2uVwQiRQZxCaaahwfcgTq2j3j66w00n/27gb/zubs=
github.com/ProtonMail/bcrypt v0.0.0-20170924085257-7509ea014998/go.mod h1:HecWFHognK8GfRDGnFQbW/LiV7A3MX3gZVs45vk5h8I=
github.com/ProtonMail/crypto v0.0.0-20200416114516-1fa7f403fb9c h1:DAvlgde2Stu18slmjwikiMPs/CKPV35wSvmJS34z0FU=
github.com/ProtonMail/crypto v0.0.0-20200416114516-1fa7f403fb9c/go.mod h1:Pxr7w4gA2ikI4sWyYwEffm+oew1WAJHzG1SiDpQMkrI=
github.com/ProtonMail/crypto v0.0.0-20200818122824-ed5d25e28db8 h1:u1j0xLTrCHpNS40B6m4Sv3IVUz5m9jt+AnTIopT3IgM=
github.com/ProtonMail/crypto v0.0.0-20200818122824-ed5d25e28db8/go.mod h1:Pxr7w4gA2ikI4sWyYwEffm+oew1WAJHzG1SiDpQMkrI=
github.com/ProtonMail/docker-credential-helpers v1.1.0 h1:+kvUIpwWcbtP3WFv5sSvkFn/XLzSqPOB5AAthuk9xPk=
github.com/ProtonMail/docker-credential-helpers v1.1.0/go.mod h1:mK0aBveCxhnQ756AmaTfXMZDeULvheYVhF/MWMErN5g=
github.com/ProtonMail/go-appdir v1.1.0 h1:9hdNDlU9kTqRKVNzmoqah8qqrj5QZyLByQdwQNlFWig=
@ -16,20 +15,20 @@ github.com/ProtonMail/go-apple-mobileconfig v0.0.0-20160701194735-7ea9927a11f6 h
github.com/ProtonMail/go-apple-mobileconfig v0.0.0-20160701194735-7ea9927a11f6/go.mod h1:EtDfBMIDWmVe4viZCuBTEfe3OIIo0ghbpOaAZVO+hVg=
github.com/ProtonMail/go-autostart v0.0.0-20181114175602-c5272053443a h1:fXK2KsfnkBV9Nh+9SKzHchYjuE9s0vI20JG1mbtEAcc=
github.com/ProtonMail/go-autostart v0.0.0-20181114175602-c5272053443a/go.mod h1:oTGdE7/DlWIr23G0IKW3OXK9wZ5Hw1GGiaJFccTvZi4=
github.com/ProtonMail/go-imap v0.0.0-20200828124548-d04b0dc1f399 h1:wBo/Xgb/Dn2loU47D+PJaOoIZ67i3AqYp51gLn8YE5U=
github.com/ProtonMail/go-imap v0.0.0-20200828124548-d04b0dc1f399/go.mod h1:yKASt+C3ZiDAiCSssxg9caIckWF/JG7ZQTO7GAmvicU=
github.com/ProtonMail/go-imap v0.0.0-20201102134601-418cd74e9474 h1:D0RwDtkBw0Gt7hmbb1ivdEulplJAwu1i2jzh4HM45fo=
github.com/ProtonMail/go-imap v0.0.0-20201102134601-418cd74e9474/go.mod h1:yKASt+C3ZiDAiCSssxg9caIckWF/JG7ZQTO7GAmvicU=
github.com/ProtonMail/go-imap-id v0.0.0-20190926060100-f94a56b9ecde h1:5koQozTDELymYOyFbQ/VSubexAEXzDR8qGM5mO8GRdw=
github.com/ProtonMail/go-imap-id v0.0.0-20190926060100-f94a56b9ecde/go.mod h1:795VPXcRUIQ9JyMNHP4el582VokQfippgjkQP3Gk0r0=
github.com/ProtonMail/go-mime v0.0.0-20190923161245-9b5a4261663a h1:W6RrgN/sTxg1msqzFFb+G80MFmpjMw61IU+slm+wln4=
github.com/ProtonMail/go-mime v0.0.0-20190923161245-9b5a4261663a/go.mod h1:NYt+V3/4rEeDuaev/zw1zCq8uqVEuPHzDPo3OZrlGJ4=
github.com/ProtonMail/go-rfc5322 v0.2.0 h1:tndoDGFtiCvESta9KLUeMksojz8qf76PefnkoQ+fqeg=
github.com/ProtonMail/go-rfc5322 v0.2.0/go.mod h1:mzZWlMWnQJuYLL7JpzuPF5+FimV2lZ9f0jeq24kJjpU=
github.com/ProtonMail/go-smtp v0.0.0-20181206232543-8261df20d309 h1:2pzfKjhBjSnw3BgmfTYRFQr1rFGxhfhUY0KKkg+RYxE=
github.com/ProtonMail/go-smtp v0.0.0-20181206232543-8261df20d309/go.mod h1:6UoBvDAMA/cTBwS3Y7tGpKnY5RH1F1uYHschT6eqAkI=
github.com/ProtonMail/go-vcard v0.0.0-20180326232728-33aaa0a0c8a5 h1:Uga1DHFN4GUxuDQr0F71tpi8I9HqPIlZodZAI1lR6VQ=
github.com/ProtonMail/go-vcard v0.0.0-20180326232728-33aaa0a0c8a5/go.mod h1:oeP9CMN+ajWp5jKp1kue5daJNwMMxLF+ujPaUIoJWlA=
github.com/ProtonMail/gopenpgp/v2 v2.0.1 h1:x0uvDhry5WzoHeJO4J3dgMLhG4Z9PeBJ2O+sDOY0LcU=
github.com/ProtonMail/gopenpgp/v2 v2.0.1/go.mod h1:wQQCJo7DURO6S9VwH+kSDEYs/B63yZnAEfGlOg8YNBY=
github.com/ProtonMail/mbox v0.0.0-20200918064939-909a18c9af45 h1:GDh55hDI2sNiirDqEWV8b6EB729u78Qxu3nKF970n6g=
github.com/ProtonMail/mbox v0.0.0-20200918064939-909a18c9af45/go.mod h1:Yp9IVuuOYLEuMv4yjgDHvhb5mHOcYH6x92Oas3QqEZI=
github.com/PuerkitoBio/goquery v1.5.1 h1:PSPBGne8NIUWw+/7vFBV+kG2J/5MOjbzc7154OaKCSE=
github.com/PuerkitoBio/goquery v1.5.1/go.mod h1:GsLWisAFVj4WgDibEWF4pvYnkVQBpKBKeU+7zCJoLcc=
github.com/abiosoft/ishell v2.0.0+incompatible h1:zpwIuEHc37EzrsIYah3cpevrIc8Oma7oZPxr03tlmmw=
@ -40,6 +39,8 @@ github.com/allan-simon/go-singleinstance v0.0.0-20160830203053-79edcfdc2dfc h1:m
github.com/allan-simon/go-singleinstance v0.0.0-20160830203053-79edcfdc2dfc/go.mod h1:qqsTQiwdyqxU05iDCsi0oN3P4nrVxAmn8xCtODDSf/U=
github.com/andybalholm/cascadia v1.1.0 h1:BuuO6sSfQNFRu1LppgbD25Hr2vLYW25JvxHs5zzsLTo=
github.com/andybalholm/cascadia v1.1.0/go.mod h1:GsXiBklL0woXo1j/WYWtSYYC4ouU9PqHO0sqidkEA4Y=
github.com/antlr/antlr4 v0.0.0-20201029161626-9a95f0cc3d7c h1:j/C2kxPfyE0d87/ggAjIsCV5Cdkqmjb+O0W8W+1J+IY=
github.com/antlr/antlr4 v0.0.0-20201029161626-9a95f0cc3d7c/go.mod h1:T7PbCXFs94rrTttyxjbyT5+/1V8T2TYDejxUfHJjw1Y=
github.com/certifi/gocertifi v0.0.0-20200211180108-c7c1fbc02894 h1:JLaf/iINcLyjwbtTsCJjc6rtlASgHeIJPrB6QmwURnA=
github.com/certifi/gocertifi v0.0.0-20200211180108-c7c1fbc02894/go.mod h1:sGbDF6GwGcLpkNXPUTkMRoywsNa/ol15pxFe6ERfguA=
github.com/chzyer/logex v1.1.10 h1:Swpa1K6QvQznwJRcfTfQJmTE72DqScAa40E+fbHEXEE=
@ -68,8 +69,8 @@ github.com/emersion/go-imap-specialuse v0.0.0-20200722111535-598ff00e4075 h1:z8T
github.com/emersion/go-imap-specialuse v0.0.0-20200722111535-598ff00e4075/go.mod h1:/nybxhI8kXom8Tw6BrHMl42usALvka6meORflnnYwe4=
github.com/emersion/go-imap-unselect v0.0.0-20171113212723-b985794e5f26 h1:FiSb8+XBQQSkcX3ubr+1tAtlRJBYaFmRZqOAweZ9Wy8=
github.com/emersion/go-imap-unselect v0.0.0-20171113212723-b985794e5f26/go.mod h1:+gnnZx3Mg3MnCzZrv0eZdp5puxXQUgGT/6N6L7ShKfM=
github.com/emersion/go-mbox v1.0.0 h1:HN6aKbyqmgIfK9fS/gen+NRr2wXLSxZXWfdAIAnzQPc=
github.com/emersion/go-mbox v1.0.0/go.mod h1:Yp9IVuuOYLEuMv4yjgDHvhb5mHOcYH6x92Oas3QqEZI=
github.com/emersion/go-mbox v1.0.2 h1:tE/rT+lEugK9y0myEymCCHnwlZN04hlXPrbKkxRBA5I=
github.com/emersion/go-mbox v1.0.2/go.mod h1:Yp9IVuuOYLEuMv4yjgDHvhb5mHOcYH6x92Oas3QqEZI=
github.com/emersion/go-message v0.11.1/go.mod h1:C4jnca5HOTo4bGN9YdqNQM9sITuT3Y0K6bSUw9RklvY=
github.com/emersion/go-message v0.12.1-0.20200903165315-e1abe21f389a h1:3C6qIGgPr1qAT0ikRD5NbyKpME/iHCDeXhpv/JJsFsE=
github.com/emersion/go-message v0.12.1-0.20200903165315-e1abe21f389a/go.mod h1:kYIioST9GDHte9/BRWgi93rpqbDuFftMjKSMaXS8ABo=
@ -101,8 +102,6 @@ github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/U
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-multierror v1.1.0 h1:B9UzwGQJehnUY1yNrnwREHc3fGbC2xefo8g4TbElacI=
github.com/hashicorp/go-multierror v1.1.0/go.mod h1:spPvp8C1qA32ftKqdAHm4hHTbPw+vmowP0z+KUhOZdA=
github.com/jameshoulahan/go-imap v0.0.0-20200728140727-d57327f48843 h1:suxlO4AC4E4bjueAsL0m+qp8kmkxRWMGj+5bBU/KJ8g=
github.com/jameshoulahan/go-imap v0.0.0-20200728140727-d57327f48843/go.mod h1:yKASt+C3ZiDAiCSssxg9caIckWF/JG7ZQTO7GAmvicU=
github.com/jaytaylor/html2text v0.0.0-20200412013138-3577fbdbcff7 h1:g0fAGBisHaEQ0TRq1iBvemFRf+8AEWEmBESSiWB3Vsc=
github.com/jaytaylor/html2text v0.0.0-20200412013138-3577fbdbcff7/go.mod h1:CVKlgaMiht+LXvHG173ujK6JUhZXKb2u/BQtjPDIvyk=
github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0 h1:iQTw/8FWTuc7uiaSepXwyf3o52HaUYcV+Tu66S3F5GA=
@ -112,8 +111,6 @@ github.com/keybase/go-keychain v0.0.0-20200502122510-cda31fe0c86d/go.mod h1:W6Eb
github.com/keybase/go.dbus v0.0.0-20200324223359-a94be52c0b03/go.mod h1:a8clEhrrGV/d76/f9r2I41BwANMihfZYV9C223vaxqE=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.3 h1:CE8S1cTafDpPvMhIxNJKvHsGVBgn1xWYf1NbHQhywc8=
github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
@ -151,8 +148,8 @@ github.com/shurcooL/sanitized_anchor_name v1.0.0 h1:PdmoCO6wvbs+7yrJyMORt4/BmY5I
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/sirupsen/logrus v1.6.0 h1:UBcNElsrwanuuMsnGSlYmtmgbb23qDR5dG+6X6Oo89I=
github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88=
github.com/sirupsen/logrus v1.7.0 h1:ShrD1U9pZB12TX0cVy0DtePoCH97K8EtX+mg7ZARUtM=
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966 h1:JIAuq3EEf9cgbU6AtGPK4CTG3Zf6CKMNqf0MHTggAUA=
github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966/go.mod h1:sUM3LWHvSMaG192sy56D9F7CNvL7jUJVXoqM1QKLnog=
github.com/ssor/bom v0.0.0-20170718123548-6386211fdfcf h1:pvbZ0lM0XWPBqUKqFU8cmavspvIl9nulOYwdy6IFRRo=

View File

@ -15,8 +15,8 @@
// You should have received a copy of the GNU General Public License
// along with ProtonMail Bridge. If not, see <https://www.gnu.org/licenses/>.
// Code generated by ./credits.sh at Wed Sep 16 16:48:58 CEST 2020. DO NOT EDIT.
// Code generated by ./credits.sh at Wed Nov 4 13:57:47 CET 2020. DO NOT EDIT.
package bridge
const Credits = "github.com/0xAX/notificator;github.com/ProtonMail/bcrypt;github.com/ProtonMail/crypto;github.com/ProtonMail/docker-credential-helpers;github.com/ProtonMail/go-appdir;github.com/ProtonMail/go-apple-mobileconfig;github.com/ProtonMail/go-autostart;github.com/ProtonMail/go-imap;github.com/ProtonMail/go-imap-id;github.com/ProtonMail/go-smtp;github.com/ProtonMail/go-vcard;github.com/ProtonMail/gopenpgp/v2;github.com/PuerkitoBio/goquery;github.com/abiosoft/ishell;github.com/abiosoft/readline;github.com/allan-simon/go-singleinstance;github.com/certifi/gocertifi;github.com/chzyer/logex;github.com/chzyer/test;github.com/cucumber/godog;github.com/docker/docker-credential-helpers;github.com/emersion/go-imap;github.com/emersion/go-imap-appendlimit;github.com/emersion/go-imap-idle;github.com/emersion/go-imap-move;github.com/emersion/go-imap-quota;github.com/emersion/go-imap-specialuse;github.com/emersion/go-imap-unselect;github.com/emersion/go-mbox;github.com/emersion/go-message;github.com/emersion/go-sasl;github.com/emersion/go-smtp;github.com/emersion/go-textwrapper;github.com/emersion/go-vcard;github.com/fatih/color;github.com/flynn-archive/go-shlex;github.com/getsentry/raven-go;github.com/go-resty/resty/v2;github.com/golang/mock;github.com/google/go-cmp;github.com/google/uuid;github.com/gopherjs/gopherjs;github.com/hashicorp/go-multierror;github.com/jameskeane/bcrypt;github.com/jaytaylor/html2text;github.com/kardianos/osext;github.com/keybase/go-keychain;github.com/logrusorgru/aurora;github.com/mattn/go-runewidth;github.com/miekg/dns;github.com/myesui/uuid;github.com/nsf/jsondiff;github.com/olekukonko/tablewriter;github.com/pkg/errors;github.com/sirupsen/logrus;github.com/skratchdot/open-golang;github.com/ssor/bom;github.com/stretchr/testify;github.com/therecipe/qt;github.com/twinj/uuid;github.com/urfave/cli;go.etcd.io/bbolt;golang.org/x/crypto;golang.org/x/net;golang.org/x/text;gopkg.in/stretchr/testify.v1;;Font Awesome 4.7.0;;Qt 5.13 by Qt group;"
const Credits = "github.com/0xAX/notificator;github.com/Masterminds/semver/v3;github.com/ProtonMail/bcrypt;github.com/ProtonMail/crypto;github.com/ProtonMail/docker-credential-helpers;github.com/ProtonMail/go-appdir;github.com/ProtonMail/go-apple-mobileconfig;github.com/ProtonMail/go-autostart;github.com/ProtonMail/go-imap;github.com/ProtonMail/go-imap-id;github.com/ProtonMail/go-smtp;github.com/ProtonMail/go-vcard;github.com/ProtonMail/gopenpgp/v2;github.com/PuerkitoBio/goquery;github.com/abiosoft/ishell;github.com/abiosoft/readline;github.com/allan-simon/go-singleinstance;github.com/antlr/antlr4;github.com/certifi/gocertifi;github.com/chzyer/logex;github.com/chzyer/test;github.com/cucumber/godog;github.com/docker/docker-credential-helpers;github.com/emersion/go-imap;github.com/emersion/go-imap-appendlimit;github.com/emersion/go-imap-idle;github.com/emersion/go-imap-move;github.com/emersion/go-imap-quota;github.com/emersion/go-imap-specialuse;github.com/emersion/go-imap-unselect;github.com/emersion/go-mbox;github.com/emersion/go-message;github.com/emersion/go-sasl;github.com/emersion/go-smtp;github.com/emersion/go-textwrapper;github.com/emersion/go-vcard;github.com/fatih/color;github.com/flynn-archive/go-shlex;github.com/getsentry/raven-go;github.com/go-resty/resty/v2;github.com/golang/mock;github.com/google/go-cmp;github.com/google/uuid;github.com/gopherjs/gopherjs;github.com/hashicorp/go-multierror;github.com/jameskeane/bcrypt;github.com/jaytaylor/html2text;github.com/kardianos/osext;github.com/keybase/go-keychain;github.com/logrusorgru/aurora;github.com/mattn/go-runewidth;github.com/miekg/dns;github.com/myesui/uuid;github.com/nsf/jsondiff;github.com/olekukonko/tablewriter;github.com/pkg/errors;github.com/sirupsen/logrus;github.com/skratchdot/open-golang;github.com/ssor/bom;github.com/stretchr/testify;github.com/therecipe/qt;github.com/twinj/uuid;github.com/urfave/cli;go.etcd.io/bbolt;golang.org/x/crypto;golang.org/x/net;golang.org/x/text;gopkg.in/stretchr/testify.v1;;Font Awesome 4.7.0;;Qt 5.13 by Qt group;"

View File

@ -15,21 +15,17 @@
// You should have received a copy of the GNU General Public License
// along with ProtonMail Bridge. If not, see <https://www.gnu.org/licenses/>.
// Code generated by ./release-notes.sh at 'Mon Sep 21 01:29:10 PM CEST 2020'. DO NOT EDIT.
// Code generated by ./release-notes.sh at 'Wed Nov 4 12:24:35 PM CET 2020'. DO NOT EDIT.
package bridge
const ReleaseNotes = `Bulletproofing against any potential data loss and/or duplication
Performance improvements for handling attachments and non-standard formatting
• Better stability of the message parser
Additional foreign encoding support for outgoing messages
• Complete refactor of the way messages are parsed to simplify code maintenance
• Improved User-Agent detection
• Added MacOS Big Sur compatibility
• Added persistent anonymous API cookies
const ReleaseNotes = `Ensured better message flow by refactoring both address and date parsing
Improved secure connectivity checks
• Better deb packaging
More robust error handling
`
const ReleaseFixedBugs = `Fixed rare mail loss when moving from Spam folder
Limited log size
Fixed Linux font issues (mouse hover).
const ReleaseFixedBugs = `Ensured that conversations are properly threaded
Fixed Linux font issues (Fedora)
Better handling of Mime encrypted messages
`

View File

@ -40,7 +40,6 @@ const (
NoActiveKeyForRecipientEvent = "noActiveKeyForRecipient"
UpgradeApplicationEvent = "upgradeApplication"
TLSCertIssue = "tlsCertPinningIssue"
IMAPTLSBadCert = "imapTLSBadCert"
// LogoutEventTimeout is the minimum time to permit between logout events being sent.
LogoutEventTimeout = 3 * time.Minute

View File

@ -184,9 +184,17 @@ func (f *frontendCLI) setTransferRules(t *transfer.Transfer) bool {
}
func (f *frontendCLI) printTransferProgress(progress *transfer.Progress) {
failed, imported, exported, added, total := progress.GetCounts()
if total != 0 {
f.Println(fmt.Sprintf("Progress update: %d (%d / %d) / %d, failed: %d", imported, exported, added, total, failed))
counts := progress.GetCounts()
if counts.Total != 0 {
f.Println(fmt.Sprintf(
"Progress update: %d (%d / %d) / %d, skipped: %d, failed: %d",
counts.Imported,
counts.Exported,
counts.Added,
counts.Total,
counts.Skipped,
counts.Failed,
))
}
if progress.IsPaused() {

View File

@ -48,6 +48,7 @@ Item {
text : qsTr("Clear", "clickable link next to clear cache button in settings")
color: Style.main.text
font {
family : cacheClear.font.family // use default font, not font-awesome
pointSize : Style.settings.fontSize * Style.pt
underline : true
}
@ -66,6 +67,7 @@ Item {
text : qsTr("Clear", "clickable link next to clear keychain button in settings")
color: Style.main.text
font {
family : cacheKeychain.font.family // use default font, not font-awesome
pointSize : Style.settings.fontSize * Style.pt
underline : true
}
@ -125,6 +127,7 @@ Item {
text : qsTr("Change", "clickable link next to change ports button in settings")
color: Style.main.text
font {
family : changePort.font.family // use default font, not font-awesome
pointSize : Style.settings.fontSize * Style.pt
underline : true
}

View File

@ -237,13 +237,6 @@ Item {
winMain.tlsBarState="notOK"
}
onShowIMAPCertTroubleshoot : {
go.notifyBubble(1, qsTr(
"Bridge was unable to establish a connection with your Email client. <br> <a href=\"https://protonmail.com/support/knowledge-base/bridge-ssl-connection-issue\">Learn more</a> <br>",
"notification message"
))
}
}

View File

@ -276,6 +276,10 @@ Item {
winMain.dialogExport.hide()
}
}
onUpdateFinished : {
winMain.dialogUpdate.finished(hasError)
}
}
function folderIcon(folderName, folderType) { // translations

View File

@ -217,7 +217,10 @@ Dialog {
Text {
anchors.centerIn: parent
text: {
if (progressbarExport.isFinished) return qsTr("Export finished","todo")
if (progressbarExport.isFinished) {
if (go.progressDescription=="") return qsTr("Export finished","todo")
else return qsTr("Export failed: %1").arg(go.progressDescription)
}
if (
go.progressDescription == gui.enums.progressInit ||
(go.progress==0 && go.description=="")
@ -450,7 +453,6 @@ Dialog {
errorPopup.hide()
}
onClickedNo : {
go.resumeProcess()
errorPopup.hide()
}
}

View File

@ -279,9 +279,8 @@ Dialog {
titleTo : root.address
}
Rectangle {
Column {
id: masterImportSettings
height: 150 // fixme
anchors {
right : parent.right
left : parent.left
@ -291,45 +290,47 @@ Dialog {
rightMargin : Style.main.leftMargin
bottomMargin : Style.main.bottomMargin
}
color: Style.dialog.background
Text {
id: labelMasterImportSettings
text: qsTr("Master import settings:")
spacing: Style.main.bottomMargin
font {
bold: true
family: Style.fontawesome.name
pointSize: Style.main.fontSize * Style.pt
}
color: Style.main.text
Row {
spacing: masterImportSettings.width - labelMasterImportSettings.width - resetSourceButton.width
InfoToolTip {
info: qsTr(
"If master import date range is selected only emails within this range will be imported, unless it is specified differently in folder date range.",
"Text in master import settings tooltip."
)
anchors {
left: parent.right
bottom: parent.bottom
leftMargin : Style.dialog.leftMargin
Text {
id: labelMasterImportSettings
text: qsTr("Master import settings:")
font {
bold: true
family: Style.fontawesome.name
pointSize: Style.main.fontSize * Style.pt
}
color: Style.main.text
InfoToolTip {
anchors {
left: parent.right
bottom: parent.bottom
leftMargin : Style.dialog.leftMargin
}
info: qsTr(
"If master import date range is selected only emails within this range will be imported, unless it is specified differently in folder date range.",
"Text in master import settings tooltip."
)
}
}
}
// Reset all to default
ClickIconText {
anchors {
right: parent.right
bottom: labelMasterImportSettings.bottom
}
text:qsTr("Reset all settings to default")
iconText: Style.fa.refresh
textColor: Style.main.textBlue
onClicked: {
go.resetSource()
root.decrementCurrentIndex()
timer.start()
// Reset all to default
ClickIconText {
id: resetSourceButton
text:qsTr("Reset all settings to default")
iconText: Style.fa.refresh
textColor: Style.main.textBlue
onClicked: {
go.resetSource()
root.decrementCurrentIndex()
timer.start()
}
}
}
@ -348,49 +349,40 @@ Dialog {
InlineDateRange {
id: globalDateRange
anchors {
left : parent.left
top : line.bottom
topMargin : Style.dialog.topMargin
}
}
// Add global label (inline)
InlineLabelSelect {
id: globalLabels
anchors {
left : parent.left
top : globalDateRange.bottom
topMargin : Style.dialog.topMargin
}
//labelWidth : globalDateRange.labelWidth
}
}
// Buttons
Row {
spacing: Style.dialog.spacing
anchors {
right: parent.right
bottom: parent.bottom
rightMargin: Style.main.leftMargin
bottomMargin: Style.main.bottomMargin
}
// Buttons
Row {
spacing: Style.dialog.spacing
anchors{
bottom : parent.bottom
right : parent.right
}
ButtonRounded {
id: buttonCancelThree
fa_icon : Style.fa.times
text : qsTr("Cancel", "todo")
color_main : Style.dialog.textBlue
onClicked : root.cancel()
}
ButtonRounded {
id: buttonCancelThree
fa_icon : Style.fa.times
text : qsTr("Cancel", "todo")
color_main : Style.dialog.textBlue
onClicked : root.cancel()
}
ButtonRounded {
id: buttonNextThree
fa_icon : Style.fa.check
text : qsTr("Import", "todo")
color_main : Style.dialog.background
color_minor : Style.dialog.textBlue
isOpaque : true
onClicked : root.okay()
}
ButtonRounded {
id: buttonNextThree
fa_icon : Style.fa.check
text : qsTr("Import", "todo")
color_main : Style.dialog.background
color_minor : Style.dialog.textBlue
isOpaque : true
onClicked : root.okay()
}
}
}
@ -483,18 +475,30 @@ Dialog {
}
}
Text {
Row {
property int fails: go.progressFails
visible: fails > 0
color : Style.main.textRed
font.family: Style.fontawesome.name
font.pointSize: Style.main.fontSize * Style.pt
anchors.horizontalCenter: parent.horizontalCenter
text: Style.fa.exclamation_circle + " " + (
fails == 1 ?
qsTr("%1 message failed to be imported").arg(fails) :
qsTr("%1 messages failed to be imported").arg(fails)
)
Text {
color: Style.main.textRed
font {
pointSize : Style.dialog.fontSize * Style.pt
family : Style.fontawesome.name
}
text: Style.fa.exclamation_circle
}
Text {
property int fails: go.progressFails
color: Style.main.textRed
font.pointSize: Style.main.fontSize * Style.pt
text: " " + (
fails == 1 ?
qsTr("%1 message failed to be imported").arg(fails) :
qsTr("%1 messages failed to be imported").arg(fails)
)
}
}
Row { // buttons
@ -575,16 +579,27 @@ Dialog {
anchors.centerIn : finalReport
spacing : Style.dialog.heightSeparator
Text {
text: go.progressDescription!="" ? qsTr("Import failed: %1").arg(go.progressDescription) : Style.fa.check_circle + " " + qsTr("Import completed successfully")
Row {
anchors.horizontalCenter: parent.horizontalCenter
color: go.progressDescription!="" ? Style.main.textRed : Style.main.textGreen
font.bold : true
font.family: Style.fontawesome.name
Text {
font {
pointSize: Style.dialog.fontSize * Style.pt
family: Style.fontawesome.name
}
color: Style.main.textGreen
text: go.progressDescription!="" ? "" : Style.fa.check_circle
}
Text {
text: go.progressDescription!="" ? qsTr("Import failed: %1").arg(go.progressDescription) : " " + qsTr("Import completed successfully")
color: go.progressDescription!="" ? Style.main.textRed : Style.main.textGreen
font.bold : true
}
}
Text {
text: qsTr("<b>Import summary:</b><br>Total number of emails: %1<br>Imported emails: %2<br>Errors: %3").arg(go.total).arg(finalReport.imported).arg(go.progressFails)
text: qsTr("<b>Import summary:</b><br>Total number of emails: %1<br>Imported emails: %2<br>Filtered out emails: %3<br>Errors: %4").arg(go.total).arg(go.progressImported).arg(go.progressSkipped).arg(go.progressFails)
anchors.horizontalCenter: parent.horizontalCenter
textFormat: Text.RichText
horizontalAlignment: Text.AlignHCenter
@ -773,11 +788,6 @@ Dialog {
errorPopup.hide()
}
onClickedNo : {
if (errorPopup.msgID == "ask_send_report") {
errorPopup.hide()
return
}
go.resumeProcess()
errorPopup.hide()
}

View File

@ -74,7 +74,7 @@ Item {
)
width: wrapper.width
color : Style.transparent
Text {
AccessibleText {
id: aboutText
anchors {
bottom: parent.bottom
@ -82,8 +82,8 @@ Item {
}
color: Style.main.textDisabled
horizontalAlignment: Qt.AlignHCenter
font.family : Style.fontawesome.name
text: "ProtonMail Import-Export app Version "+go.getBackendVersion()+"\n"+Style.fa.copyright + " 2020 Proton Technologies AG"
font.pointSize : Style.main.fontSize * Style.pt
text: "ProtonMail Import-Export app Version "+go.getBackendVersion()+"\n© 2020 Proton Technologies AG"
}
}

View File

@ -42,34 +42,50 @@ ComboBox {
root.below = popup.y>0
}
contentItem : Text {
contentItem : Row {
id: boxText
verticalAlignment: Text.AlignVCenter
font {
family: Style.fontawesome.name
pointSize : Style.dialog.fontSize * Style.pt
bold: root.down
}
elide: Text.ElideRight
textFormat: Text.StyledText
text : root.displayText
color: !root.enabled ? Style.main.textDisabled : ( root.down ? Style.main.background : Style.main.text )
Text {
anchors.verticalCenter: parent.verticalCenter
font {
pointSize: Style.dialog.fontSize * Style.pt
family: Style.fontawesome.name
}
text: {
if (view.currentIndex >= 0) {
if (!root.isFolderType) {
return Style.fa.tags + " "
}
var tgtIcon = view.currentItem.folderIcon
var tgtColor = view.currentItem.folderColor
if (tgtIcon != Style.fa.folder_open) {
return tgtIcon + " "
}
return '<font color="'+tgtColor+'">'+ tgtIcon + "</font> "
}
return ""
}
color: !root.enabled ? Style.main.textDisabled : ( root.down ? Style.main.background : Style.main.text )
}
Text {
anchors.verticalCenter: parent.verticalCenter
font {
pointSize : Style.dialog.fontSize * Style.pt
bold: root.down
}
elide: Text.ElideRight
textFormat: Text.StyledText
text : root.displayText
color: !root.enabled ? Style.main.textDisabled : ( root.down ? Style.main.background : Style.main.text )
}
}
displayText: {
if (view.currentIndex >= 0) {
if (!root.isFolderType) return Style.fa.tags + " " + qsTr("Add/Remove labels")
var tgtName = view.currentItem.folderName
var tgtIcon = view.currentItem.folderIcon
var tgtColor = view.currentItem.folderColor
if (tgtIcon != Style.fa.folder_open) {
return tgtIcon + " " + tgtName
}
return '<font color="'+tgtColor+'">'+ tgtIcon + "</font> " + tgtName
if (!root.isFolderType) return qsTr("Add/Remove labels")
return view.currentItem.folderName
}
if (root.isFolderType) return qsTr("No folder selected")
return qsTr("No labels selected")

View File

@ -44,6 +44,7 @@ Item {
text : qsTr("Clear")
color: Style.main.text
font {
family : cacheKeychain.font.family // use default font, not font-awesome
pointSize : Style.settings.fontSize * Style.pt
underline : true
}

View File

@ -84,7 +84,7 @@ Window {
height: content.height - (
(clientVersion.visible ? clientVersion.height + Style.dialog.fontSize : 0) +
userAddress.height + Style.dialog.fontSize +
securityNote.contentHeight + Style.dialog.fontSize +
securityNoteText.contentHeight + Style.dialog.fontSize +
cancelButton.height + Style.dialog.fontSize
)
clip: true
@ -215,7 +215,7 @@ Window {
}
// Note
AccessibleText {
Row {
id: securityNote
anchors {
left: parent.left
@ -223,14 +223,32 @@ Window {
top: userAddress.bottom
topMargin: Style.dialog.fontSize
}
wrapMode: Text.Wrap
color: Style.dialog.text
font.pointSize : Style.dialog.fontSize * Style.pt
text:
"<span style='font-family: " + Style.fontawesome.name + "'>" + Style.fa.exclamation_triangle + "</span> " +
qsTr("Bug reports are not end-to-end encrypted!", "The first part of warning in bug report form") + " " +
qsTr("Please do not send any sensitive information.", "The second part of warning in bug report form") + " " +
qsTr("Contact us at security@protonmail.com for critical security issues.", "The third part of warning in bug report form")
Text {
id: securityNoteIcon
font {
pointSize : Style.dialog.fontSize * Style.pt
family : Style.fontawesome.name
}
color: Style.dialog.text
text : Style.fa.exclamation_triangle
}
AccessibleText {
id: securityNoteText
anchors {
left: securityNoteIcon.right
leftMargin: 5 * Style.pt
right: parent.right
}
wrapMode: Text.Wrap
color: Style.dialog.text
font.pointSize : Style.dialog.fontSize * Style.pt
text:
qsTr("Bug reports are not end-to-end encrypted!", "The first part of warning in bug report form") + " " +
qsTr("Please do not send any sensitive information.", "The second part of warning in bug report form") + " " +
qsTr("Contact us at security@protonmail.com for critical security issues.", "The third part of warning in bug report form")
}
}
// buttons

View File

@ -122,7 +122,6 @@ Window {
ListElement { title: "Minimize this" }
ListElement { title: "SendAlertPopup" }
ListElement { title: "TLSCertError" }
ListElement { title: "IMAPCertError" }
}
ListView {
@ -209,9 +208,6 @@ Window {
case "TLSCertError" :
go.showCertIssue()
break;
case "IMAPCertError" :
go.showIMAPCertTroubleshoot()
break;
default :
console.log("Not implemented " + data)
}
@ -314,7 +310,6 @@ Window {
signal failedAutostartCode(string code)
signal showCertIssue()
signal showIMAPCertTroubleshoot()
signal updateFinished(bool hasError)

View File

@ -840,6 +840,8 @@ Window {
property real progress: 0.0
property int progressFails: 0
property int progressImported: 0
property int progressSkipped: 0
property string progressDescription: "nothing"
property string progressInit: "init"
property int total: 42
@ -1011,6 +1013,8 @@ Window {
property SequentialAnimation animateProgressBar : SequentialAnimation {
id: apb
property real speedup : 1.0;
PropertyAnimation{ target: go; properties: "progressSkipped"; to: 0; duration: 1; }
PropertyAnimation{ target: go; properties: "progressImported"; to: 0; duration: 1; }
PropertyAnimation{ target: go; properties: "importLogFileName"; to: ""; duration: 1; }
PropertyAnimation{ target: go; properties: "progressDescription"; to: go.progressInit; duration: 1; }
PropertyAnimation{ duration: 2000/apb.speedup; }
@ -1024,6 +1028,8 @@ Window {
PropertyAnimation{ target: go; properties: "progress"; to: 0.01; duration: 1; }
PropertyAnimation{ duration: 1000/apb.speedup; }
PropertyAnimation{ target: go; properties: "progress"; to: 0.1; duration: 1; }
PropertyAnimation{ target: go; properties: "progressSkipped"; to: 12; duration: 1; }
PropertyAnimation{ target: go; properties: "progressImported"; to: 13.1; duration: 1; }
PropertyAnimation{ duration: 1000/apb.speedup; }
PropertyAnimation{ target: go; properties: "progress"; to: 0.3; duration: 1; }
PropertyAnimation{ target: go; properties: "progressFails"; to: 1; duration: 1; }

View File

@ -21,6 +21,7 @@ package qtcommon
import (
"fmt"
"github.com/therecipe/qt/core"
)

View File

@ -72,6 +72,9 @@ func (e *ErrorListModel) data(index *core.QModelIndex, role int) *core.QVariant
case MailSubject:
return qtcommon.NewQVariantString(r.Subject)
case MailDate:
if r.Time.IsZero() {
return qtcommon.NewQVariantString("Unavailable")
}
return qtcommon.NewQVariantString(r.Time.String())
case MailFrom:
return qtcommon.NewQVariantString(r.From)

View File

@ -337,18 +337,25 @@ func (f *FrontendQt) setProgressManager(progress *transfer.Progress) {
if progress.IsStopped() {
break
}
failed, imported, _, _, total := progress.GetCounts()
f.Qml.SetTotal(int(total))
f.Qml.SetProgressFails(int(failed))
counts := progress.GetCounts()
f.Qml.SetTotal(int(counts.Total))
f.Qml.SetProgressImported(int(counts.Imported))
f.Qml.SetProgressSkipped(int(counts.Skipped))
f.Qml.SetProgressFails(int(counts.Failed))
f.Qml.SetProgressDescription(progress.PauseReason())
if total > 0 {
newProgress := float32(imported+failed) / float32(total)
if counts.Total > 0 {
newProgress := counts.Progress()
if newProgress >= 0 && newProgress != f.Qml.Progress() {
f.Qml.SetProgress(newProgress)
f.Qml.ProgressChanged(newProgress)
}
}
}
// Counts will add lost messages only once the progress is completeled.
counts := progress.GetCounts()
f.Qml.SetProgressImported(int(counts.Imported))
f.Qml.SetProgressSkipped(int(counts.Skipped))
f.Qml.SetProgressFails(int(counts.Failed))
if err := progress.GetFatalError(); err != nil {
f.Qml.SetProgressDescription(err.Error())

View File

@ -77,6 +77,8 @@ func (f *FrontendQt) StartImport(email string) { // TODO email not needed
log.Trace("Starting import")
f.Qml.SetProgressDescription("init") // TODO use const
f.Qml.SetProgressImported(0)
f.Qml.SetProgressSkipped(0)
f.Qml.SetProgressFails(0)
f.Qml.SetProgress(0.0)
f.Qml.SetTotal(1)

View File

@ -43,6 +43,8 @@ type GoQMLInterface struct {
_ string `property:lastError`
_ float32 `property:progress`
_ string `property:progressDescription`
_ int `property:progressImported`
_ int `property:progressSkipped`
_ int `property:progressFails`
_ int `property:total`
_ string `property:importLogFileName`

View File

@ -40,17 +40,17 @@ import (
"github.com/ProtonMail/proton-bridge/internal/bridge"
"github.com/ProtonMail/proton-bridge/internal/events"
"github.com/ProtonMail/proton-bridge/internal/frontend/autoconfig"
"github.com/ProtonMail/proton-bridge/internal/frontend/qt-common"
qtcommon "github.com/ProtonMail/proton-bridge/internal/frontend/qt-common"
"github.com/ProtonMail/proton-bridge/internal/frontend/types"
"github.com/ProtonMail/proton-bridge/internal/preferences"
"github.com/ProtonMail/proton-bridge/internal/updates"
"github.com/ProtonMail/proton-bridge/pkg/config"
"github.com/ProtonMail/proton-bridge/pkg/ports"
"github.com/ProtonMail/proton-bridge/pkg/useragent"
"github.com/ProtonMail/proton-bridge/pkg/listener"
"github.com/ProtonMail/proton-bridge/pkg/pmapi"
"github.com/sirupsen/logrus"
"github.com/ProtonMail/proton-bridge/pkg/ports"
"github.com/ProtonMail/proton-bridge/pkg/useragent"
"github.com/kardianos/osext"
"github.com/sirupsen/logrus"
"github.com/skratchdot/open-golang/open"
"github.com/therecipe/qt/core"
"github.com/therecipe/qt/gui"
@ -187,7 +187,6 @@ func (s *FrontendQt) watchEvents() {
updateApplicationCh := s.getEventChannel(events.UpgradeApplicationEvent)
newUserCh := s.getEventChannel(events.UserRefreshEvent)
certIssue := s.getEventChannel(events.TLSCertIssue)
imapCertIssue := s.getEventChannel(events.IMAPTLSBadCert)
for {
select {
case errorDetails := <-errorCh:
@ -227,8 +226,6 @@ func (s *FrontendQt) watchEvents() {
s.Qml.LoadAccounts()
case <-certIssue:
s.Qml.ShowCertIssue()
case <-imapCertIssue:
s.Qml.ShowIMAPCertTroubleshoot()
}
}
}

View File

@ -135,7 +135,6 @@ type GoQMLInterface struct {
_ func(x, y float32) `slot:"saveOutgoingNoEncPopupCoord"`
_ func(recipient string) `signal:"showNoActiveKeyForRecipient"`
_ func() `signal:"showCertIssue"`
_ func() `signal:"ShowIMAPCertTroubleshoot"`
_ func() `slot:"startUpdate"`
_ func(hasError bool) `signal:"updateFinished"`

View File

@ -28,7 +28,6 @@ import (
"github.com/ProtonMail/proton-bridge/pkg/listener"
"github.com/emersion/go-imap"
goIMAPBackend "github.com/emersion/go-imap/backend"
"github.com/sirupsen/logrus"
)
type panicHandler interface {
@ -198,11 +197,3 @@ func (ib *imapBackend) monitorDisconnectedUsers() {
ib.deleteUser(address)
}
}
func (ib *imapBackend) upgradeError(err error) {
logrus.WithError(err).Error("IMAP connection couldn't be upgraded to TLS during STARTTLS")
if strings.Contains(err.Error(), "remote error: tls: bad certificate") {
ib.eventListener.Emit(events.IMAPTLSBadCert, err.Error())
}
}

View File

@ -76,5 +76,9 @@ func newBridgeUserWrap(bridgeUser *users.User) *bridgeUserWrap {
}
func (u *bridgeUserWrap) GetStore() storeUserProvider {
return newStoreUserWrap(u.User.GetStore())
store := u.User.GetStore()
if store == nil {
return nil
}
return newStoreUserWrap(store)
}

View File

@ -80,7 +80,10 @@ func (im *imapMailbox) Info() (*imap.MailboxInfo, error) {
}
func (im *imapMailbox) getFlags() []string {
flags := []string{imap.NoInferiorsAttr} // Subfolders are not yet supported by API.
flags := []string{}
if !im.storeMailbox.IsFolder() || im.storeMailbox.IsSystem() {
flags = append(flags, imap.NoInferiorsAttr) // Subfolders are not supported for System or Label
}
switch im.storeMailbox.LabelID() {
case pmapi.SentLabel:
flags = append(flags, specialuse.Sent)

View File

@ -24,7 +24,6 @@ import (
"mime/multipart"
"net/mail"
"net/textproto"
"regexp"
"sort"
"strings"
"time"
@ -141,18 +140,19 @@ func (im *imapMailbox) CreateMessage(flags []string, date time.Time, body imap.L
references := m.Header.Get("References")
referenceList := strings.Fields(references)
if len(referenceList) > 0 {
// In case there is a mail client which corrupts headers, try
// "References" too.
if internalID == "" && len(referenceList) > 0 {
lastReference := referenceList[len(referenceList)-1]
// In case we are using a mail client which corrupts headers, try "References" too.
re := regexp.MustCompile(pmapi.InternalReferenceFormat)
match := re.FindStringSubmatch(lastReference)
if len(match) > 0 {
internalID = match[0]
match := pmapi.RxInternalReferenceFormat.FindStringSubmatch(lastReference)
if len(match) == 2 {
internalID = match[1]
}
}
// Avoid appending a message which is already on the server. Apply the new
// label instead. This sometimes happens which Outlook (it uses APPEND instead of COPY).
// Avoid appending a message which is already on the server. Apply the
// new label instead. This always happens with Outlook (it uses APPEND
// instead of COPY).
if internalID != "" {
// Check to see if this belongs to a different address in split mode or another ProtonMail account.
msg, err := im.storeMailbox.GetMessage(internalID)

View File

@ -57,6 +57,10 @@ func (im *imapMailbox) UpdateMessagesFlags(uid bool, seqSet *imap.SeqSet, operat
return im.addOrRemoveFlags(operation, messageIDs, flags)
}
// setFlags is used for FLAGS command (not +FLAGS or -FLAGS), which means
// to set flags passed as an argument and unset the rest. For example,
// if message is not read, is flagged and is not deleted, call FLAGS \Seen
// should flag message as read, unflagged and keep undeleted.
func (im *imapMailbox) setFlags(messageIDs, flags []string) error { //nolint
seen := false
flagged := false
@ -106,16 +110,17 @@ func (im *imapMailbox) setFlags(messageIDs, flags []string) error { //nolint
}
}
spamMailbox, err := im.storeAddress.GetMailbox("Spam")
if err != nil {
return err
}
// Spam should not be taken into action here as Outlook is using FLAGS
// without preserving junk flag. Probably it's because junk is not standard
// in the rfc3501 and thus Outlook expects calling FLAGS \Seen will not
// change the state of junk or other non-standard flags.
// Still, its safe to label as spam once any client sends the request.
if spam {
if err := spamMailbox.LabelMessages(messageIDs); err != nil {
spamMailbox, err := im.storeAddress.GetMailbox("Spam")
if err != nil {
return err
}
} else {
if err := spamMailbox.UnlabelMessages(messageIDs); err != nil {
if err := spamMailbox.LabelMessages(messageIDs); err != nil {
return err
}
}

View File

@ -58,7 +58,6 @@ func NewIMAPServer(debugClient, debugServer bool, port int, tls *tls.Config, ima
s.AllowInsecureAuth = true
s.ErrorLog = newServerErrorLogger("server-imap")
s.AutoLogout = 30 * time.Minute
s.UpgradeError = imapBackend.upgradeError
serverID := imapid.ID{
imapid.FieldName: "ProtonMail Bridge",

View File

@ -15,8 +15,8 @@
// You should have received a copy of the GNU General Public License
// along with ProtonMail Bridge. If not, see <https://www.gnu.org/licenses/>.
// Code generated by ./credits.sh at Wed Sep 23 01:34:10 PM CEST 2020. DO NOT EDIT.
// Code generated by ./credits.sh at Wed Nov 4 13:57:47 CET 2020. DO NOT EDIT.
package importexport
const Credits = "github.com/0xAX/notificator;github.com/abiosoft/ishell;github.com/abiosoft/readline;github.com/allan-simon/go-singleinstance;github.com/certifi/gocertifi;github.com/chzyer/logex;github.com/chzyer/test;github.com/cucumber/godog;github.com/docker/docker-credential-helpers;github.com/emersion/go-imap;github.com/emersion/go-imap-appendlimit;github.com/emersion/go-imap-idle;github.com/emersion/go-imap-move;github.com/emersion/go-imap-quota;github.com/emersion/go-imap-specialuse;github.com/emersion/go-imap-unselect;github.com/emersion/go-mbox;github.com/emersion/go-message;github.com/emersion/go-sasl;github.com/emersion/go-smtp;github.com/emersion/go-textwrapper;github.com/emersion/go-vcard;github.com/fatih/color;github.com/flynn-archive/go-shlex;github.com/getsentry/raven-go;github.com/golang/mock;github.com/google/go-cmp;github.com/google/uuid;github.com/gopherjs/gopherjs;github.com/go-resty/resty/v2;github.com/hashicorp/go-multierror;github.com/jameskeane/bcrypt;github.com/jaytaylor/html2text;github.com/kardianos/osext;github.com/keybase/go-keychain;github.com/logrusorgru/aurora;github.com/Masterminds/semver/v3;github.com/mattn/go-runewidth;github.com/miekg/dns;github.com/myesui/uuid;github.com/nsf/jsondiff;github.com/olekukonko/tablewriter;github.com/pkg/errors;github.com/ProtonMail/bcrypt;github.com/ProtonMail/crypto;github.com/ProtonMail/docker-credential-helpers;github.com/ProtonMail/go-appdir;github.com/ProtonMail/go-apple-mobileconfig;github.com/ProtonMail/go-autostart;github.com/ProtonMail/go-imap;github.com/ProtonMail/go-imap-id;github.com/ProtonMail/gopenpgp/v2;github.com/ProtonMail/go-smtp;github.com/ProtonMail/go-vcard;github.com/ProtonMail/mbox;github.com/PuerkitoBio/goquery;github.com/sirupsen/logrus;github.com/skratchdot/open-golang;github.com/ssor/bom;github.com/stretchr/testify;github.com/therecipe/qt;github.com/twinj/uuid;github.com/urfave/cli;go.etcd.io/bbolt;golang.org/x/crypto;golang.org/x/net;golang.org/x/text;gopkg.in/stretchr/testify.v1;;Font Awesome 4.7.0;;Qt 5.13 by Qt group;"
const Credits = "github.com/0xAX/notificator;github.com/Masterminds/semver/v3;github.com/ProtonMail/bcrypt;github.com/ProtonMail/crypto;github.com/ProtonMail/docker-credential-helpers;github.com/ProtonMail/go-appdir;github.com/ProtonMail/go-apple-mobileconfig;github.com/ProtonMail/go-autostart;github.com/ProtonMail/go-imap;github.com/ProtonMail/go-imap-id;github.com/ProtonMail/go-smtp;github.com/ProtonMail/go-vcard;github.com/ProtonMail/gopenpgp/v2;github.com/PuerkitoBio/goquery;github.com/abiosoft/ishell;github.com/abiosoft/readline;github.com/allan-simon/go-singleinstance;github.com/antlr/antlr4;github.com/certifi/gocertifi;github.com/chzyer/logex;github.com/chzyer/test;github.com/cucumber/godog;github.com/docker/docker-credential-helpers;github.com/emersion/go-imap;github.com/emersion/go-imap-appendlimit;github.com/emersion/go-imap-idle;github.com/emersion/go-imap-move;github.com/emersion/go-imap-quota;github.com/emersion/go-imap-specialuse;github.com/emersion/go-imap-unselect;github.com/emersion/go-mbox;github.com/emersion/go-message;github.com/emersion/go-sasl;github.com/emersion/go-smtp;github.com/emersion/go-textwrapper;github.com/emersion/go-vcard;github.com/fatih/color;github.com/flynn-archive/go-shlex;github.com/getsentry/raven-go;github.com/go-resty/resty/v2;github.com/golang/mock;github.com/google/go-cmp;github.com/google/uuid;github.com/gopherjs/gopherjs;github.com/hashicorp/go-multierror;github.com/jameskeane/bcrypt;github.com/jaytaylor/html2text;github.com/kardianos/osext;github.com/keybase/go-keychain;github.com/logrusorgru/aurora;github.com/mattn/go-runewidth;github.com/miekg/dns;github.com/myesui/uuid;github.com/nsf/jsondiff;github.com/olekukonko/tablewriter;github.com/pkg/errors;github.com/sirupsen/logrus;github.com/skratchdot/open-golang;github.com/ssor/bom;github.com/stretchr/testify;github.com/therecipe/qt;github.com/twinj/uuid;github.com/urfave/cli;go.etcd.io/bbolt;golang.org/x/crypto;golang.org/x/net;golang.org/x/text;gopkg.in/stretchr/testify.v1;;Font Awesome 4.7.0;;Qt 5.13 by Qt group;"

View File

@ -15,17 +15,19 @@
// You should have received a copy of the GNU General Public License
// along with ProtonMail Bridge. If not, see <https://www.gnu.org/licenses/>.
// Code generated by ./release-notes.sh at 'Wed Sep 23 01:31:53 PM CEST 2020'. DO NOT EDIT.
// Code generated by ./release-notes.sh at 'Fri Nov 27 07:35:38 AM CET 2020'. DO NOT EDIT.
package importexport
const ReleaseNotes = `Speed up import by implementing parallel processing (parallel fetch, encrypt and upload of messages)
Optimising the initial fetch of messages from external accounts
• Better handling of attachments and non-standard formatting
Improved stability of the message parser
Added persistent anonymous API cookies
const ReleaseNotes = `Improvements to the import from large mbox files with multiple labels
Not allow to run multiple instances of the app or transfers at the same time
• Better handling and displaying of skipped messages
Various enhancements of the import process related to parsing
Cosmetic GUI changes
• Better error handling
`
const ReleaseFixedBugs = `Import from mbox files with long lines
Improvements to import from Yahoo accounts
const ReleaseFixedBugs = `Linux font issues - Fedora specific
App response to the user pausing and canceling import or export
• Upgrade errors
`

View File

@ -503,13 +503,6 @@ func (b *sendPreferencesBuilder) setEncryptionPreferences(mailSettings pmapi.Mai
}
func (b *sendPreferencesBuilder) setMIMEPreferences(composerMIMEType string) {
// If the sign flag (that we just determined above) is true we use the scheme
// in the encryption preferences, unless the plain text format has been
// selected in the composer, in which case we must enforce PGP/INLINE.
if !b.isInternal() && b.shouldSign() && composerMIMEType == "text/plain" {
b.withScheme(pgpInline)
}
// If the sign flag (that we just determined above) is true, then the MIME
// type is determined by the PGP scheme (also determined above): we should
// use 'text/plain' for a PGP/Inline scheme, and 'multipart/mixed' otherwise.

View File

@ -254,6 +254,20 @@ func TestPreferencesBuilder(t *testing.T) {
wantMIMEType: "multipart/mixed",
},
{
name: "external with sign enabled, sending plaintext, should still send as ClearMIME",
contactMeta: &ContactMetadata{Sign: true, SignIsSet: true},
receivedKeys: []pmapi.PublicKey{},
isInternal: false,
mailSettings: pmapi.MailSettings{PGPScheme: pmapi.PGPMIMEPackage, DraftMIMEType: "text/plain"},
wantEncrypt: false,
wantSign: true,
wantScheme: pmapi.ClearMIMEPackage,
wantMIMEType: "multipart/mixed",
},
{
name: "external with pinned contact public key but no intention to encrypt/sign",

View File

@ -20,6 +20,7 @@ package smtp
import (
"crypto/sha256"
"fmt"
"strings"
"sync"
"time"
@ -48,6 +49,15 @@ func newSendRecorder() *sendRecorder {
}
func (q *sendRecorder) getMessageHash(message *pmapi.Message) string {
// Outlook Calendar updates has only headers (no body) and thus have always
// the same hash. If the message is type of calendar, the "is sending"
// check to avoid potential duplicates is skipped. Duplicates should not
// be a problem in this case as calendar updates are small.
contentType := message.Header.Get("Content-Type")
if strings.HasPrefix(contentType, "text/calendar") {
return ""
}
h := sha256.New()
_, _ = h.Write([]byte(message.AddressID + message.Subject))
if message.Sender != nil {
@ -101,6 +111,10 @@ func (q *sendRecorder) isSendingOrSent(client messageGetter, hash string) (isSen
q.lock.Lock()
defer q.lock.Unlock()
if hash == "" {
return false, false
}
q.deleteExpiredKeys()
value, ok := q.hashes[hash]
if !ok {

View File

@ -349,6 +349,32 @@ func TestSendRecorder_getMessageHash(t *testing.T) {
},
false,
},
{ // Different content type - calendar
&pmapi.Message{
Header: mail.Header{
"Content-Type": []string{"text/calendar"},
},
AddressID: "address123",
Subject: "Subject #1",
Sender: &mail.Address{
Address: "from@pm.me",
},
ToList: []*mail.Address{
{Address: "to@pm.me"},
},
CCList: []*mail.Address{},
BCCList: []*mail.Address{},
Body: "body",
Attachments: []*pmapi.Attachment{
{
Name: "att1",
MIMEType: "image/png",
Size: 12345,
},
},
},
false,
},
}
for i, tc := range testCases {
tc := tc // bind
@ -382,12 +408,13 @@ func TestSendRecorder_isSendingOrSent(t *testing.T) {
{"hash", &pmapi.Message{Type: pmapi.MessageTypeDraft, Time: time.Now().Unix()}, nil, true, false},
{"hash", &pmapi.Message{Type: pmapi.MessageTypeSent}, nil, false, true},
{"hash", &pmapi.Message{Type: pmapi.MessageTypeInboxAndSent}, nil, false, true},
{"", &pmapi.Message{Type: pmapi.MessageTypeInboxAndSent}, nil, false, false},
}
for i, tc := range testCases {
tc := tc // bind
t.Run(fmt.Sprintf("%d / %v / %v / %v", i, tc.hash, tc.message, tc.err), func(t *testing.T) {
messageGetter := &testSendRecorderGetMessageMock{message: tc.message, err: tc.err}
isSending, wasSent := q.isSendingOrSent(messageGetter, "hash")
isSending, wasSent := q.isSendingOrSent(messageGetter, tc.hash)
assert.Equal(t, tc.wantIsSending, isSending, "isSending does not match")
assert.Equal(t, tc.wantWasSent, wasSent, "wasSent does not match")
})

View File

@ -25,7 +25,6 @@ import (
"io"
"mime"
"net/mail"
"regexp"
"strings"
"time"
@ -408,9 +407,9 @@ func (su *smtpUser) handleReferencesHeader(m *pmapi.Message) (draftID, parentID
if !strings.Contains(reference, "@"+pmapi.InternalIDDomain) {
newReferences = append(newReferences, reference)
} else { // internalid is the parentID.
idMatch := regexp.MustCompile(pmapi.InternalReferenceFormat).FindStringSubmatch(reference)
if len(idMatch) > 0 {
lastID := strings.TrimSuffix(strings.Trim(idMatch[0], "<>"), "@protonmail.internalid")
idMatch := pmapi.RxInternalReferenceFormat.FindStringSubmatch(reference)
if len(idMatch) == 2 {
lastID := idMatch[1]
filter := &pmapi.MessagesFilter{ID: []string{lastID}}
if su.addressID != "" {
filter.AddressID = su.addressID

View File

@ -69,7 +69,7 @@ func (storeAddress *Address) init(foldersAndLabels []*pmapi.Label) (err error) {
prefix := getLabelPrefix(label)
var mailbox *Mailbox
if mailbox, err = txNewMailbox(tx, storeAddress, label.ID, prefix, label.Name, label.Color); err != nil {
if mailbox, err = txNewMailbox(tx, storeAddress, label.ID, prefix, label.Path, label.Color); err != nil {
storeAddress.log.
WithError(err).
WithField("labelID", label.ID).

View File

@ -73,14 +73,14 @@ func (storeAddress *Address) createOrUpdateMailboxEvent(label *pmapi.Label) erro
prefix := getLabelPrefix(label)
mailbox, ok := storeAddress.mailboxes[label.ID]
if !ok {
mailbox, err := newMailbox(storeAddress, label.ID, prefix, label.Name, label.Color)
mailbox, err := newMailbox(storeAddress, label.ID, prefix, label.Path, label.Color)
if err != nil {
return err
}
storeAddress.mailboxes[label.ID] = mailbox
mailbox.store.imapMailboxCreated(storeAddress.address, mailbox.labelName)
} else {
mailbox.labelName = prefix + label.Name
mailbox.labelName = prefix + label.Path
mailbox.color = label.Color
}
return nil

View File

@ -37,7 +37,7 @@ func (store *Store) SetIMAPUpdateChannel(updates chan imapBackend.Update) {
}
}
func (store *Store) imapNotice(address, notice string) {
func (store *Store) imapNotice(address, notice string) *imapBackend.StatusUpdate {
update := new(imapBackend.StatusUpdate)
update.Update = imapBackend.NewUpdate(address, "")
update.StatusResp = &imap.StatusResp{
@ -46,13 +46,14 @@ func (store *Store) imapNotice(address, notice string) {
Info: notice,
}
store.imapSendUpdate(update)
return update
}
func (store *Store) imapUpdateMessage(
address, mailboxName string,
uid, sequenceNumber uint32,
msg *pmapi.Message, hasDeletedFlag bool,
) {
) *imapBackend.MessageUpdate {
store.log.WithFields(logrus.Fields{
"address": address,
"mailbox": mailboxName,
@ -70,9 +71,10 @@ func (store *Store) imapUpdateMessage(
}
update.Message.Uid = uid
store.imapSendUpdate(update)
return update
}
func (store *Store) imapDeleteMessage(address, mailboxName string, sequenceNumber uint32) {
func (store *Store) imapDeleteMessage(address, mailboxName string, sequenceNumber uint32) *imapBackend.ExpungeUpdate {
store.log.WithFields(logrus.Fields{
"address": address,
"mailbox": mailboxName,
@ -82,9 +84,10 @@ func (store *Store) imapDeleteMessage(address, mailboxName string, sequenceNumbe
update.Update = imapBackend.NewUpdate(address, mailboxName)
update.SeqNum = sequenceNumber
store.imapSendUpdate(update)
return update
}
func (store *Store) imapMailboxCreated(address, mailboxName string) {
func (store *Store) imapMailboxCreated(address, mailboxName string) *imapBackend.MailboxInfoUpdate {
store.log.WithFields(logrus.Fields{
"address": address,
"mailbox": mailboxName,
@ -97,9 +100,10 @@ func (store *Store) imapMailboxCreated(address, mailboxName string) {
Name: mailboxName,
}
store.imapSendUpdate(update)
return update
}
func (store *Store) imapMailboxStatus(address, mailboxName string, total, unread, unreadSeqNum uint) {
func (store *Store) imapMailboxStatus(address, mailboxName string, total, unread, unreadSeqNum uint) *imapBackend.MailboxUpdate {
store.log.WithFields(logrus.Fields{
"address": address,
"mailbox": mailboxName,
@ -114,6 +118,7 @@ func (store *Store) imapMailboxStatus(address, mailboxName string, total, unread
update.MailboxStatus.Unseen = uint32(unread)
update.MailboxStatus.UnseenSeqNum = uint32(unreadSeqNum)
store.imapSendUpdate(update)
return update
}
func (store *Store) imapSendUpdate(update imapBackend.Update) {
@ -122,22 +127,10 @@ func (store *Store) imapSendUpdate(update imapBackend.Update) {
return
}
done := update.Done()
go func() {
// This timeout is to not keep running many blocked goroutines.
// In case nothing listens to this channel, this thread should stop.
select {
case store.imapUpdates <- update:
case <-time.After(1 * time.Second):
store.log.Warn("IMAP update could not be sent (timeout).")
}
}()
// This timeout is to not block IMAP backend by wait for IMAP client.
select {
case <-done:
case <-time.After(1 * time.Second):
store.log.Warn("IMAP update could not be delivered (timeout).")
store.log.Warn("IMAP update could not be sent (timeout)")
return
case store.imapUpdates <- update:
}
}

View File

@ -21,6 +21,7 @@ import (
"encoding/json"
"fmt"
"strings"
"sync/atomic"
"github.com/ProtonMail/proton-bridge/pkg/pmapi"
"github.com/sirupsen/logrus"
@ -38,6 +39,8 @@ type Mailbox struct {
color string
log *logrus.Entry
isDeleting atomic.Value
}
func newMailbox(storeAddress *Address, labelID, labelPrefix, labelName, color string) (mb *Mailbox, err error) {
@ -59,6 +62,7 @@ func txNewMailbox(tx *bolt.Tx, storeAddress *Address, labelID, labelPrefix, labe
color: color,
log: l,
}
mb.isDeleting.Store(false)
err := initMailboxBucket(tx, mb.getBucketName())
if err != nil {
@ -142,6 +146,9 @@ func initMailboxBucket(tx *bolt.Tx, bucketName []byte) error {
if _, err := bucket.CreateBucketIfNotExists(apiIDsBucket); err != nil {
return err
}
if _, err := bucket.CreateBucketIfNotExists(deletedIDsBucket); err != nil {
return err
}
return nil
}
@ -212,6 +219,7 @@ func (storeMailbox *Mailbox) Rename(newName string) error {
// Deletion has to be propagated to all the same mailboxes in all addresses.
// The propagation is processed by the event loop.
func (storeMailbox *Mailbox) Delete() error {
storeMailbox.isDeleting.Store(true)
return storeMailbox.storeAddress.deleteMailbox(storeMailbox.labelID)
}
@ -223,6 +231,14 @@ func (storeMailbox *Mailbox) GetDelimiter() string {
// deleteMailboxEvent deletes the mailbox bucket.
// This is called from the event loop.
func (storeMailbox *Mailbox) deleteMailboxEvent() error {
if !storeMailbox.isDeleting.Load().(bool) {
// Deleting label removes bucket. Any ongoing connection selected
// in such mailbox then might panic because of non-existing bucket.
// Closing connetions prevents that panic but if the connection
// asked for deletion, it should not be closed so it can receive
// successful response.
storeMailbox.store.user.CloseAllConnections()
}
return storeMailbox.db().Update(func(tx *bolt.Tx) error {
return tx.Bucket(mailboxesBucket).DeleteBucket(storeMailbox.getBucketName())
})
@ -240,13 +256,7 @@ func (storeMailbox *Mailbox) txGetAPIIDsBucket(tx *bolt.Tx) *bolt.Bucket {
// txGetDeletedIDsBucket returns the bucket with messagesID marked as deleted
func (storeMailbox *Mailbox) txGetDeletedIDsBucket(tx *bolt.Tx) *bolt.Bucket {
// There should be no error since it _...returns an error if the bucket
// name is blank, or if the bucket name is too long._
bucket, err := storeMailbox.txGetBucket(tx).CreateBucketIfNotExists(deletedIDsBucket)
if err != nil || bucket == nil {
storeMailbox.log.WithError(err).Error("Cannot create or get bucket with deleted IDs.")
}
return bucket
return storeMailbox.txGetBucket(tx).Bucket(deletedIDsBucket)
}
// txGetBucket returns the bucket of mailbox containing mapping buckets.

View File

@ -125,6 +125,7 @@ func (mc *mailboxCounts) getPMLabel() *pmapi.Label {
return &pmapi.Label{
ID: mc.LabelID,
Name: mc.LabelName,
Path: mc.LabelName,
Color: mc.Color,
Order: mc.Order,
Type: pmapi.LabelTypeMailbox,
@ -158,7 +159,7 @@ func (store *Store) createOrUpdateMailboxCountsBuckets(labels []*pmapi.Label) er
}
// Update mailbox info, but dont change on-API-counts.
mailbox.LabelName = label.Name
mailbox.LabelName = label.Path
mailbox.Color = label.Color
mailbox.Order = label.Order
mailbox.IsFolder = label.Exclusive == 1

View File

@ -18,6 +18,8 @@
package store
import (
"time"
"github.com/ProtonMail/proton-bridge/pkg/pmapi"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
@ -501,7 +503,7 @@ func (storeMailbox *Mailbox) txMarkMessagesAsDeleted(tx *bolt.Tx, apiIDs []strin
// In order to send flags in format
// S: * 2 FETCH (FLAGS (\Deleted \Seen))
storeMailbox.store.imapUpdateMessage(
update := storeMailbox.store.imapUpdateMessage(
storeMailbox.storeAddress.address,
storeMailbox.labelName,
uid,
@ -509,6 +511,14 @@ func (storeMailbox *Mailbox) txMarkMessagesAsDeleted(tx *bolt.Tx, apiIDs []strin
msg,
markAsDeleted,
)
// txMarkMessagesAsDeleted is called only during processing request
// from IMAP call (i.e., not from event loop) and in such cases we
// have to wait to propagate update back before closing the response.
select {
case <-time.After(1 * time.Second):
case <-update.Done():
}
}
return nil

View File

@ -66,7 +66,7 @@ func (message *Message) Message() *pmapi.Message {
// mailbox
func (message *Message) IsMarkedDeleted() bool {
isMarkedAsDeleted := false
err := message.storeMailbox.db().Update(func(tx *bolt.Tx) error {
err := message.storeMailbox.db().View(func(tx *bolt.Tx) error {
isMarkedAsDeleted = message.storeMailbox.txGetDeletedIDsBucket(tx).Get([]byte(message.msg.ID)) != nil
return nil
})

View File

@ -5,9 +5,10 @@
package mocks
import (
reflect "reflect"
pmapi "github.com/ProtonMail/proton-bridge/pkg/pmapi"
gomock "github.com/golang/mock/gomock"
reflect "reflect"
)
// MockPanicHandler is a mock of PanicHandler interface
@ -105,6 +106,18 @@ func (m *MockBridgeUser) EXPECT() *MockBridgeUserMockRecorder {
return m.recorder
}
// CloseAllConnections mocks base method
func (m *MockBridgeUser) CloseAllConnections() {
m.ctrl.T.Helper()
m.ctrl.Call(m, "CloseAllConnections")
}
// CloseAllConnections indicates an expected call of CloseAllConnections
func (mr *MockBridgeUserMockRecorder) CloseAllConnections() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "CloseAllConnections", reflect.TypeOf((*MockBridgeUser)(nil).CloseAllConnections))
}
// CloseConnection mocks base method
func (m *MockBridgeUser) CloseConnection(arg0 string) {
m.ctrl.T.Helper()

View File

@ -5,9 +5,10 @@
package mocks
import (
gomock "github.com/golang/mock/gomock"
reflect "reflect"
time "time"
gomock "github.com/golang/mock/gomock"
)
// MockListener is a mock of Listener interface

View File

@ -36,6 +36,7 @@ type BridgeUser interface {
GetPrimaryAddress() string
GetStoreAddresses() []string
UpdateUser() error
CloseAllConnections()
CloseConnection(string)
Logout() error
}

View File

@ -29,18 +29,36 @@ type Message struct {
ID string
Unread bool
Body []byte
Source Mailbox
Sources []Mailbox
Targets []Mailbox
}
// sourceNames returns array of source mailbox names.
func (msg Message) sourceNames() (names []string) {
for _, mailbox := range msg.Sources {
names = append(names, mailbox.Name)
}
return
}
// targetNames returns array of target mailbox names.
func (msg Message) targetNames() (names []string) {
for _, mailbox := range msg.Targets {
names = append(names, mailbox.Name)
}
return
}
// MessageStatus holds status for message used by progress manager.
type MessageStatus struct {
eventTime time.Time // Time of adding message to the process.
rule *Rule // Rule with source and target mailboxes.
SourceID string // Message ID at the source.
targetID string // Message ID at the target (if any).
bodyHash string // Hash of the message body.
eventTime time.Time // Time of adding message to the process.
sourceNames []string // Source mailbox names message is in.
SourceID string // Message ID at the source.
targetNames []string // Target mailbox names message is in.
targetID string // Message ID at the target (if any).
bodyHash string // Hash of the message body.
skipped bool
exported bool
imported bool
exportErr error
@ -79,7 +97,7 @@ func (status *MessageStatus) setDetailsFromHeader(header mail.Header) {
}
func (status *MessageStatus) hasError(includeMissing bool) bool {
return status.exportErr != nil || status.importErr != nil || (includeMissing && !status.imported)
return status.exportErr != nil || status.importErr != nil || (includeMissing && !status.skipped && !status.imported)
}
// GetErrorMessage returns error message.
@ -88,6 +106,9 @@ func (status *MessageStatus) GetErrorMessage() string {
}
func (status *MessageStatus) getErrorMessage(includeMissing bool) string {
if status.skipped {
return ""
}
if status.exportErr != nil {
return fmt.Sprintf("failed to export: %s", status.exportErr)
}

View File

@ -5,9 +5,10 @@
package mocks
import (
reflect "reflect"
pmapi "github.com/ProtonMail/proton-bridge/pkg/pmapi"
gomock "github.com/golang/mock/gomock"
reflect "reflect"
)
// MockPanicHandler is a mock of PanicHandler interface

View File

@ -93,7 +93,7 @@ func (p *Progress) fatal(err error) {
defer p.lock.Unlock()
log.WithError(err).Error("Progress finished")
p.isStopped = true
p.setStop()
p.fatalError = err
p.cleanUpdateCh()
}
@ -126,19 +126,33 @@ func (p *Progress) updateCount(mailbox string, count uint) {
}
// addMessage should be called as soon as there is ID of the message.
func (p *Progress) addMessage(messageID string, rule *Rule) {
func (p *Progress) addMessage(messageID string, sourceNames, targetNames []string) {
p.lock.Lock()
defer p.lock.Unlock()
defer p.update()
p.log.WithField("id", messageID).Trace("Message added")
p.messageStatuses[messageID] = &MessageStatus{
eventTime: time.Now(),
rule: rule,
SourceID: messageID,
eventTime: time.Now(),
sourceNames: sourceNames,
SourceID: messageID,
targetNames: targetNames,
}
}
// messageSkipped should be called once the message is skipped due to some
// filter such as time or folder and so on.
func (p *Progress) messageSkipped(messageID string) {
p.lock.Lock()
defer p.lock.Unlock()
defer p.update()
p.log.WithField("id", messageID).Debug("Message skipped")
p.messageStatuses[messageID].skipped = true
p.logMessage(messageID)
}
// messageExported should be called right before message is exported.
func (p *Progress) messageExported(messageID string, body []byte, err error) {
p.lock.Lock()
@ -282,6 +296,15 @@ func (p *Progress) Stop() {
defer p.update()
p.log.Info("Progress stopped")
p.setStop()
// Once progress is stopped, some calls might be in progress. Results from
// those calls are irrelevant so we can close update channel sooner to not
// propagate any progress to user interface anymore.
p.cleanUpdateCh()
}
func (p *Progress) setStop() {
p.isStopped = true
p.pauseReason = "" // Clear pause to run paused code and stop it.
}
@ -320,35 +343,40 @@ func (p *Progress) GetFailedMessages() []*MessageStatus {
}
// GetCounts returns counts of exported and imported messages.
func (p *Progress) GetCounts() (failed, imported, exported, added, total uint) {
func (p *Progress) GetCounts() ProgressCounts {
p.lock.Lock()
defer p.lock.Unlock()
counts := ProgressCounts{}
// Return counts only once total is estimated or the process already
// ended (for a case when it ended quickly to report it correctly).
if p.updateCh != nil && !p.messageCounted {
return
return counts
}
// Include lost messages in the process only when transfer is done.
includeMissing := p.updateCh == nil
for _, mailboxCount := range p.messageCounts {
total += mailboxCount
counts.Total += mailboxCount
}
for _, status := range p.messageStatuses {
added++
counts.Added++
if status.skipped {
counts.Skipped++
}
if status.exported {
exported++
counts.Exported++
}
if status.imported {
imported++
counts.Imported++
}
if status.hasError(includeMissing) {
failed++
counts.Failed++
}
}
return
return counts
}
// GenerateBugReport generates similar file to import log except private information.

View File

@ -0,0 +1,35 @@
// Copyright (c) 2020 Proton Technologies AG
//
// This file is part of ProtonMail Bridge.
//
// ProtonMail Bridge is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ProtonMail Bridge is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with ProtonMail Bridge. If not, see <https://www.gnu.org/licenses/>.
package transfer
// ProgressCounts holds counts counted by Progress.
type ProgressCounts struct {
Failed,
Skipped,
Imported,
Exported,
Added,
Total uint
}
// Progress returns ratio between processed messages (fully imported, skipped
// and failed ones) and total number of messages as percentage (0 - 1).
func (c *ProgressCounts) Progress() float32 {
progressed := c.Imported + c.Skipped + c.Failed
return float32(progressed) / float32(c.Total)
}

View File

@ -19,6 +19,7 @@ package transfer
import (
"testing"
"time"
"github.com/pkg/errors"
a "github.com/stretchr/testify/assert"
@ -38,8 +39,8 @@ func TestProgressUpdateCount(t *testing.T) {
progress.finish()
_, _, _, _, total := progress.GetCounts() //nolint[dogsled]
r.Equal(t, uint(42), total)
counts := progress.GetCounts()
r.Equal(t, uint(42), counts.Total)
}
func TestProgressAddingMessages(t *testing.T) {
@ -47,31 +48,36 @@ func TestProgressAddingMessages(t *testing.T) {
drainProgressUpdateChannel(&progress)
// msg1 has no problem.
progress.addMessage("msg1", nil)
progress.addMessage("msg1", []string{}, []string{})
progress.messageExported("msg1", []byte(""), nil)
progress.messageImported("msg1", "", nil)
// msg2 has an import problem.
progress.addMessage("msg2", nil)
progress.addMessage("msg2", []string{}, []string{})
progress.messageExported("msg2", []byte(""), nil)
progress.messageImported("msg2", "", errors.New("failed import"))
// msg3 has an export problem.
progress.addMessage("msg3", nil)
progress.addMessage("msg3", []string{}, []string{})
progress.messageExported("msg3", []byte(""), errors.New("failed export"))
// msg4 has an export problem and import is also called.
progress.addMessage("msg4", nil)
progress.addMessage("msg4", []string{}, []string{})
progress.messageExported("msg4", []byte(""), errors.New("failed export"))
progress.messageImported("msg4", "", nil)
// msg5 is skipped.
progress.addMessage("msg5", []string{}, []string{})
progress.messageSkipped("msg5")
progress.finish()
failed, imported, exported, added, _ := progress.GetCounts()
a.Equal(t, uint(4), added)
a.Equal(t, uint(2), exported)
a.Equal(t, uint(2), imported)
a.Equal(t, uint(3), failed)
counts := progress.GetCounts()
a.Equal(t, uint(5), counts.Added)
a.Equal(t, uint(2), counts.Exported)
a.Equal(t, uint(2), counts.Imported)
a.Equal(t, uint(1), counts.Skipped)
a.Equal(t, uint(3), counts.Failed)
errorsMap := map[string]string{}
for _, status := range progress.GetFailedMessages() {
@ -91,7 +97,7 @@ func TestProgressFinish(t *testing.T) {
progress.finish()
r.Nil(t, progress.updateCh)
r.NotPanics(t, func() { progress.addMessage("msg", nil) })
r.NotPanics(t, func() { progress.addMessage("msg", []string{}, []string{}) })
}
func TestProgressFatalError(t *testing.T) {
@ -101,7 +107,29 @@ func TestProgressFatalError(t *testing.T) {
progress.fatal(errors.New("fatal error"))
r.Nil(t, progress.updateCh)
r.NotPanics(t, func() { progress.addMessage("msg", nil) })
r.NotPanics(t, func() { progress.addMessage("msg", []string{}, []string{}) })
}
func TestFailUnpauseAndStops(t *testing.T) {
progress := newProgress(log, nil)
drainProgressUpdateChannel(&progress)
progress.Pause("pausing")
progress.fatal(errors.New("fatal error"))
r.Nil(t, progress.updateCh)
r.True(t, progress.isStopped)
r.False(t, progress.IsPaused())
r.Eventually(t, progress.shouldStop, time.Second, 10*time.Millisecond)
}
func TestStopClosesUpdates(t *testing.T) {
progress := newProgress(log, nil)
ch := progress.updateCh
progress.Stop()
r.Nil(t, progress.updateCh)
r.PanicsWithError(t, "send on closed channel", func() { ch <- struct{}{} })
}
func drainProgressUpdateChannel(progress *Progress) {

View File

@ -82,8 +82,6 @@ func (p *EMLProvider) getFilePathsPerFolder(rules transferRules) (map[string][]s
}
func (p *EMLProvider) exportMessages(rule *Rule, filePaths []string, progress *Progress, ch chan<- Message) {
count := uint(len(filePaths))
for _, filePath := range filePaths {
if progress.shouldStop() {
break
@ -91,6 +89,8 @@ func (p *EMLProvider) exportMessages(rule *Rule, filePaths []string, progress *P
msg, err := p.exportMessage(rule, filePath)
progress.addMessage(filePath, msg.sourceNames(), msg.targetNames())
// Read and check time in body only if the rule specifies it
// to not waste energy.
if err == nil && rule.HasTimeLimit() {
@ -99,17 +99,11 @@ func (p *EMLProvider) exportMessages(rule *Rule, filePaths []string, progress *P
err = msgTimeErr
} else if !rule.isTimeInRange(msgTime) {
log.WithField("msg", filePath).Debug("Message skipped due to time")
count--
progress.updateCount(rule.SourceMailbox.Name, count)
progress.messageSkipped(filePath)
continue
}
}
// addMessage is called after time check to not report message
// which should not be exported but any error from reading body
// or parsing time is reported as an error.
progress.addMessage(filePath, rule)
progress.messageExported(filePath, msg.Body, err)
if err == nil {
ch <- msg
@ -134,7 +128,7 @@ func (p *EMLProvider) exportMessage(rule *Rule, filePath string) (Message, error
ID: filePath,
Unread: false,
Body: body,
Source: rule.SourceMailbox,
Sources: []Mailbox{rule.SourceMailbox},
Targets: rule.TargetMailboxes,
}, nil
}

View File

@ -61,7 +61,7 @@ func (p *EMLProvider) TransferFrom(rules transferRules, progress *Progress, ch <
func (p *EMLProvider) createFolders(rules transferRules) error {
for rule := range rules.iterateActiveRules() {
for _, mailbox := range rule.TargetMailboxes {
path := filepath.Join(p.root, mailbox.Name)
path := filepath.Join(p.root, sanitizeFileName(mailbox.Name))
if err := os.MkdirAll(path, os.ModePerm); err != nil {
return err
}
@ -71,7 +71,7 @@ func (p *EMLProvider) createFolders(rules transferRules) error {
}
func (p *EMLProvider) writeFile(msg Message) error {
fileName := filepath.Base(msg.ID)
fileName := sanitizeFileName(filepath.Base(msg.ID))
if filepath.Ext(fileName) != ".eml" {
fileName += ".eml"
}

View File

@ -124,7 +124,7 @@ func (p *IMAPProvider) loadMessagesInfo(rule *Rule, progress *Progress, uidValid
uid: imapMessage.Uid,
size: imapMessage.Size,
}
progress.addMessage(id, rule)
progress.addMessage(id, []string{rule.SourceMailbox.Name}, rule.TargetMailboxNames())
}
progress.callWrap(func() error {
@ -231,7 +231,7 @@ func (p *IMAPProvider) exportMessage(rule *Rule, id string, imapMessage *imap.Me
ID: id,
Unread: unread,
Body: body,
Source: rule.SourceMailbox,
Sources: []Mailbox{rule.SourceMailbox},
Targets: rule.TargetMailboxes,
}
}

View File

@ -18,8 +18,11 @@
package transfer
import (
"os"
"path/filepath"
"strings"
"github.com/pkg/errors"
)
// MBOXProvider implements import and export to/from MBOX structure.
@ -44,16 +47,35 @@ func (p *MBOXProvider) ID() string {
// In case the same folder name is used more than once (for example root/a/foo
// and root/b/foo), it's treated as the same folder.
func (p *MBOXProvider) Mailboxes(includeEmpty, includeAllMail bool) ([]Mailbox, error) {
filePaths, err := getFilePathsWithSuffix(p.root, "mbox")
filePaths, err := getAllPathsWithSuffix(p.root, ".mbox")
if err != nil {
return nil, err
}
mailboxes := []Mailbox{}
mailboxNames := map[string]bool{}
for _, filePath := range filePaths {
fileName := filepath.Base(filePath)
mailboxName := strings.TrimSuffix(fileName, ".mbox")
filePath, err := p.handleAppleMailMBOXStructure(filePath)
if err != nil {
log.WithError(err).Warn("Failed to handle MBOX structure")
continue
}
mailboxName := strings.TrimSuffix(fileName, ".mbox")
mailboxNames[mailboxName] = true
labels, err := getGmailLabelsFromMboxFile(filepath.Join(p.root, filePath))
if err != nil {
log.WithError(err).Error("Failed to get gmail labels from mbox file")
continue
}
for label := range labels {
mailboxNames[label] = true
}
}
mailboxes := []Mailbox{}
for mailboxName := range mailboxNames {
mailboxes = append(mailboxes, Mailbox{
ID: "",
Name: mailboxName,
@ -61,6 +83,20 @@ func (p *MBOXProvider) Mailboxes(includeEmpty, includeAllMail bool) ([]Mailbox,
IsExclusive: false,
})
}
return mailboxes, nil
}
// handleAppleMailMBOXStructure changes the path of mailbox directory to
// the path of mbox file. Apple Mail MBOX exports has this structure:
// `Folder.mbox` directory with `mbox` file inside.
// Example: `Folder.mbox/mbox` (and this function converts `Folder.mbox`
// to `Folder.mbox/mbox`).
func (p *MBOXProvider) handleAppleMailMBOXStructure(filePath string) (string, error) {
if info, err := os.Stat(filepath.Join(p.root, filePath)); err == nil && info.IsDir() {
if _, err := os.Stat(filepath.Join(p.root, filePath, "mbox")); err != nil {
return "", errors.Wrap(err, "wrong mbox structure")
}
return filepath.Join(filePath, "mbox"), nil
}
return filePath, nil
}

View File

@ -0,0 +1,118 @@
// Copyright (c) 2020 Proton Technologies AG
//
// This file is part of ProtonMail Bridge.
//
// ProtonMail Bridge is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ProtonMail Bridge is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with ProtonMail Bridge. If not, see <https://www.gnu.org/licenses/>.
package transfer
import (
"bufio"
"bytes"
"io"
"mime"
"os"
"strings"
)
type stringSet map[string]bool
const xGmailLabelsHeader = "X-Gmail-Labels"
// filteredOutGmailLabels is set of labels which we don't want to show to users
// as they might be auto-generated by Gmail and unwanted.
var filteredOutGmailLabels = []string{ //nolint[gochecknoglobals]
"Unread",
"Opened",
"IMAP_Junk",
"IMAP_NonJunk",
"IMAP_NotJunk",
"IMAP_$NotJunk",
}
func getGmailLabelsFromMboxFile(filePath string) (stringSet, error) {
f, err := os.Open(filePath) //nolint[gosec]
if err != nil {
return nil, err
}
return getGmailLabelsFromMboxReader(f)
}
func getGmailLabelsFromMboxReader(f io.Reader) (stringSet, error) {
allLabels := stringSet{}
// Scanner is not used as it does not support long lines and some mbox
// files contain very long lines even though that should not be happening.
r := bufio.NewReader(f)
for {
b, isPrefix, err := r.ReadLine()
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
for isPrefix {
_, isPrefix, err = r.ReadLine()
if err != nil {
break
}
}
if bytes.HasPrefix(b, []byte(xGmailLabelsHeader)) {
for label := range getGmailLabelsFromValue(string(b)) {
allLabels[label] = true
}
}
}
return allLabels, nil
}
func getGmailLabelsFromMessage(body []byte) (stringSet, error) {
header, err := getMessageHeader(body)
if err != nil {
return nil, err
}
labels := header.Get(xGmailLabelsHeader)
return getGmailLabelsFromValue(labels), nil
}
func getGmailLabelsFromValue(value string) stringSet {
value = strings.TrimPrefix(value, xGmailLabelsHeader+":")
if decoded, err := new(mime.WordDecoder).DecodeHeader(value); err != nil {
log.WithError(err).Error("Failed to decode header")
} else {
value = decoded
}
labels := stringSet{}
for _, label := range strings.Split(value, ",") {
label = strings.TrimSpace(label)
if label == "" {
continue
}
skip := false
for _, filteredOutLabel := range filteredOutGmailLabels {
if label == filteredOutLabel {
skip = true
break
}
}
if skip {
continue
}
labels[label] = true
}
return labels
}

View File

@ -0,0 +1,135 @@
// Copyright (c) 2020 Proton Technologies AG
//
// This file is part of ProtonMail Bridge.
//
// ProtonMail Bridge is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ProtonMail Bridge is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with ProtonMail Bridge. If not, see <https://www.gnu.org/licenses/>.
package transfer
import (
"fmt"
"strings"
"testing"
r "github.com/stretchr/testify/require"
)
func TestGetGmailLabelsFromMboxReader(t *testing.T) {
mboxFile := `From - Mon May 4 16:40:31 2020
Subject: Test 1
X-Gmail-Labels: Foo,Bar
hello
From - Mon May 4 16:40:31 2020
Subject: Test 2
X-Gmail-Labels: Foo , Baz
hello
From - Mon May 4 16:40:31 2020
Subject: Test 3
X-Gmail-Labels: ,
hello
From - Mon May 4 16:40:31 2020
Subject: Test 4
X-Gmail-Labels:
hello
From - Mon May 4 16:40:31 2020
Subject: Test 5
hello
`
mboxReader := strings.NewReader(mboxFile)
labels, err := getGmailLabelsFromMboxReader(mboxReader)
r.NoError(t, err)
r.Equal(t, toSet("Foo", "Bar", "Baz"), labels)
}
func TestGetGmailLabelsFromMessage(t *testing.T) {
tests := []struct {
body string
wantLabels stringSet
}{
{`Subject: One
X-Gmail-Labels: Foo,Bar
Hello
`, toSet("Foo", "Bar")},
{`Subject: Two
X-Gmail-Labels: Foo , Bar ,
Hello
`, toSet("Foo", "Bar")},
{`Subject: Three
X-Gmail-Labels: ,
Hello
`, toSet()},
{`Subject: Four
X-Gmail-Labels:
Hello
`, toSet()},
{`Subject: Five
Hello
`, toSet()},
}
for _, tc := range tests {
tc := tc
t.Run(fmt.Sprintf("%v", tc.body), func(t *testing.T) {
labels, err := getGmailLabelsFromMessage([]byte(tc.body))
r.NoError(t, err)
r.Equal(t, tc.wantLabels, labels)
})
}
}
func TestGetGmailLabelsFromValue(t *testing.T) {
tests := []struct {
value string
wantLabels stringSet
}{
{"Foo,Bar", toSet("Foo", "Bar")},
{" Foo , Bar ", toSet("Foo", "Bar")},
{" Foo , Bar , ", toSet("Foo", "Bar")},
{" Foo Bar ", toSet("Foo Bar")},
{" , ", toSet()},
{" ", toSet()},
{"", toSet()},
{"=?UTF-8?Q?Archived,Category_personal,test_=F0=9F=98=80=F0=9F=99=83?=", toSet("Archived", "Category personal", "test 😀🙃")},
{"IMAP_NotJunk,Foo,Opened,bar,Unread", toSet("Foo", "bar")},
}
for _, tc := range tests {
tc := tc
t.Run(fmt.Sprintf("%v", tc.value), func(t *testing.T) {
labels := getGmailLabelsFromValue(tc.value)
r.Equal(t, tc.wantLabels, labels)
})
}
}
func toSet(items ...string) stringSet {
set := map[string]bool{}
for _, item := range items {
set[item] = true
}
return set
}

View File

@ -34,7 +34,7 @@ func (p *MBOXProvider) TransferTo(rules transferRules, progress *Progress, ch ch
log.Info("Started transfer from MBOX to channel")
defer log.Info("Finished transfer from MBOX to channel")
filePathsPerFolder, err := p.getFilePathsPerFolder(rules)
filePathsPerFolder, err := p.getFilePathsPerFolder()
if err != nil {
progress.fatal(err)
return
@ -45,32 +45,29 @@ func (p *MBOXProvider) TransferTo(rules transferRules, progress *Progress, ch ch
}
for folderName, filePaths := range filePathsPerFolder {
// No error guaranteed by getFilePathsPerFolder.
rule, _ := rules.getRuleBySourceMailboxName(folderName)
log.WithField("folder", folderName).Debug("Estimating folder counts")
for _, filePath := range filePaths {
if progress.shouldStop() {
break
}
p.updateCount(rule, progress, filePath)
p.updateCount(progress, filePath)
}
}
progress.countsFinal()
for folderName, filePaths := range filePathsPerFolder {
// No error guaranteed by getFilePathsPerFolder.
rule, _ := rules.getRuleBySourceMailboxName(folderName)
log.WithField("rule", rule).Debug("Processing rule")
log.WithField("folder", folderName).Debug("Processing folder")
for _, filePath := range filePaths {
if progress.shouldStop() {
break
}
p.transferTo(rule, progress, ch, filePath)
p.transferTo(rules, progress, ch, folderName, filePath)
}
}
}
func (p *MBOXProvider) getFilePathsPerFolder(rules transferRules) (map[string][]string, error) {
filePaths, err := getFilePathsWithSuffix(p.root, ".mbox")
func (p *MBOXProvider) getFilePathsPerFolder() (map[string][]string, error) {
filePaths, err := getAllPathsWithSuffix(p.root, ".mbox")
if err != nil {
return nil, err
}
@ -78,19 +75,19 @@ func (p *MBOXProvider) getFilePathsPerFolder(rules transferRules) (map[string][]
filePathsMap := map[string][]string{}
for _, filePath := range filePaths {
fileName := filepath.Base(filePath)
folder := strings.TrimSuffix(fileName, ".mbox")
_, err := rules.getRuleBySourceMailboxName(folder)
filePath, err := p.handleAppleMailMBOXStructure(filePath)
// Skip unsupported MBOX structures. It was already filtered out in configuration step.
if err != nil {
log.WithField("msg", filePath).Trace("Mailbox skipped due to folder name")
continue
}
folder := strings.TrimSuffix(fileName, ".mbox")
filePathsMap[folder] = append(filePathsMap[folder], filePath)
}
return filePathsMap, nil
}
func (p *MBOXProvider) updateCount(rule *Rule, progress *Progress, filePath string) {
func (p *MBOXProvider) updateCount(progress *Progress, filePath string) {
mboxReader := p.openMbox(progress, filePath)
if mboxReader == nil {
return
@ -107,17 +104,16 @@ func (p *MBOXProvider) updateCount(rule *Rule, progress *Progress, filePath stri
}
count++
}
progress.updateCount(rule.SourceMailbox.Name, uint(count))
progress.updateCount(filePath, uint(count))
}
func (p *MBOXProvider) transferTo(rule *Rule, progress *Progress, ch chan<- Message, filePath string) {
func (p *MBOXProvider) transferTo(rules transferRules, progress *Progress, ch chan<- Message, folderName, filePath string) {
mboxReader := p.openMbox(progress, filePath)
if mboxReader == nil {
return
}
index := 0
count := 0
for {
if progress.shouldStop() {
break
@ -134,50 +130,118 @@ func (p *MBOXProvider) transferTo(rule *Rule, progress *Progress, ch chan<- Mess
break
}
msg, err := p.exportMessage(rule, id, msgReader)
msg, err := p.exportMessage(rules, folderName, id, msgReader)
// Read and check time in body only if the rule specifies it
// to not waste energy.
if err == nil && rule.HasTimeLimit() {
msgTime, msgTimeErr := getMessageTime(msg.Body)
if msgTimeErr != nil {
err = msgTimeErr
} else if !rule.isTimeInRange(msgTime) {
log.WithField("msg", id).Debug("Message skipped due to time")
continue
}
progress.addMessage(id, msg.sourceNames(), msg.targetNames())
if err == nil && len(msg.Targets) == 0 {
progress.messageSkipped(id)
continue
}
// Counting only messages filtered by time to update count to correct total.
count++
// addMessage is called after time check to not report message
// which should not be exported but any error from reading body
// or parsing time is reported as an error.
progress.addMessage(id, rule)
progress.messageExported(id, msg.Body, err)
if err == nil {
ch <- msg
}
}
progress.updateCount(rule.SourceMailbox.Name, uint(count))
}
func (p *MBOXProvider) exportMessage(rule *Rule, id string, msgReader io.Reader) (Message, error) {
func (p *MBOXProvider) exportMessage(rules transferRules, folderName, id string, msgReader io.Reader) (Message, error) {
body, err := ioutil.ReadAll(msgReader)
if err != nil {
return Message{}, errors.Wrap(err, "failed to read message")
}
msgRules := p.getMessageRules(rules, folderName, id, body)
sources := p.getMessageSources(msgRules)
targets := p.getMessageTargets(msgRules, id, body)
return Message{
ID: id,
Unread: false,
Body: body,
Source: rule.SourceMailbox,
Targets: rule.TargetMailboxes,
Sources: sources,
Targets: targets,
}, nil
}
func (p *MBOXProvider) getMessageRules(rules transferRules, folderName, id string, body []byte) []*Rule {
msgRules := []*Rule{}
folderRule, err := rules.getRuleBySourceMailboxName(folderName)
if err != nil {
log.WithField("msg", id).WithField("source", folderName).Debug("Message source doesn't have a rule")
} else if folderRule.Active {
msgRules = append(msgRules, folderRule)
}
gmailLabels, err := getGmailLabelsFromMessage(body)
if err != nil {
log.WithError(err).Error("Failed to get gmail labels, ")
} else {
for label := range gmailLabels {
rule, err := rules.getRuleBySourceMailboxName(label)
if err != nil {
log.WithField("msg", id).WithField("source", label).Debug("Message source doesn't have a rule")
continue
}
if rule.Active {
msgRules = append(msgRules, rule)
}
}
}
return msgRules
}
func (p *MBOXProvider) getMessageSources(msgRules []*Rule) []Mailbox {
sources := []Mailbox{}
for _, rule := range msgRules {
sources = append(sources, rule.SourceMailbox)
}
return sources
}
func (p *MBOXProvider) getMessageTargets(msgRules []*Rule, id string, body []byte) []Mailbox {
targets := []Mailbox{}
haveExclusiveMailbox := false
for _, rule := range msgRules {
// Read and check time in body only if the rule specifies it
// to not waste energy.
if rule.HasTimeLimit() {
msgTime, err := getMessageTime(body)
if err != nil {
log.WithError(err).Error("Failed to parse time, time check skipped")
} else if !rule.isTimeInRange(msgTime) {
log.WithField("msg", id).WithField("source", rule.SourceMailbox.Name).Debug("Message skipped due to time")
continue
}
}
for _, newTarget := range rule.TargetMailboxes {
// msgRules is sorted. The first rule is based on the folder name,
// followed by the order from X-Gmail-Labels. The rule based on
// the folder name should have priority for exclusive target.
if newTarget.IsExclusive && haveExclusiveMailbox {
continue
}
found := false
for _, target := range targets {
if target.Hash() == newTarget.Hash() {
found = true
break
}
}
if found {
continue
}
if newTarget.IsExclusive {
haveExclusiveMailbox = true
}
targets = append(targets, newTarget)
}
}
return targets
}
func (p *MBOXProvider) openMbox(progress *Progress, mboxPath string) *mbox.Reader {
mboxPath = filepath.Join(p.root, mboxPath)
mboxFile, err := os.Open(mboxPath) //nolint[gosec]

View File

@ -57,7 +57,7 @@ func (p *MBOXProvider) TransferFrom(rules transferRules, progress *Progress, ch
func (p *MBOXProvider) writeMessage(msg Message) error {
var multiErr error
for _, mailbox := range msg.Targets {
mboxName := filepath.Base(mailbox.Name)
mboxName := sanitizeFileName(mailbox.Name)
if !strings.HasSuffix(mboxName, ".mbox") {
mboxName += ".mbox"
}

View File

@ -35,27 +35,35 @@ func newTestMBOXProvider(path string) *MBOXProvider {
}
func TestMBOXProviderMailboxes(t *testing.T) {
provider := newTestMBOXProvider("")
tests := []struct {
provider *MBOXProvider
includeEmpty bool
wantMailboxes []Mailbox
}{
{true, []Mailbox{
{newTestMBOXProvider(""), true, []Mailbox{
{Name: "All Mail"},
{Name: "Foo"},
{Name: "Bar"},
{Name: "Inbox"},
}},
{false, []Mailbox{
{newTestMBOXProvider(""), false, []Mailbox{
{Name: "All Mail"},
{Name: "Foo"},
{Name: "Bar"},
{Name: "Inbox"},
}},
{newTestMBOXProvider("testdata/mbox-applemail"), true, []Mailbox{
{Name: "All Mail"},
{Name: "Foo"},
{Name: "Bar"},
}},
}
for _, tc := range tests {
tc := tc
t.Run(fmt.Sprintf("%v", tc.includeEmpty), func(t *testing.T) {
mailboxes, err := provider.Mailboxes(tc.includeEmpty, false)
mailboxes, err := tc.provider.Mailboxes(tc.includeEmpty, false)
r.NoError(t, err)
r.Equal(t, tc.wantMailboxes, mailboxes)
r.ElementsMatch(t, tc.wantMailboxes, mailboxes)
})
}
}
@ -67,14 +75,47 @@ func TestMBOXProviderTransferTo(t *testing.T) {
defer rulesClose()
setupMBOXRules(rules)
testTransferTo(t, rules, provider, []string{
msgs := testTransferTo(t, rules, provider, []string{
"All Mail.mbox:1",
"All Mail.mbox:2",
"Foo.mbox:1",
"Inbox.mbox:1",
})
got := map[string][]string{}
for _, msg := range msgs {
got[msg.ID] = msg.targetNames()
}
r.Equal(t, map[string][]string{
"All Mail.mbox:1": {"Archive", "Foo"}, // Bar is not in rules.
"All Mail.mbox:2": {"Archive", "Foo"},
"Foo.mbox:1": {"Foo"},
"Inbox.mbox:1": {"Inbox"},
}, got)
}
func TestMBOXProviderTransferToAppleMail(t *testing.T) {
provider := newTestMBOXProvider("testdata/mbox-applemail")
rules, rulesClose := newTestRules(t)
defer rulesClose()
setupMBOXRules(rules)
msgs := testTransferTo(t, rules, provider, []string{
"All Mail.mbox/mbox:1",
"All Mail.mbox/mbox:2",
})
got := map[string][]string{}
for _, msg := range msgs {
got[msg.ID] = msg.targetNames()
}
r.Equal(t, map[string][]string{
"All Mail.mbox/mbox:1": {"Archive", "Foo"}, // Bar is not in rules.
"All Mail.mbox/mbox:2": {"Archive", "Foo"},
}, got)
}
func TestMBOXProviderTransferFrom(t *testing.T) {
dir, err := ioutil.TempDir("", "eml")
dir, err := ioutil.TempDir("", "mbox")
r.NoError(t, err)
defer os.RemoveAll(dir) //nolint[errcheck]
@ -94,7 +135,7 @@ func TestMBOXProviderTransferFrom(t *testing.T) {
}
func TestMBOXProviderTransferFromTo(t *testing.T) {
dir, err := ioutil.TempDir("", "eml")
dir, err := ioutil.TempDir("", "mbox")
r.NoError(t, err)
defer os.RemoveAll(dir) //nolint[errcheck]
@ -103,23 +144,80 @@ func TestMBOXProviderTransferFromTo(t *testing.T) {
rules, rulesClose := newTestRules(t)
defer rulesClose()
setupEMLRules(rules)
setupMBOXRules(rules)
testTransferFromTo(t, rules, source, target, 5*time.Second)
checkMBOXFileStructure(t, dir, []string{
"Archive.mbox",
"Foo.mbox",
"Inbox.mbox",
})
}
func TestMBOXProviderGetMessageRules(t *testing.T) {
provider := newTestMBOXProvider("")
body := []byte(`Subject: Test
X-Gmail-Labels: foo,bar
`)
rules := transferRules{
rules: map[string]*Rule{
"1": {Active: true, SourceMailbox: Mailbox{Name: "folder"}},
"2": {Active: false, SourceMailbox: Mailbox{Name: "foo"}},
"3": {Active: true, SourceMailbox: Mailbox{Name: "bar"}},
"4": {Active: false, SourceMailbox: Mailbox{Name: "baz"}},
"5": {Active: true, SourceMailbox: Mailbox{Name: "other"}},
},
}
gotRules := provider.getMessageRules(rules, "folder", "id", body)
r.Equal(t, 2, len(gotRules))
r.Equal(t, "folder", gotRules[0].SourceMailbox.Name)
r.Equal(t, "bar", gotRules[1].SourceMailbox.Name)
}
func TestMBOXProviderGetMessageTargetsReturnsOnlyOneFolder(t *testing.T) {
provider := newTestMBOXProvider("")
folderA := Mailbox{Name: "Folder A", IsExclusive: true}
folderB := Mailbox{Name: "Folder B", IsExclusive: true}
labelA := Mailbox{Name: "Label A", IsExclusive: false}
labelB := Mailbox{Name: "Label B", IsExclusive: false}
labelC := Mailbox{Name: "Label C", IsExclusive: false}
rule1 := &Rule{TargetMailboxes: []Mailbox{folderA, labelA, labelB}}
rule2 := &Rule{TargetMailboxes: []Mailbox{folderB, labelC}}
rule3 := &Rule{TargetMailboxes: []Mailbox{folderB}}
tests := []struct {
rules []*Rule
wantMailboxes []Mailbox
}{
{[]*Rule{}, []Mailbox{}},
{[]*Rule{rule1}, []Mailbox{folderA, labelA, labelB}},
{[]*Rule{rule1, rule2}, []Mailbox{folderA, labelA, labelB, labelC}},
{[]*Rule{rule1, rule3}, []Mailbox{folderA, labelA, labelB}},
{[]*Rule{rule3, rule1}, []Mailbox{folderB, labelA, labelB}},
}
for _, tc := range tests {
tc := tc
t.Run(fmt.Sprintf("%v", tc.rules), func(t *testing.T) {
mailboxes := provider.getMessageTargets(tc.rules, "", []byte(""))
r.Equal(t, tc.wantMailboxes, mailboxes)
})
}
}
func setupMBOXRules(rules transferRules) {
_ = rules.setRule(Mailbox{Name: "All Mail"}, []Mailbox{{Name: "Archive"}}, 0, 0)
_ = rules.setRule(Mailbox{Name: "Inbox"}, []Mailbox{{Name: "Inbox"}}, 0, 0)
_ = rules.setRule(Mailbox{Name: "Foo"}, []Mailbox{{Name: "Foo"}}, 0, 0)
}
func checkMBOXFileStructure(t *testing.T, root string, expectedFiles []string) {
files, err := getFilePathsWithSuffix(root, ".mbox")
files, err := getAllPathsWithSuffix(root, ".mbox")
r.NoError(t, err)
r.Equal(t, expectedFiles, files)
}

View File

@ -123,7 +123,7 @@ func (p *PMAPIProvider) transferTo(rule *Rule, progress *Progress, ch chan<- Mes
}
msgID := fmt.Sprintf("%s_%s", rule.SourceMailbox.ID, pmapiMessage.ID)
progress.addMessage(msgID, rule)
progress.addMessage(msgID, []string{rule.SourceMailbox.Name}, rule.TargetMailboxNames())
msg, err := p.exportMessage(rule, progress, pmapiMessage.ID, msgID, skipEncryptedMessages)
progress.messageExported(msgID, msg.Body, err)
if err == nil {
@ -177,7 +177,7 @@ func (p *PMAPIProvider) exportMessage(rule *Rule, progress *Progress, pmapiMsgID
ID: msgID,
Unread: unread,
Body: body,
Source: rule.SourceMailbox,
Sources: []Mailbox{rule.SourceMailbox},
Targets: rule.TargetMailboxes,
}, nil
}

View File

@ -177,6 +177,10 @@ func (p *PMAPIProvider) transferMessage(rules transferRules, progress *Progress,
return
}
if progress.shouldStop() {
return
}
importMsgReqSize := len(importMsgReq.Body)
if p.nextImportRequestsSize+importMsgReqSize > pmapiImportBatchMaxSize || len(p.nextImportRequests) == pmapiImportBatchMaxItems {
preparedImportRequestsCh <- p.nextImportRequests

View File

@ -72,7 +72,8 @@ func (p *PMAPIProvider) tryReconnect() error {
func (p *PMAPIProvider) listMessages(filter *pmapi.MessagesFilter) (messages []*pmapi.Message, count int, err error) {
err = p.ensureConnection(func() error {
key := fmt.Sprintf("%s_%d", filter.LabelID, filter.Page)
// Sort is used in the key so the filter is different for estimating and real fetching.
key := fmt.Sprintf("%s_%s_%d", filter.LabelID, filter.Sort, filter.Page)
p.timeIt.start("listing", key)
defer p.timeIt.stop("listing", key)
@ -117,8 +118,10 @@ func (p *PMAPIProvider) createDraft(msgSourceID string, message *pmapi.Message,
func (p *PMAPIProvider) createAttachment(msgSourceID string, att *pmapi.Attachment, r io.Reader, sig io.Reader) (created *pmapi.Attachment, err error) {
err = p.ensureConnection(func() error {
p.timeIt.start("upload", msgSourceID)
defer p.timeIt.stop("upload", msgSourceID)
// Use some attributes from attachment to have unique key for each call.
key := fmt.Sprintf("%s_%s_%d", msgSourceID, att.Name, att.Size)
p.timeIt.start("upload", key)
defer p.timeIt.stop("upload", key)
created, err = p.client().CreateAttachment(att, r, sig)
return err

View File

@ -43,7 +43,7 @@ hello
`, subject))
}
func testTransferTo(t *testing.T, rules transferRules, provider SourceProvider, expectedMessageIDs []string) {
func testTransferTo(t *testing.T, rules transferRules, provider SourceProvider, expectedMessageIDs []string) []Message {
progress := newProgress(log, nil)
drainProgressUpdateChannel(&progress)
@ -53,13 +53,17 @@ func testTransferTo(t *testing.T, rules transferRules, provider SourceProvider,
close(ch)
}()
msgs := []Message{}
gotMessageIDs := []string{}
for msg := range ch {
msgs = append(msgs, msg)
gotMessageIDs = append(gotMessageIDs, msg.ID)
}
r.ElementsMatch(t, expectedMessageIDs, gotMessageIDs)
r.Empty(t, progress.GetFailedMessages())
return msgs
}
func testTransferFrom(t *testing.T, rules transferRules, provider TargetProvider, messages []Message) {
@ -69,7 +73,7 @@ func testTransferFrom(t *testing.T, rules transferRules, provider TargetProvider
ch := make(chan Message)
go func() {
for _, message := range messages {
progress.addMessage(message.ID, nil)
progress.addMessage(message.ID, []string{}, []string{})
progress.messageExported(message.ID, []byte(""), nil)
ch <- message
}

View File

@ -114,7 +114,7 @@ type messageReport struct {
SourceID string
TargetID string
BodyHash string
SourceMailbox string
SourceMailboxes []string
TargetMailboxes []string
Error string
@ -130,8 +130,8 @@ func newMessageReportFromMessageStatus(messageStatus *MessageStatus, includePriv
SourceID: messageStatus.SourceID,
TargetID: messageStatus.targetID,
BodyHash: messageStatus.bodyHash,
SourceMailbox: messageStatus.rule.SourceMailbox.Name,
TargetMailboxes: messageStatus.rule.TargetMailboxNames(),
SourceMailboxes: messageStatus.sourceNames,
TargetMailboxes: messageStatus.targetNames,
Error: messageStatus.GetErrorMessage(),
}

View File

@ -0,0 +1,16 @@
From - Mon May 4 16:40:31 2020
From: Bridge Test <bridgetest@pm.test>
To: Bridge Test <bridgetest@protonmail.com>
Subject: Test 1
X-Gmail-Labels: Foo,Bar
hello
From - Mon May 4 16:40:31 2020
From: Bridge Test <bridgetest@pm.test>
To: Bridge Test <bridgetest@protonmail.com>
Subject: Test 2
X-Gmail-Labels: Foo
hello

View File

@ -0,0 +1,16 @@
From - Mon May 4 16:40:31 2020
From: Bridge Test <bridgetest@pm.test>
To: Bridge Test <bridgetest@protonmail.com>
Subject: Test 1
X-Gmail-Labels: Foo,Bar
hello
From - Mon May 4 16:40:31 2020
From: Bridge Test <bridgetest@pm.test>
To: Bridge Test <bridgetest@protonmail.com>
Subject: Test 2
X-Gmail-Labels: Foo
hello

View File

@ -24,9 +24,11 @@ import (
"net/mail"
"net/textproto"
"path/filepath"
"runtime"
"sort"
"strings"
"github.com/ProtonMail/go-rfc5322"
"github.com/pkg/errors"
)
@ -81,7 +83,7 @@ func getFolderNamesWithFileSuffix(root, fileSuffix string) ([]string, error) {
// getFilePathsWithSuffix collects all file names with `suffix` under `root`.
// File names will be with relative path based to `root`.
func getFilePathsWithSuffix(root, suffix string) ([]string, error) {
fileNames, err := getFilePathsWithSuffixInner("", root, suffix)
fileNames, err := getFilePathsWithSuffixInner("", root, suffix, false)
if err != nil {
return nil, err
}
@ -89,7 +91,18 @@ func getFilePathsWithSuffix(root, suffix string) ([]string, error) {
return fileNames, err
}
func getFilePathsWithSuffixInner(prefix, root, suffix string) ([]string, error) {
// getAllPathsWithSuffix is the same as getFilePathsWithSuffix but includes
// also directories.
func getAllPathsWithSuffix(root, suffix string) ([]string, error) {
fileNames, err := getFilePathsWithSuffixInner("", root, suffix, true)
if err != nil {
return nil, err
}
sort.Strings(fileNames)
return fileNames, err
}
func getFilePathsWithSuffixInner(prefix, root, suffix string, includeDir bool) ([]string, error) {
fileNames := []string{}
files, err := ioutil.ReadDir(root)
@ -103,10 +116,14 @@ func getFilePathsWithSuffixInner(prefix, root, suffix string) ([]string, error)
fileNames = append(fileNames, filepath.Join(prefix, file.Name()))
}
} else {
if includeDir && strings.HasSuffix(file.Name(), suffix) {
fileNames = append(fileNames, filepath.Join(prefix, file.Name()))
}
subfolderFileNames, err := getFilePathsWithSuffixInner(
filepath.Join(prefix, file.Name()),
filepath.Join(root, file.Name()),
suffix,
includeDir,
)
if err != nil {
return nil, err
@ -120,14 +137,21 @@ func getFilePathsWithSuffixInner(prefix, root, suffix string) ([]string, error)
// getMessageTime returns time of the message specified in the message header.
func getMessageTime(body []byte) (int64, error) {
mailHeader, err := getMessageHeader(body)
hdr, err := getMessageHeader(body)
if err != nil {
return 0, err
}
if t, err := mailHeader.Date(); err == nil && !t.IsZero() {
return t.Unix(), nil
t, err := rfc5322.ParseDateTime(hdr.Get("Date"))
if err != nil {
return 0, err
}
return 0, nil
if t.IsZero() {
return 0, nil
}
return t.Unix(), nil
}
// getMessageHeader returns headers of the message body.
@ -139,3 +163,24 @@ func getMessageHeader(body []byte) (mail.Header, error) {
}
return mail.Header(header), nil
}
// sanitizeFileName replaces problematic special characters with underscore.
func sanitizeFileName(fileName string) string {
if len(fileName) == 0 {
return fileName
}
if runtime.GOOS != "windows" && (fileName[0] == '-' || fileName[0] == '.') { //nolint[goconst]
fileName = "_" + fileName[1:]
}
return strings.Map(func(r rune) rune {
switch r {
case '\\', '/', ':', '*', '?', '"', '<', '>', '|':
return '_'
case '[', ']', '(', ')', '{', '}', '^', '#', '%', '&', '!', '@', '+', '=', '\'', '~':
if runtime.GOOS != "windows" {
return '_'
}
}
return r
}, fileName)
}

View File

@ -21,6 +21,7 @@ import (
"io/ioutil"
"os"
"path/filepath"
"runtime"
"testing"
r "github.com/stretchr/testify/require"
@ -38,6 +39,7 @@ func TestGetFolderNames(t *testing.T) {
"",
[]string{
"bar",
"bar.mbox",
"baz",
filepath.Base(root),
"foo",
@ -94,6 +96,13 @@ func TestGetFilePathsWithSuffix(t *testing.T) {
"test/foo/msg9.eml",
},
},
{
".mbox",
[]string{
"bar.mbox",
"foo.mbox",
},
},
{
".txt",
[]string{
@ -108,7 +117,7 @@ func TestGetFilePathsWithSuffix(t *testing.T) {
for _, tc := range tests {
tc := tc
t.Run(tc.suffix, func(t *testing.T) {
paths, err := getFilePathsWithSuffix(root, tc.suffix)
paths, err := getAllPathsWithSuffix(root, tc.suffix)
r.NoError(t, err)
r.Equal(t, tc.wantPaths, paths)
})
@ -124,6 +133,7 @@ func createTestingFolderStructure(t *testing.T) (string, func()) {
"foo/baz",
"test/foo",
"qwerty",
"bar.mbox",
} {
err = os.MkdirAll(filepath.Join(root, path), os.ModePerm)
r.NoError(t, err)
@ -141,6 +151,8 @@ func createTestingFolderStructure(t *testing.T) (string, func()) {
"test/foo/msg9.eml",
"msg10.eml",
"info.txt",
"foo.mbox",
"bar.mbox/mbox", // Apple Mail mbox export format.
} {
f, err := os.Create(filepath.Join(root, path))
r.NoError(t, err)
@ -188,3 +200,26 @@ Body
r.Equal(t, header.Get("subject"), "Hello")
r.Equal(t, header.Get("from"), "user@example.com")
}
func TestSanitizeFileName(t *testing.T) {
tests := map[string]string{
"hello": "hello",
"a\\b/c:*?d\"<>|e": "a_b_c___d____e",
}
if runtime.GOOS == "darwin" || runtime.GOOS == "linux" {
tests[".hello"] = "_hello"
tests["-hello"] = "_hello"
}
if runtime.GOOS == "windows" {
tests["[hello]&@=~~"] = "_hello______"
}
for path, wantPath := range tests {
path := path
wantPath := wantPath
t.Run(path, func(t *testing.T) {
gotPath := sanitizeFileName(path)
r.Equal(t, wantPath, gotPath)
})
}
}

View File

@ -45,7 +45,7 @@ func syncFolders(localPath, updatePath string) (err error) {
}
func removeMissing(folderToCleanPath, itemsToKeepPath string) (err error) {
log.Debug("remove missing")
log.WithField("from", folderToCleanPath).Debug("Remove missing.")
// Create list of files.
existingRelPaths := map[string]bool{}
err = filepath.Walk(itemsToKeepPath, func(keepThis string, _ os.FileInfo, walkErr error) error {
@ -56,7 +56,7 @@ func removeMissing(folderToCleanPath, itemsToKeepPath string) (err error) {
if walkErr != nil {
return walkErr
}
log.Debug("path to keep ", relPath)
log.WithField("path", relPath).Trace("Keep the path.")
existingRelPaths[relPath] = true
return nil
})
@ -95,12 +95,18 @@ func removeMissing(folderToCleanPath, itemsToKeepPath string) (err error) {
}
func restoreFromBackup(backupDir, localPath string) {
log.Error("recovering from ", backupDir, " to ", localPath)
_ = copyRecursively(backupDir, localPath)
log.WithField("from", backupDir).
WithField("to", localPath).
Error("recovering")
if err := copyRecursively(backupDir, localPath); err != nil {
log.WithField("from", backupDir).
WithField("to", localPath).
Error("Not able to recover.")
}
}
func createBackup(srcFile, dstDir string) (err error) {
log.Debug("backup ", srcFile, " in ", dstDir)
log.WithField("from", srcFile).WithField("to", dstDir).Debug("Create backup")
if err = mkdirAllClear(dstDir); err != nil {
return
}

View File

@ -107,7 +107,7 @@ func NewImportExport(updateTempDir string) *Updates {
versionFileBaseName: "current_version_ie",
updateFileBaseName: "ie/ie_upgrade",
linuxFileBaseName: "ie/protonmail-import-export-app",
macAppBundleName: "Import-Export app.app",
macAppBundleName: "ProtonMail Import-Export app.app",
}
}
@ -310,7 +310,9 @@ func (u *Updates) StartUpgrade(currentStatus chan<- Progress) { // nolint[funlen
status.UpdateDescription(InfoUpgrading)
switch runtime.GOOS {
case "windows": //nolint[goconst]
installerFile := strings.Split(u.winInstallerFile, "/")[1]
// Cannot use filepath.Base on windows it has different delimiter
split := strings.Split(u.winInstallerFile, "/")
installerFile := split[len(split)-1]
cmd := exec.Command("./" + installerFile) // nolint[gosec]
cmd.Dir = u.updateTempDir
status.Err = cmd.Start()
@ -326,10 +328,15 @@ func (u *Updates) StartUpgrade(currentStatus chan<- Progress) { // nolint[funlen
localPath = filepath.Dir(localPath) // .app
updatePath := filepath.Join(u.updateTempDir, u.macAppBundleName)
log.Warn("localPath ", localPath)
log.Warn("updatePath ", updatePath)
log.WithField("local", localPath).
WithField("update", updatePath).
Info("Syncing folders..")
status.Err = syncFolders(localPath, updatePath)
if status.Err != nil {
log.WithField("from", localPath).
WithField("to", updatePath).
WithError(status.Err).
Error("Sync failed.")
return
}
status.UpdateDescription(InfoRestartApp)

View File

@ -5,11 +5,12 @@
package mocks
import (
reflect "reflect"
store "github.com/ProtonMail/proton-bridge/internal/store"
credentials "github.com/ProtonMail/proton-bridge/internal/users/credentials"
pmapi "github.com/ProtonMail/proton-bridge/pkg/pmapi"
gomock "github.com/golang/mock/gomock"
reflect "reflect"
)
// MockConfiger is a mock of Configer interface

View File

@ -437,7 +437,7 @@ func (u *User) SwitchAddressMode() (err error) {
u.lock.Lock()
defer u.lock.Unlock()
u.closeAllConnections()
u.CloseAllConnections()
if u.store == nil {
err = errors.New("store is not initialised")
@ -509,7 +509,7 @@ func (u *User) Logout() (err error) {
// Do not close whole store, just event loop. Some information might be needed offline (e.g. addressID)
u.closeEventLoop()
u.closeAllConnections()
u.CloseAllConnections()
runtime.GC()
@ -532,8 +532,8 @@ func (u *User) closeEventLoop() {
u.store.CloseEventLoop()
}
// closeAllConnections calls CloseConnection for all users addresses.
func (u *User) closeAllConnections() {
// CloseAllConnections calls CloseConnection for all users addresses.
func (u *User) CloseAllConnections() {
for _, address := range u.creds.EmailList() {
u.CloseConnection(address)
}

View File

@ -186,7 +186,7 @@ func (u *Users) watchAPIAuths() {
func (u *Users) closeAllConnections() {
for _, user := range u.users {
user.closeAllConnections()
user.CloseAllConnections()
}
}

View File

@ -19,9 +19,7 @@ package message
import (
"mime"
"net/mail"
"net/textproto"
"regexp"
"strings"
"time"
@ -86,10 +84,6 @@ func GetHeader(msg *pmapi.Message) textproto.MIMEHeader { //nolint[funlen]
}
if msg.ConversationID != "" {
h.Set("X-Pm-ConversationID-Id", msg.ConversationID)
if references := h.Get("References"); !strings.Contains(references, msg.ConversationID) {
references += " <" + msg.ConversationID + "@" + pmapi.ConversationIDDomain + ">"
h.Set("References", references)
}
}
return h
@ -141,46 +135,3 @@ func GetAttachmentHeader(att *pmapi.Attachment) textproto.MIMEHeader {
return h
}
var reEmailComment = regexp.MustCompile("[(][^)]*[)]") // nolint[gochecknoglobals]
// parseAddressComment removes the comments completely even though they should be allowed
// http://tools.wordtothewise.com/rfc/822
// NOTE: This should be supported in go>1.10 but it seems it's not ¯\_(ツ)_/¯
func parseAddressComment(raw string) string {
return reEmailComment.ReplaceAllString(raw, "")
}
func parseAddressList(val string) (addrs []*mail.Address, err error) {
if val == "" || val == "<>" {
return
}
addrs, err = mail.ParseAddressList(parseAddressComment(val))
if err == nil {
if addrs == nil {
addrs = []*mail.Address{}
}
return
}
// Probably missing encoding error -- try to at least parse addresses in brackets.
first := strings.Index(val, "<")
last := strings.LastIndex(val, ">")
if first < 0 || last < 0 || first >= last {
return
}
var addrList []string
open := first
for open < last && 0 <= open {
val = val[open:]
close := strings.Index(val, ">")
addrList = append(addrList, val[:close+1])
val = val[close:]
open = strings.Index(val, "<")
last = strings.LastIndex(val, ">")
}
val = strings.Join(addrList, ", ")
return mail.ParseAddressList(val)
}

27
pkg/message/init.go Normal file
View File

@ -0,0 +1,27 @@
// Copyright (c) 2020 Proton Technologies AG
//
// This file is part of ProtonMail Bridge.
//
// ProtonMail Bridge is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ProtonMail Bridge is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with ProtonMail Bridge. If not, see <https://www.gnu.org/licenses/>.
package message
import (
"github.com/ProtonMail/go-rfc5322"
pmmime "github.com/ProtonMail/proton-bridge/pkg/mime"
)
func init() { // nolint[noinit]
rfc5322.CharsetReader = pmmime.CharsetReader
}

View File

@ -26,6 +26,7 @@ import (
"net/textproto"
"strings"
"github.com/ProtonMail/go-rfc5322"
"github.com/ProtonMail/proton-bridge/pkg/message/parser"
pmmime "github.com/ProtonMail/proton-bridge/pkg/mime"
"github.com/ProtonMail/proton-bridge/pkg/pmapi"
@ -365,7 +366,6 @@ func attachPublicKey(p *parser.Part, key, keyName string) {
})
}
// NOTE: We should use our own ParseAddressList here.
func parseMessageHeader(m *pmapi.Message, h message.Header) error { // nolint[funlen]
mimeHeader, err := toMailHeader(h)
if err != nil {
@ -373,59 +373,64 @@ func parseMessageHeader(m *pmapi.Message, h message.Header) error { // nolint[fu
}
m.Header = mimeHeader
if err := forEachDecodedHeaderField(h, func(key, val string) error {
switch strings.ToLower(key) {
fields := h.Fields()
for fields.Next() {
switch strings.ToLower(fields.Key()) {
case "subject":
m.Subject = val
s, err := fields.Text()
if err != nil {
if s, err = pmmime.DecodeHeader(fields.Value()); err != nil {
return errors.Wrap(err, "failed to parse subject")
}
}
m.Subject = s
case "from":
sender, err := parseAddressList(val)
sender, err := rfc5322.ParseAddressList(fields.Value())
if err != nil {
return err
return errors.Wrap(err, "failed to parse from")
}
if len(sender) > 0 {
m.Sender = sender[0]
}
case "to":
toList, err := parseAddressList(val)
toList, err := rfc5322.ParseAddressList(fields.Value())
if err != nil {
return err
return errors.Wrap(err, "failed to parse to")
}
m.ToList = toList
case "reply-to":
replyTos, err := parseAddressList(val)
replyTos, err := rfc5322.ParseAddressList(fields.Value())
if err != nil {
return err
return errors.Wrap(err, "failed to parse reply-to")
}
m.ReplyTos = replyTos
case "cc":
ccList, err := parseAddressList(val)
ccList, err := rfc5322.ParseAddressList(fields.Value())
if err != nil {
return err
return errors.Wrap(err, "failed to parse cc")
}
m.CCList = ccList
case "bcc":
bccList, err := parseAddressList(val)
bccList, err := rfc5322.ParseAddressList(fields.Value())
if err != nil {
return err
return errors.Wrap(err, "failed to parse bcc")
}
m.BCCList = bccList
case "date":
date, err := mail.ParseDate(val)
date, err := rfc5322.ParseDateTime(fields.Value())
if err != nil {
return err
return errors.Wrap(err, "failed to parse date")
}
m.Time = date.Unix()
}
return nil
}); err != nil {
return err
}
return nil
@ -469,29 +474,6 @@ func parseAttachment(h message.Header) (*pmapi.Attachment, error) {
return att, nil
}
func forEachDecodedHeaderField(h message.Header, fn func(string, string) error) error {
fields := h.Fields()
for fields.Next() {
text, err := fields.Text()
if err != nil {
if !message.IsUnknownCharset(err) {
return err
}
if text, err = pmmime.DecodeHeader(fields.Value()); err != nil {
return err
}
}
if err := fn(fields.Key(), text); err != nil {
return err
}
}
return nil
}
func toMailHeader(h message.Header) (mail.Header, error) {
mimeHeader := make(mail.Header)
@ -517,3 +499,26 @@ func toMIMEHeader(h message.Header) (textproto.MIMEHeader, error) {
return mimeHeader, nil
}
func forEachDecodedHeaderField(h message.Header, fn func(string, string) error) error {
fields := h.Fields()
for fields.Next() {
text, err := fields.Text()
if err != nil {
if !message.IsUnknownCharset(err) {
return err
}
if text, err = pmmime.DecodeHeader(fields.Value()); err != nil {
return err
}
}
if err := fn(fields.Key(), text); err != nil {
return err
}
}
return nil
}

View File

@ -32,7 +32,7 @@ type Parser struct {
func New(r io.Reader) (*Parser, error) {
p := new(Parser)
entity, err := message.Read(r)
entity, err := message.Read(newEndOfMailTrimmer(r))
if err != nil && !message.IsUnknownCharset(err) {
return nil, err
}

View File

@ -0,0 +1,56 @@
// Copyright (c) 2020 Proton Technologies AG
//
// This file is part of ProtonMail Bridge.
//
// ProtonMail Bridge is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ProtonMail Bridge is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with ProtonMail Bridge. If not, see <https://www.gnu.org/licenses/>.
package parser
import (
"bytes"
"io"
)
const endOfMail = "\r\n.\r\n"
// endOfMailTrimmer wraps a reader to trim the End-Of-Mail indicator at the end
// of the input, if present.
//
// During SMTP sending of a message, the DATA command indicates that you are
// about to send the text (or body) of the message. The message text must end
// with "\r\n.\r\n." I'm 99% sure that these 5 bytes should not be considered
// part of the message body. However, some mail servers keep them as part of
// the message, which our parser sometimes doesn't like. Therefore, we strip
// them if we find them.
type endOfMailTrimmer struct {
r io.Reader
buf bytes.Buffer
}
func newEndOfMailTrimmer(r io.Reader) *endOfMailTrimmer {
return &endOfMailTrimmer{r: r}
}
func (r *endOfMailTrimmer) Read(p []byte) (int, error) {
_, err := io.CopyN(&r.buf, r.r, int64(len(p)+len(endOfMail)-r.buf.Len()))
if err != nil && err != io.EOF {
return 0, err
}
if err == io.EOF && bytes.HasSuffix(r.buf.Bytes(), []byte(endOfMail)) {
r.buf.Truncate(r.buf.Len() - len(endOfMail))
}
return r.buf.Read(p)
}

View File

@ -0,0 +1,55 @@
// Copyright (c) 2020 Proton Technologies AG
//
// This file is part of ProtonMail Bridge.
//
// ProtonMail Bridge is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ProtonMail Bridge is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with ProtonMail Bridge. If not, see <https://www.gnu.org/licenses/>.
package parser
import (
"io"
"strings"
"testing"
"github.com/stretchr/testify/assert"
)
func TestEndOfMailTrimmer(t *testing.T) {
var tests = []struct {
in string
out string
}{
{"string without eom", "string without eom"},
{"string with eom\r\n.\r\n", "string with eom"},
{"string with eom\r\n.\r\nin the middle", "string with eom\r\n.\r\nin the middle"},
}
for _, tt := range tests {
t.Run(tt.in, func(t *testing.T) {
res := dumbRead(newEndOfMailTrimmer(strings.NewReader(tt.in)))
assert.Equal(t, tt.out, string(res))
})
}
}
func dumbRead(r io.Reader) []byte {
out := []byte{}
b := make([]byte, 1)
for _, err := r.Read(b); err == nil; _, err = r.Read(b) {
out = append(out, b...)
}
return out
}

View File

@ -35,7 +35,7 @@ func newWriter(root *Part) *Writer {
func (w *Writer) Write(ww io.Writer) error {
if !w.root.is7BitClean() {
w.root.Header.Add("Content-Transfer-Encoding", "base64")
w.root.Header.Set("Content-Transfer-Encoding", "base64")
}
msgWriter, err := message.CreateWriter(ww, w.root.Header)
@ -68,7 +68,7 @@ func (w *Writer) write(writer *message.Writer, p *Part) error {
func (w *Writer) writeAsChild(writer *message.Writer, p *Part) error {
if !p.is7BitClean() {
p.Header.Add("Content-Transfer-Encoding", "base64")
p.Header.Set("Content-Transfer-Encoding", "base64")
}
childWriter, err := writer.CreatePart(p.Header)

View File

@ -467,6 +467,19 @@ func TestParseMultipartAlternativeLatin1(t *testing.T) {
assert.Equal(t, "*aoeuaoeu*\n\n", plainBody)
}
func TestParseWithTrailingEndOfMailIndicator(t *testing.T) {
f := getFileReader("text_html_trailing_end_of_mail.eml")
m, _, plainBody, _, err := Parse(f, "", "")
require.NoError(t, err)
assert.Equal(t, `"Sender" <sender@sender.com>`, m.Sender.String())
assert.Equal(t, `"Receiver" <receiver@receiver.com>`, m.ToList[0].String())
assert.Equal(t, "<!DOCTYPE html><html><head></head><body>boo!</body></html>", m.Body)
assert.Equal(t, "boo!", plainBody)
}
func getFileReader(filename string) io.Reader {
f, err := os.Open(filepath.Join("testdata", filename))
if err != nil {
@ -485,80 +498,3 @@ func readerToString(r io.Reader) string {
return string(b)
}
func TestRFC822AddressFormat(t *testing.T) { //nolint[funlen]
tests := []struct {
address string
expected []string
}{
{
" normal name <username@server.com>",
[]string{
"\"normal name\" <username@server.com>",
},
},
{
" \"comma, name\" <username@server.com>",
[]string{
"\"comma, name\" <username@server.com>",
},
},
{
" name <username@server.com> (ignore comment)",
[]string{
"\"name\" <username@server.com>",
},
},
{
" name (ignore comment) <username@server.com>, (Comment as name) username2@server.com",
[]string{
"\"name\" <username@server.com>",
"<username2@server.com>",
},
},
{
" normal name <username@server.com>, (comment)All.(around)address@(the)server.com",
[]string{
"\"normal name\" <username@server.com>",
"<All.address@server.com>",
},
},
{
" normal name <username@server.com>, All.(\"comma, in comment\")address@(the)server.com",
[]string{
"\"normal name\" <username@server.com>",
"<All.address@server.com>",
},
},
{
" \"normal name\" <username@server.com>, \"comma, name\" <address@server.com>",
[]string{
"\"normal name\" <username@server.com>",
"\"comma, name\" <address@server.com>",
},
},
{
" \"comma, one\" <username@server.com>, \"comma, two\" <address@server.com>",
[]string{
"\"comma, one\" <username@server.com>",
"\"comma, two\" <address@server.com>",
},
},
{
" \"comma, name\" <username@server.com>, another, name <address@server.com>",
[]string{
"\"comma, name\" <username@server.com>",
"\"another, name\" <address@server.com>",
},
},
}
for _, data := range tests {
result, err := parseAddressList(data.address)
assert.NoError(t, err)
assert.Len(t, result, len(data.expected))
for i, result := range result {
assert.Equal(t, data.expected[i], result.String())
}
}
}

View File

@ -0,0 +1,8 @@
From: "Sender" <sender@sender.com>
To: "Receiver" <receiver@receiver.com>
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
PCFET0NUWVBFIEhUTUw+CjxodG1sPjxib2R5PmJvbyE8L2JvZHk+PC9odG1sPg==
.

View File

@ -32,17 +32,19 @@ import (
"golang.org/x/text/encoding/htmlindex"
)
var wordDec = &mime.WordDecoder{
CharsetReader: func(charset string, input io.Reader) (io.Reader, error) {
dec, err := SelectDecoder(charset)
if err != nil {
return nil, err
}
if dec == nil { // utf-8
return input, nil
}
return dec.Reader(input), nil
},
func CharsetReader(charset string, input io.Reader) (io.Reader, error) {
dec, err := SelectDecoder(charset)
if err != nil {
return nil, err
}
if dec == nil { // utf-8
return input, nil
}
return dec.Reader(input), nil
}
var WordDec = &mime.WordDecoder{
CharsetReader: CharsetReader,
}
// Expects trimmed lowercase.
@ -180,7 +182,7 @@ func SelectDecoder(charset string) (decoder *encoding.Decoder, err error) {
// DecodeHeader if needed. Returns error if raw contains non-utf8 characters.
func DecodeHeader(raw string) (decoded string, err error) {
if decoded, err = wordDec.DecodeHeader(raw); err != nil {
if decoded, err = WordDec.DecodeHeader(raw); err != nil {
decoded = raw
}
if !utf8.ValidString(decoded) {

View File

@ -0,0 +1,94 @@
// Copyright (c) 2020 Proton Technologies AG
//
// This file is part of ProtonMail Bridge.
//
// ProtonMail Bridge is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ProtonMail Bridge is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with ProtonMail Bridge. If not, see <https://www.gnu.org/licenses/>.
package pmapi
import (
"errors"
"fmt"
"net/http"
"time"
)
const protonStatusURL = "http://protonstatus.com/vpn_status"
// ErrNoInternetConnection indicates that both protonstatus and the API are unreachable.
var ErrNoInternetConnection = errors.New("no internet connection")
// CheckConnection returns an error if there is no internet connection.
// This should be moved to the ConnectionManager when it is implemented.
func (cm *ClientManager) CheckConnection() error {
// We use a normal dialer here which doesn't check tls fingerprints.
client := &http.Client{Timeout: time.Second * 10}
// Do not cumulate timeouts, use goroutines.
retStatus := make(chan error)
retAPI := make(chan error)
// vpn_status endpoint is fast and returns only OK. We check the connection only.
go checkConnection(client, protonStatusURL, retStatus)
// Check of API reachability also uses a fast endpoint.
go checkConnection(client, cm.GetRootURL()+"/tests/ping", retAPI)
errStatus := <-retStatus
errAPI := <-retAPI
switch {
case errStatus == nil && errAPI == nil:
return nil
case errStatus == nil && errAPI != nil:
cm.log.Error("ProtonStatus is reachable but API is not")
return ErrAPINotReachable
case errStatus != nil && errAPI == nil:
cm.log.Warn("API is reachable but protonstatus is not")
return nil
case errStatus != nil && errAPI != nil:
cm.log.Error("Both ProtonStatus and API are unreachable")
return ErrNoInternetConnection
}
return nil
}
// CheckConnection returns an error if there is no internet connection.
func CheckConnection() error {
client := &http.Client{Timeout: time.Second * 10}
retStatus := make(chan error)
go checkConnection(client, protonStatusURL, retStatus)
return <-retStatus
}
func checkConnection(client *http.Client, url string, errorChannel chan error) {
resp, err := client.Get(url)
if err != nil {
errorChannel <- err
return
}
_ = resp.Body.Close()
if resp.StatusCode != 200 {
errorChannel <- fmt.Errorf("HTTP status code %d", resp.StatusCode)
return
}
errorChannel <- nil
}

View File

@ -305,72 +305,6 @@ func (cm *ClientManager) GetAuthUpdateChannel() chan ClientAuth {
return cm.authUpdates
}
// ErrNoInternetConnection indicates that both protonstatus and the API are unreachable.
var ErrNoInternetConnection = errors.New("no internet connection")
// CheckConnection returns an error if there is no internet connection.
// This should be moved to the ConnectionManager when it is implemented.
func (cm *ClientManager) CheckConnection() error {
client := getHTTPClient(cm.config, cm.roundTripper, cm.cookieJar)
// Do not cumulate timeouts, use goroutines.
retStatus := make(chan error)
retAPI := make(chan error)
// vpn_status endpoint is fast and returns only OK. We check the connection only.
go checkConnection(client, "https://protonstatus.com/vpn_status", retStatus)
// Check of API reachability also uses a fast endpoint.
go checkConnection(client, cm.GetRootURL()+"/tests/ping", retAPI)
errStatus := <-retStatus
errAPI := <-retAPI
switch {
case errStatus == nil && errAPI == nil:
return nil
case errStatus == nil && errAPI != nil:
cm.log.Error("ProtonStatus is reachable but API is not")
return ErrAPINotReachable
case errStatus != nil && errAPI == nil:
cm.log.Warn("API is reachable but protonstatus is not")
return nil
case errStatus != nil && errAPI != nil:
cm.log.Error("Both ProtonStatus and API are unreachable")
return ErrNoInternetConnection
}
return nil
}
// CheckConnection returns an error if there is no internet connection.
func CheckConnection() error {
client := &http.Client{Timeout: time.Second * 10}
retStatus := make(chan error)
go checkConnection(client, "https://protonstatus.com/vpn_status", retStatus)
return <-retStatus
}
func checkConnection(client *http.Client, url string, errorChannel chan error) {
resp, err := client.Get(url)
if err != nil {
errorChannel <- err
return
}
_ = resp.Body.Close()
if resp.StatusCode != 200 {
errorChannel <- fmt.Errorf("HTTP status code %d", resp.StatusCode)
return
}
errorChannel <- nil
}
// setTokenIfUnset sets the token for the given userID if it wasn't already set.
// The set token does not expire.
func (cm *ClientManager) setTokenIfUnset(userID, token string) {

View File

@ -15,7 +15,7 @@
// You should have received a copy of the GNU General Public License
// along with ProtonMail Bridge. If not, see <https://www.gnu.org/licenses/>.
// +build pmapi_env
// +build pmapi_qa
package pmapi

View File

@ -80,6 +80,7 @@ const (
type Label struct {
ID string
Name string
Path string
Color string
Order int `json:",omitempty"`
Display int // Not used for now, leave it empty.

View File

@ -29,6 +29,7 @@ import (
"net/http"
"net/mail"
"net/url"
"regexp"
"strconv"
"strings"
@ -149,8 +150,9 @@ const ConversationIDDomain = `protonmail.conversationid`
// InternalIDDomain is used as a placeholder for reference/message ID headers to improve compatibility with various clients.
const InternalIDDomain = `protonmail.internalid`
// InternalReferenceFormat describes format of the message ID (as regex) used for parsing reference headers.
const InternalReferenceFormat = `(?U)<.*@` + InternalIDDomain + `>`
// RxInternalReferenceFormat is compiled regexp which describes the match for
// a message ID used in reference headers.
var RxInternalReferenceFormat = regexp.MustCompile(`(?U)<(.+)@` + regexp.QuoteMeta(InternalIDDomain) + `>`) //nolint[gochecknoglobals]
// Message structure.
type Message struct {

View File

@ -382,7 +382,7 @@ func TestProxyProvider_UseProxy_RevertIfProxyStopsWorkingAndOriginalAPIIsReachab
// The error should be ErrAPINotReachable because the connection dropped intermittently but
// the original API is now reachable (see Alternative-Routing-v2 spec for details).
url, err = cm.switchToReachableServer()
require.EqualError(t, err, ErrAPINotReachable.Error())
require.Error(t, err)
require.Equal(t, rootURL, url)
require.Equal(t, rootURL, cm.getHost())
}

View File

@ -35,13 +35,21 @@ var ErrTLSMismatch = errors.New("no TLS fingerprint match found")
// TrustedAPIPins contains trusted public keys of the protonmail API and proxies.
// NOTE: the proxy pins are the same for all proxy servers, guaranteed by infra team ;)
var TrustedAPIPins = []string{ // nolint[gochecknoglobals]
// api.protonmail.ch
`pin-sha256="drtmcR2kFkM8qJClsuWgUzxgBkePfRCkRpqUesyDmeE="`, // current
`pin-sha256="YRGlaY0jyJ4Jw2/4M8FIftwbDIQfh8Sdro96CeEel54="`, // hot
`pin-sha256="AfMENBVvOS8MnISprtvyPsjKlPooqh8nMB/pvCrpJpw="`, // cold
`pin-sha256="EU6TS9MO0L/GsDHvVc9D5fChYLNy5JdGYpJw0ccgetM="`, // proxy main
`pin-sha256="iKPIHPnDNqdkvOnTClQ8zQAIKG0XavaPkcEo0LBAABA="`, // proxy backup 1
`pin-sha256="MSlVrBCdL0hKyczvgYVSRNm88RicyY04Q2y5qrBt0xA="`, // proxy backup 2
`pin-sha256="C2UxW0T1Ckl9s+8cXfjXxlEqwAfPM4HiW2y3UdtBeCw="`, // proxy backup 3
`pin-sha256="YRGlaY0jyJ4Jw2/4M8FIftwbDIQfh8Sdro96CeEel54="`, // hot backup
`pin-sha256="AfMENBVvOS8MnISprtvyPsjKlPooqh8nMB/pvCrpJpw="`, // cold backup
// protonmail.com
`pin-sha256="8joiNBdqaYiQpKskgtkJsqRxF7zN0C0aqfi8DacknnI="`, // current
`pin-sha256="JMI8yrbc6jB1FYGyyWRLFTmDNgIszrNEMGlgy972e7w="`, // hot backup
`pin-sha256="Iu44zU84EOCZ9vx/vz67/MRVrxF1IO4i4NIa8ETwiIY="`, // cold backup
// proxies
`pin-sha256="EU6TS9MO0L/GsDHvVc9D5fChYLNy5JdGYpJw0ccgetM="`, // main
`pin-sha256="iKPIHPnDNqdkvOnTClQ8zQAIKG0XavaPkcEo0LBAABA="`, // backup 1
`pin-sha256="MSlVrBCdL0hKyczvgYVSRNm88RicyY04Q2y5qrBt0xA="`, // backup 2
`pin-sha256="C2UxW0T1Ckl9s+8cXfjXxlEqwAfPM4HiW2y3UdtBeCw="`, // backup 3
}
// TLSReportURI is the address where TLS reports should be sent.

View File

@ -1,3 +1,3 @@
Fixed rare mail loss when moving from Spam folder
Limited log size
Fixed Linux font issues (mouse hover).
Ensured that conversations are properly threaded
Fixed Linux font issues (Fedora)
Better handling of Mime encrypted messages

View File

@ -1,2 +1,3 @@
Import from mbox files with long lines
Improvements to import from Yahoo accounts
Linux font issues - Fedora specific
App response to the user pausing and canceling import or export
• Upgrade errors

Some files were not shown because too many files have changed in this diff Show More