Bridge GoldenGate 1.5.0

- Ensured better message flow by refactoring both address and date parsing
- Improved secure connectivity checks
- Better deb packaging
- More robust error handling

- Ensured that conversations are properly threaded
- Fixed Linux font issues (Fedora)
- Better handling of Mime encrypted messages

.golangci.yml

@@ -20,6 +20,9 @@ issues:
         - gochecknoglobals
         - gochecknoinits
         - gosec
+    - path: pkg/message/rfc5322
+      linters:
+        - dupl
 
 linters-settings:
   godox:

BUILDS.md

@@ -19,7 +19,6 @@ Otherwise, the sending of crash reports will be disabled.
 export MSYSTEM=
 ```
 
-
 ### Build Bridge
 * in project root run
 
@@ -44,6 +43,12 @@ make build-ie
     * for `windows`, the binary will have the file extension `.exe` (e.g `proton-bridge.exe`)
     * for `darwin`, the application will be created with name of the project directory (e.g `proton-bridge.app`)
 
+### Tags
+Note that repository contains both Bridge and Import-Export apps and they are
+not released together. Therefore, each app has own tag prefix. Bridge tags
+starts with `br-` and Import-Export tags starts with `ie-`. Both tags continue
+with semantic versioning `MAJOR.MINOR.PATCH`. An example of full tag is
+`br-1.4.4` or `ie-1.1.2` (current versions in October 2020).
 
 ## Useful tests, lints and checks
 In order to be able to run following commands please install the development dependencies: 

Changelog.md

@@ -2,14 +2,87 @@
 
 Changelog [format](http://keepachangelog.com/en/1.0.0/)
 
+## Unreleased
+
+### Changed
+* Updated go-mbox dependency back to upstream.
+
+### Fixed
+* GODT-847 Waiting for unilateral update during deleting the message.
+* GODT-849 Show in error counts in the end also lost messages.
+* GODT-835 Do not include conversation ID in references to show properly conversation threads in clients.
+
+## [IE 1.2.0] Elbe
+
+### Added
+* GODT-763 Detect Gmail labels from All Mail mbox export (using X-Gmail-Label header).
+* GODT-834 Info about tags in BUILDS.md and link to Import-Export page in README.md.
+* GODT-777 Support Apple Mail MBOX export format.
+
+### Fixed
+* GODT-677 Windows IE: global import settings not fit in window.
+* GODT-794 Congo fails to update to Danube
+* GODT-749 Don't force PGP/Inline when sending plaintext messages.
+* GODT-764 Fix deadlock in integration tests for Import-Export.
+* GODT-662 Do not resume paused transfer progress after dismissing cancel popup.
+* GODT-772 Sanitize mailbox names for exporting to follow OS restrictions.
+* GODT-771 Show fatal errors after export is terminated.
+* GODT-779 Do not propagate updates when progress is stopped.
+* GODT-779 Unpause progress during fatal error to properly stop progress.
+* GODT-779 Stop ongoing transfer calls sooner (re-check after import request is generated).
+* Fix measurement of uploading attachments during transfer.
+* GODT-827 Do not spam sentry with bad ID by integration test.
+* GODT-700 Fix UTF-7 incompatibility.
+* GODT-837 Fix flaky TestFailUnpauseAndStops.
+* GODT-782 Don't use TLS pinning when checking connectivity status.
+
+### Changed
+* TLS pins conform to official list.
+
+
+## [Bridge 1.4.5] Forth
+
+### Fixed
+* GODT-829 Remove `NoInferior` to display sub-folders in apple mail.
+
+## [Bridge 1.4.4] Forth
+
+### Fixed
+* GODT-798 Replace, don't add, transfer encoding when making body 7-bit clean.
+* Move/Copy duplicate for emails with References in Outlook
+* CSB-247 Cannot update from 1.4.0
+
+
+## [Bridge 1.4.3] Forth
+
+### Changed
+* Reverted sending IMAP updates to be not blocking again.
+
+### Fixed
+* GODT-783 Settings flags by FLAGS (not using +/-FLAGS) do not change spam state.
+
+
+## [Bridge 1.4.2] Forth
+
+### Changed
+* GODT-761 Use label.Path instead of Name to partially support subfolders for webapp beta release.
+* GODT-765 Improve speed of checking whether message is deleted.
+
+
 ## [IE 1.1.2] Danube (beta 2020-09-xx)
 
 ### Fixed
 * GODT-770 Better handling of extraneous end-of-mail indicator.
+* GODT-776 Fix crash when IMAP client connects while account is logging in.
 
 ### Changed
 * Bump crypto version to v0.0.0-20200818122824-ed5d25e28db8
+* GODT-785 Clear separation of different message IDs in integration tests.
+### Changed
+* GODT-741 Import-Export shows "Unable to parse time" notice instead of zero time in error report window.
 
+* Bump crypto version to v0.0.0-20200818122824-ed5d25e28db8.
+* GODT-374 Allow to send calendar update multiple times.
 
 ## [IE 1.1.1] Danube (beta 2020-09-xx) [Bridge 1.4.1] Forth (beta 2020-09-xx)
 
@@ -18,11 +91,15 @@ Changelog [format](http://keepachangelog.com/en/1.0.0/)
 * GODT-752 Parsing non-utf8 multipart/alternative message.
 * GODT-752 Parsing message with duplicate charset parameter.
 
+
 ## [IE 1.1.0] Danube
 
 ### Fixed
 * GODT-703 Import-Export showed always at least one total message.
 * GODT-738 Fix for mbox files with long lines.
+### Fixed
+* GODT-732 Do not mix font awesome icon with regular text to avoid issues on Fedora.
+
 
 ## [Bridge 1.4.0] Forth
 

Makefile

@@ -10,8 +10,8 @@ TARGET_OS?=${GOOS}
 .PHONY: build build-ie build-nogui build-ie-nogui check-has-go
 
 # Keep version hardcoded so app build works also without Git repository.
-BRIDGE_APP_VERSION?=1.4.0-git
+BRIDGE_APP_VERSION?=1.5.0-git
-IE_APP_VERSION?=1.1.0-git
+IE_APP_VERSION?=1.2.0-git
 APP_VERSION:=${BRIDGE_APP_VERSION}
 SRC_ICO:=logo.ico
 SRC_ICNS:=Bridge.icns
@@ -57,7 +57,6 @@ ifeq "${TARGET_CMD}" "Import-Export"
     TGZ_TARGET:=ie_${TARGET_OS}_${REVISION}.tgz
 endif
 
-
 build: ${TGZ_TARGET}
 build-ie:
 	TARGET_CMD=Import-Export $(MAKE) build
@@ -265,7 +264,6 @@ run-ie-qt:
 run-ie-nogui:
 	TARGET_CMD=Import-Export $(MAKE) run-nogui
 
-
 clean-frontend-qt:
 	$(MAKE) -C internal/frontend/qt -f Makefile.local clean
 clean-frontend-qt-ie:
@@ -282,3 +280,8 @@ clean: clean-vendor
 	rm -rf cmd/Import-Export/deploy
 	rm -f build last.log mem.pprof main.go
 	rm -rf logo.ico icon.rc icon_windows.syso internal/frontend/qt/icon_windows.syso
+
+.PHONY: generate
+generate:
+	go generate ./...
+	$(MAKE) add-license

README.md

@@ -1,7 +1,7 @@
 # ProtonMail Bridge and Import Export app
 Copyright (c) 2020 Proton Technologies AG
 
-This repository holds the ProtonMail Bridge application.
+This repository holds the ProtonMail Bridge and the ProtonMail Import-Export applications.
 For a detailed build information see [BUILDS](./BUILDS.md).
 For licensing information see [COPYING](./COPYING.md).
 For contribution policy see [CONTRIBUTING](./CONTRIBUTING.md).
@@ -35,6 +35,8 @@ configure transfer rules (match source and target mailboxes, set time
 range limits and so on) and hit start. Once the transfer is complete,
 check the results.
 
+More details [on the public website](https://protonmail.com/import-export).
+
 ## Keychain
 You need to have a keychain in order to run the ProtonMail Bridge. On Mac or
 Windows, Bridge uses native credential managers. On Linux, use

go.mod

@@ -24,6 +24,7 @@ require (
 	github.com/abiosoft/ishell v2.0.0+incompatible
 	github.com/abiosoft/readline v0.0.0-20180607040430-155bce2042db // indirect
 	github.com/allan-simon/go-singleinstance v0.0.0-20160830203053-79edcfdc2dfc
+	github.com/antlr/antlr4 v0.0.0-20201020194047-0a7eaede42b0
 	github.com/certifi/gocertifi v0.0.0-20200211180108-c7c1fbc02894 // indirect
 	github.com/chzyer/logex v1.1.10 // indirect
 	github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1 // indirect
@@ -35,7 +36,7 @@ require (
 	github.com/emersion/go-imap-quota v0.0.0-20200423100218-dcfd1b7d2b41
 	github.com/emersion/go-imap-specialuse v0.0.0-20200722111535-598ff00e4075
 	github.com/emersion/go-imap-unselect v0.0.0-20171113212723-b985794e5f26
-	github.com/emersion/go-mbox v1.0.0
+	github.com/emersion/go-mbox v1.0.2
 	github.com/emersion/go-message v0.12.1-0.20200903165315-e1abe21f389a
 	github.com/emersion/go-sasl v0.0.0-20200509203442-7bfe0ed36a21
 	github.com/emersion/go-textwrapper v0.0.0-20160606182133-d0e65e56babe
@@ -74,8 +75,7 @@ require (
 
 replace (
 	github.com/docker/docker-credential-helpers => github.com/ProtonMail/docker-credential-helpers v1.1.0
-	github.com/emersion/go-imap => github.com/ProtonMail/go-imap v0.0.0-20200828124548-d04b0dc1f399
+	github.com/emersion/go-imap => github.com/ProtonMail/go-imap v0.0.0-20201016095853-a7520cc904d3
-	github.com/emersion/go-mbox => github.com/ProtonMail/mbox v0.0.0-20200918064939-909a18c9af45
 	github.com/emersion/go-smtp => github.com/ProtonMail/go-smtp v0.0.0-20181206232543-8261df20d309
 	github.com/jameskeane/bcrypt => github.com/ProtonMail/bcrypt v0.0.0-20170924085257-7509ea014998
 	golang.org/x/crypto => github.com/ProtonMail/crypto v0.0.0-20200818122824-ed5d25e28db8

go.sum

@@ -15,8 +15,8 @@ github.com/ProtonMail/go-apple-mobileconfig v0.0.0-20160701194735-7ea9927a11f6 h
 github.com/ProtonMail/go-apple-mobileconfig v0.0.0-20160701194735-7ea9927a11f6/go.mod h1:EtDfBMIDWmVe4viZCuBTEfe3OIIo0ghbpOaAZVO+hVg=
 github.com/ProtonMail/go-autostart v0.0.0-20181114175602-c5272053443a h1:fXK2KsfnkBV9Nh+9SKzHchYjuE9s0vI20JG1mbtEAcc=
 github.com/ProtonMail/go-autostart v0.0.0-20181114175602-c5272053443a/go.mod h1:oTGdE7/DlWIr23G0IKW3OXK9wZ5Hw1GGiaJFccTvZi4=
-github.com/ProtonMail/go-imap v0.0.0-20200828124548-d04b0dc1f399 h1:wBo/Xgb/Dn2loU47D+PJaOoIZ67i3AqYp51gLn8YE5U=
+github.com/ProtonMail/go-imap v0.0.0-20201016095853-a7520cc904d3 h1:Jvv9t3rSg/ID3Fh+uYsxgmvNI9fYnlab4vtBsbPtmq8=
-github.com/ProtonMail/go-imap v0.0.0-20200828124548-d04b0dc1f399/go.mod h1:yKASt+C3ZiDAiCSssxg9caIckWF/JG7ZQTO7GAmvicU=
+github.com/ProtonMail/go-imap v0.0.0-20201016095853-a7520cc904d3/go.mod h1:yKASt+C3ZiDAiCSssxg9caIckWF/JG7ZQTO7GAmvicU=
 github.com/ProtonMail/go-imap-id v0.0.0-20190926060100-f94a56b9ecde h1:5koQozTDELymYOyFbQ/VSubexAEXzDR8qGM5mO8GRdw=
 github.com/ProtonMail/go-imap-id v0.0.0-20190926060100-f94a56b9ecde/go.mod h1:795VPXcRUIQ9JyMNHP4el582VokQfippgjkQP3Gk0r0=
 github.com/ProtonMail/go-mime v0.0.0-20190923161245-9b5a4261663a h1:W6RrgN/sTxg1msqzFFb+G80MFmpjMw61IU+slm+wln4=
@@ -27,8 +27,6 @@ github.com/ProtonMail/go-vcard v0.0.0-20180326232728-33aaa0a0c8a5 h1:Uga1DHFN4GU
 github.com/ProtonMail/go-vcard v0.0.0-20180326232728-33aaa0a0c8a5/go.mod h1:oeP9CMN+ajWp5jKp1kue5daJNwMMxLF+ujPaUIoJWlA=
 github.com/ProtonMail/gopenpgp/v2 v2.0.1 h1:x0uvDhry5WzoHeJO4J3dgMLhG4Z9PeBJ2O+sDOY0LcU=
 github.com/ProtonMail/gopenpgp/v2 v2.0.1/go.mod h1:wQQCJo7DURO6S9VwH+kSDEYs/B63yZnAEfGlOg8YNBY=
-github.com/ProtonMail/mbox v0.0.0-20200918064939-909a18c9af45 h1:GDh55hDI2sNiirDqEWV8b6EB729u78Qxu3nKF970n6g=
-github.com/ProtonMail/mbox v0.0.0-20200918064939-909a18c9af45/go.mod h1:Yp9IVuuOYLEuMv4yjgDHvhb5mHOcYH6x92Oas3QqEZI=
 github.com/PuerkitoBio/goquery v1.5.1 h1:PSPBGne8NIUWw+/7vFBV+kG2J/5MOjbzc7154OaKCSE=
 github.com/PuerkitoBio/goquery v1.5.1/go.mod h1:GsLWisAFVj4WgDibEWF4pvYnkVQBpKBKeU+7zCJoLcc=
 github.com/abiosoft/ishell v2.0.0+incompatible h1:zpwIuEHc37EzrsIYah3cpevrIc8Oma7oZPxr03tlmmw=
@@ -39,6 +37,8 @@ github.com/allan-simon/go-singleinstance v0.0.0-20160830203053-79edcfdc2dfc h1:m
 github.com/allan-simon/go-singleinstance v0.0.0-20160830203053-79edcfdc2dfc/go.mod h1:qqsTQiwdyqxU05iDCsi0oN3P4nrVxAmn8xCtODDSf/U=
 github.com/andybalholm/cascadia v1.1.0 h1:BuuO6sSfQNFRu1LppgbD25Hr2vLYW25JvxHs5zzsLTo=
 github.com/andybalholm/cascadia v1.1.0/go.mod h1:GsXiBklL0woXo1j/WYWtSYYC4ouU9PqHO0sqidkEA4Y=
+github.com/antlr/antlr4 v0.0.0-20201020194047-0a7eaede42b0 h1:7RW94Pqb4Twsfpz42ALQ+sD0cUUpN8HF4uzKyQf2D8Y=
+github.com/antlr/antlr4 v0.0.0-20201020194047-0a7eaede42b0/go.mod h1:T7PbCXFs94rrTttyxjbyT5+/1V8T2TYDejxUfHJjw1Y=
 github.com/certifi/gocertifi v0.0.0-20200211180108-c7c1fbc02894 h1:JLaf/iINcLyjwbtTsCJjc6rtlASgHeIJPrB6QmwURnA=
 github.com/certifi/gocertifi v0.0.0-20200211180108-c7c1fbc02894/go.mod h1:sGbDF6GwGcLpkNXPUTkMRoywsNa/ol15pxFe6ERfguA=
 github.com/chzyer/logex v1.1.10 h1:Swpa1K6QvQznwJRcfTfQJmTE72DqScAa40E+fbHEXEE=
@@ -67,6 +67,8 @@ github.com/emersion/go-imap-specialuse v0.0.0-20200722111535-598ff00e4075 h1:z8T
 github.com/emersion/go-imap-specialuse v0.0.0-20200722111535-598ff00e4075/go.mod h1:/nybxhI8kXom8Tw6BrHMl42usALvka6meORflnnYwe4=
 github.com/emersion/go-imap-unselect v0.0.0-20171113212723-b985794e5f26 h1:FiSb8+XBQQSkcX3ubr+1tAtlRJBYaFmRZqOAweZ9Wy8=
 github.com/emersion/go-imap-unselect v0.0.0-20171113212723-b985794e5f26/go.mod h1:+gnnZx3Mg3MnCzZrv0eZdp5puxXQUgGT/6N6L7ShKfM=
+github.com/emersion/go-mbox v1.0.2 h1:tE/rT+lEugK9y0myEymCCHnwlZN04hlXPrbKkxRBA5I=
+github.com/emersion/go-mbox v1.0.2/go.mod h1:Yp9IVuuOYLEuMv4yjgDHvhb5mHOcYH6x92Oas3QqEZI=
 github.com/emersion/go-message v0.11.1/go.mod h1:C4jnca5HOTo4bGN9YdqNQM9sITuT3Y0K6bSUw9RklvY=
 github.com/emersion/go-message v0.12.1-0.20200903165315-e1abe21f389a h1:3C6qIGgPr1qAT0ikRD5NbyKpME/iHCDeXhpv/JJsFsE=
 github.com/emersion/go-message v0.12.1-0.20200903165315-e1abe21f389a/go.mod h1:kYIioST9GDHte9/BRWgi93rpqbDuFftMjKSMaXS8ABo=

internal/bridge/credits.go

@@ -15,8 +15,8 @@
 // You should have received a copy of the GNU General Public License
 // along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
 
-// Code generated by ./credits.sh at Wed Sep 16 16:48:58 CEST 2020. DO NOT EDIT.
+// Code generated by ./credits.sh at Wed Nov 4 12:24:36 PM CET 2020. DO NOT EDIT.
 
 package bridge
 
-const Credits = "github.com/0xAX/notificator;github.com/ProtonMail/bcrypt;github.com/ProtonMail/crypto;github.com/ProtonMail/docker-credential-helpers;github.com/ProtonMail/go-appdir;github.com/ProtonMail/go-apple-mobileconfig;github.com/ProtonMail/go-autostart;github.com/ProtonMail/go-imap;github.com/ProtonMail/go-imap-id;github.com/ProtonMail/go-smtp;github.com/ProtonMail/go-vcard;github.com/ProtonMail/gopenpgp/v2;github.com/PuerkitoBio/goquery;github.com/abiosoft/ishell;github.com/abiosoft/readline;github.com/allan-simon/go-singleinstance;github.com/certifi/gocertifi;github.com/chzyer/logex;github.com/chzyer/test;github.com/cucumber/godog;github.com/docker/docker-credential-helpers;github.com/emersion/go-imap;github.com/emersion/go-imap-appendlimit;github.com/emersion/go-imap-idle;github.com/emersion/go-imap-move;github.com/emersion/go-imap-quota;github.com/emersion/go-imap-specialuse;github.com/emersion/go-imap-unselect;github.com/emersion/go-mbox;github.com/emersion/go-message;github.com/emersion/go-sasl;github.com/emersion/go-smtp;github.com/emersion/go-textwrapper;github.com/emersion/go-vcard;github.com/fatih/color;github.com/flynn-archive/go-shlex;github.com/getsentry/raven-go;github.com/go-resty/resty/v2;github.com/golang/mock;github.com/google/go-cmp;github.com/google/uuid;github.com/gopherjs/gopherjs;github.com/hashicorp/go-multierror;github.com/jameskeane/bcrypt;github.com/jaytaylor/html2text;github.com/kardianos/osext;github.com/keybase/go-keychain;github.com/logrusorgru/aurora;github.com/mattn/go-runewidth;github.com/miekg/dns;github.com/myesui/uuid;github.com/nsf/jsondiff;github.com/olekukonko/tablewriter;github.com/pkg/errors;github.com/sirupsen/logrus;github.com/skratchdot/open-golang;github.com/ssor/bom;github.com/stretchr/testify;github.com/therecipe/qt;github.com/twinj/uuid;github.com/urfave/cli;go.etcd.io/bbolt;golang.org/x/crypto;golang.org/x/net;golang.org/x/text;gopkg.in/stretchr/testify.v1;;Font Awesome 4.7.0;;Qt 5.13 by Qt group;"
+const Credits = "github.com/0xAX/notificator;github.com/abiosoft/ishell;github.com/abiosoft/readline;github.com/allan-simon/go-singleinstance;github.com/antlr/antlr4;github.com/certifi/gocertifi;github.com/chzyer/logex;github.com/chzyer/test;github.com/cucumber/godog;github.com/docker/docker-credential-helpers;github.com/emersion/go-imap;github.com/emersion/go-imap-appendlimit;github.com/emersion/go-imap-idle;github.com/emersion/go-imap-move;github.com/emersion/go-imap-quota;github.com/emersion/go-imap-specialuse;github.com/emersion/go-imap-unselect;github.com/emersion/go-mbox;github.com/emersion/go-message;github.com/emersion/go-sasl;github.com/emersion/go-smtp;github.com/emersion/go-textwrapper;github.com/emersion/go-vcard;github.com/fatih/color;github.com/flynn-archive/go-shlex;github.com/getsentry/raven-go;github.com/golang/mock;github.com/google/go-cmp;github.com/google/uuid;github.com/gopherjs/gopherjs;github.com/go-resty/resty/v2;github.com/hashicorp/go-multierror;github.com/jameskeane/bcrypt;github.com/jaytaylor/html2text;github.com/kardianos/osext;github.com/keybase/go-keychain;github.com/logrusorgru/aurora;github.com/Masterminds/semver/v3;github.com/mattn/go-runewidth;github.com/miekg/dns;github.com/myesui/uuid;github.com/nsf/jsondiff;github.com/olekukonko/tablewriter;github.com/pkg/errors;github.com/ProtonMail/bcrypt;github.com/ProtonMail/crypto;github.com/ProtonMail/docker-credential-helpers;github.com/ProtonMail/go-appdir;github.com/ProtonMail/go-apple-mobileconfig;github.com/ProtonMail/go-autostart;github.com/ProtonMail/go-imap;github.com/ProtonMail/go-imap-id;github.com/ProtonMail/gopenpgp/v2;github.com/ProtonMail/go-smtp;github.com/ProtonMail/go-vcard;github.com/PuerkitoBio/goquery;github.com/sirupsen/logrus;github.com/skratchdot/open-golang;github.com/ssor/bom;github.com/stretchr/testify;github.com/therecipe/qt;github.com/twinj/uuid;github.com/urfave/cli;go.etcd.io/bbolt;golang.org/x/crypto;golang.org/x/net;golang.org/x/text;gopkg.in/stretchr/testify.v1;;Font Awesome 4.7.0;;Qt 5.13 by Qt group;"

internal/bridge/release_notes.go

@@ -15,21 +15,17 @@
 // You should have received a copy of the GNU General Public License
 // along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
 
-// Code generated by ./release-notes.sh at 'Mon Sep 21 01:29:10 PM CEST 2020'. DO NOT EDIT.
+// Code generated by ./release-notes.sh at 'Wed Nov  4 12:24:35 PM CET 2020'. DO NOT EDIT.
 
 package bridge
 
-const ReleaseNotes = `• Bulletproofing against any potential data loss and/or duplication
+const ReleaseNotes = `• Ensured better message flow by refactoring both address and date parsing
-• Performance improvements for handling attachments and non-standard formatting
+• Improved secure connectivity checks
-• Better stability of the message parser
+• Better deb packaging
-• Additional foreign encoding support for outgoing messages
+• More robust error handling
-• Complete refactor of the way messages are parsed to simplify code maintenance
-• Improved User-Agent detection
-• Added MacOS Big Sur compatibility
-• Added persistent anonymous API cookies
 `
 
-const ReleaseFixedBugs = `• Fixed rare mail loss when moving from Spam folder
+const ReleaseFixedBugs = `• Ensured that conversations are properly threaded
-• Limited log size
+• Fixed Linux font issues (Fedora)
-• Fixed Linux font issues (mouse hover).
+• Better handling of Mime encrypted messages
 `

internal/frontend/qml/BridgeUI/SettingsView.qml

@@ -48,6 +48,7 @@ Item {
                     text : qsTr("Clear", "clickable link next to clear cache button in settings")
                     color: Style.main.text
                     font {
+                        family : cacheClear.font.family // use default font, not font-awesome
                         pointSize : Style.settings.fontSize * Style.pt
                         underline : true
                     }
@@ -66,6 +67,7 @@ Item {
                     text : qsTr("Clear", "clickable link next to clear keychain button in settings")
                     color: Style.main.text
                     font {
+                        family : cacheKeychain.font.family // use default font, not font-awesome
                         pointSize : Style.settings.fontSize * Style.pt
                         underline : true
                     }
@@ -125,6 +127,7 @@ Item {
                     text : qsTr("Change", "clickable link next to change ports button in settings")
                     color: Style.main.text
                     font {
+                        family : changePort.font.family // use default font, not font-awesome
                         pointSize : Style.settings.fontSize * Style.pt
                         underline : true
                     }

internal/frontend/qml/GuiIE.qml

@@ -276,6 +276,10 @@ Item {
                 winMain.dialogExport.hide()
             }
         }
+
+        onUpdateFinished : {
+            winMain.dialogUpdate.finished(hasError)
+        }
     }
 
     function folderIcon(folderName, folderType) { // translations

internal/frontend/qml/ImportExportUI/DialogExport.qml

@@ -217,7 +217,10 @@ Dialog {
                 Text {
                     anchors.centerIn: parent
                     text: {
-                        if (progressbarExport.isFinished) return qsTr("Export finished","todo")
+                        if (progressbarExport.isFinished) {
+                            if (go.progressDescription=="") return qsTr("Export finished","todo")
+                            else return qsTr("Export failed: %1").arg(go.progressDescription)
+                        }
                         if (
                             go.progressDescription == gui.enums.progressInit || 
                             (go.progress==0 && go.description=="")
@@ -450,7 +453,6 @@ Dialog {
             errorPopup.hide()
         }
         onClickedNo  : {
-            go.resumeProcess()
             errorPopup.hide()
         }
     }

internal/frontend/qml/ImportExportUI/DialogImport.qml

@@ -279,9 +279,8 @@ Dialog {
             titleTo        : root.address
         }
 
-        Rectangle {
+        Column {
             id: masterImportSettings
-            height: 150 // fixme
             anchors {
                 right  : parent.right
                 left   : parent.left
@@ -291,45 +290,47 @@ Dialog {
                 rightMargin  : Style.main.leftMargin
                 bottomMargin : Style.main.bottomMargin
             }
-            color: Style.dialog.background
 
-            Text {
+            spacing: Style.main.bottomMargin
-                id: labelMasterImportSettings
-                text: qsTr("Master import settings:")
 
-                font {
+            Row {
-                    bold: true
+                spacing: masterImportSettings.width - labelMasterImportSettings.width - resetSourceButton.width
-                    family: Style.fontawesome.name
-                    pointSize: Style.main.fontSize * Style.pt
-                }
-                color: Style.main.text
 
-                InfoToolTip {
+                Text {
-                    info: qsTr(
+                    id: labelMasterImportSettings
-                        "If master import date range is selected only emails within this range will be imported, unless it is specified differently in folder date range.",
+                    text: qsTr("Master import settings:")
-                        "Text in master import settings tooltip."
+
-                    )
+                    font {
-                    anchors {
+                        bold: true
-                        left: parent.right
+                        family: Style.fontawesome.name
-                        bottom: parent.bottom
+                        pointSize: Style.main.fontSize * Style.pt
-                        leftMargin : Style.dialog.leftMargin
+                    }
+                    color: Style.main.text
+
+                    InfoToolTip {
+                        anchors {
+                            left: parent.right
+                            bottom: parent.bottom
+                            leftMargin : Style.dialog.leftMargin
+                        }
+                        info: qsTr(
+                            "If master import date range is selected only emails within this range will be imported, unless it is specified differently in folder date range.",
+                            "Text in master import settings tooltip."
+                        )
                     }
                 }
-            }
 
-            // Reset all to default
+                // Reset all to default
-            ClickIconText {
+                ClickIconText {
-                anchors {
+                    id: resetSourceButton
-                    right: parent.right
+                    text:qsTr("Reset all settings to default")
-                    bottom: labelMasterImportSettings.bottom
+                    iconText: Style.fa.refresh
-                }
+                    textColor: Style.main.textBlue
-                text:qsTr("Reset all settings to default")
+                    onClicked: {
-                iconText: Style.fa.refresh
+                        go.resetSource()
-                textColor: Style.main.textBlue
+                        root.decrementCurrentIndex()
-                onClicked: {
+                        timer.start()
-                    go.resetSource()
+                    }
-                    root.decrementCurrentIndex()
-                    timer.start()
                 }
             }
 
@@ -348,49 +349,40 @@ Dialog {
 
             InlineDateRange {
                 id: globalDateRange
-                anchors {
-                    left      : parent.left
-                    top       : line.bottom
-                    topMargin : Style.dialog.topMargin
-                }
             }
 
             // Add global label (inline)
             InlineLabelSelect {
                 id: globalLabels
-                anchors {
+            }
-                    left      : parent.left
+        }
-                    top       : globalDateRange.bottom
+
-                    topMargin : Style.dialog.topMargin
+        // Buttons
-                }
+        Row {
-                //labelWidth : globalDateRange.labelWidth
+            spacing: Style.dialog.spacing
+            anchors {
+                right: parent.right
+                bottom: parent.bottom
+                rightMargin: Style.main.leftMargin
+                bottomMargin: Style.main.bottomMargin
             }
 
-            // Buttons
+            ButtonRounded {
-            Row {
+                id: buttonCancelThree
-                spacing: Style.dialog.spacing
+                fa_icon    : Style.fa.times
-                anchors{
+                text       : qsTr("Cancel", "todo")
-                    bottom       : parent.bottom
+                color_main : Style.dialog.textBlue
-                    right        : parent.right
+                onClicked  : root.cancel()
-                }
+            }
 
-                ButtonRounded {
+            ButtonRounded {
-                    id: buttonCancelThree
+                id: buttonNextThree
-                    fa_icon    : Style.fa.times
+                fa_icon     : Style.fa.check
-                    text       : qsTr("Cancel", "todo")
+                text        : qsTr("Import", "todo")
-                    color_main : Style.dialog.textBlue
+                color_main  : Style.dialog.background
-                    onClicked  : root.cancel()
+                color_minor : Style.dialog.textBlue
-                }
+                isOpaque    : true
-
+                onClicked   : root.okay()
-                ButtonRounded {
-                    id: buttonNextThree
-                    fa_icon     : Style.fa.check
-                    text        : qsTr("Import", "todo")
-                    color_main  : Style.dialog.background
-                    color_minor : Style.dialog.textBlue
-                    isOpaque    : true
-                    onClicked   : root.okay()
-                }
             }
         }
     }
@@ -483,18 +475,30 @@ Dialog {
                 }
             }
 
-            Text {
+            Row {
                 property int fails: go.progressFails
                 visible: fails > 0
-                color : Style.main.textRed
-                font.family: Style.fontawesome.name
-                font.pointSize: Style.main.fontSize * Style.pt
                 anchors.horizontalCenter: parent.horizontalCenter
-                text: Style.fa.exclamation_circle + " " + (
+
-                    fails == 1 ?
+                Text {
-                    qsTr("%1 message failed to be imported").arg(fails) :
+                    color: Style.main.textRed
-                    qsTr("%1 messages failed to be imported").arg(fails)
+                    font {
-                )
+                        pointSize : Style.dialog.fontSize * Style.pt
+                        family    : Style.fontawesome.name
+                    }
+                    text: Style.fa.exclamation_circle
+                }
+
+                Text {
+                    property int fails: go.progressFails
+                    color: Style.main.textRed
+                    font.pointSize: Style.main.fontSize * Style.pt
+                    text: " " + (
+                        fails == 1 ?
+                        qsTr("%1 message failed to be imported").arg(fails) :
+                        qsTr("%1 messages failed to be imported").arg(fails)
+                    )
+                }
             }
 
             Row { // buttons
@@ -575,12 +579,23 @@ Dialog {
             anchors.centerIn : finalReport
             spacing          : Style.dialog.heightSeparator
 
-            Text {
+            Row {
-                text: go.progressDescription!="" ? qsTr("Import failed: %1").arg(go.progressDescription) : Style.fa.check_circle + " " + qsTr("Import completed successfully")
                 anchors.horizontalCenter: parent.horizontalCenter
-                color: go.progressDescription!="" ? Style.main.textRed : Style.main.textGreen
+
-                font.bold : true
+                Text {
-                font.family: Style.fontawesome.name
+                    font {
+                        pointSize: Style.dialog.fontSize * Style.pt
+                        family: Style.fontawesome.name
+                    }
+                    color: Style.main.textGreen
+                    text: go.progressDescription!="" ? "" : Style.fa.check_circle
+                }
+
+                Text {
+                    text: go.progressDescription!="" ? qsTr("Import failed: %1").arg(go.progressDescription) : " " + qsTr("Import completed successfully")
+                    color: go.progressDescription!="" ? Style.main.textRed : Style.main.textGreen
+                    font.bold : true
+                }
             }
 
             Text {
@@ -773,11 +788,6 @@ Dialog {
             errorPopup.hide()
         }
         onClickedNo  : {
-            if (errorPopup.msgID == "ask_send_report") {
-                errorPopup.hide()
-                return
-            }
-            go.resumeProcess()
             errorPopup.hide()
         }
 

internal/frontend/qml/ImportExportUI/HelpView.qml

@@ -74,7 +74,7 @@ Item {
                 )
                 width: wrapper.width
                 color : Style.transparent
-                Text {
+                AccessibleText {
                     id: aboutText
                     anchors {
                         bottom: parent.bottom
@@ -82,8 +82,8 @@ Item {
                     }
                     color: Style.main.textDisabled
                     horizontalAlignment: Qt.AlignHCenter
-                    font.family : Style.fontawesome.name
+                    font.pointSize :  Style.main.fontSize * Style.pt
-                    text: "ProtonMail Import-Export app Version "+go.getBackendVersion()+"\n"+Style.fa.copyright + " 2020 Proton Technologies AG"
+                    text: "ProtonMail Import-Export app Version "+go.getBackendVersion()+"\n© 2020 Proton Technologies AG"
                 }
             }
 

internal/frontend/qml/ImportExportUI/SelectFolderMenu.qml

@@ -42,34 +42,50 @@ ComboBox {
         root.below = popup.y>0
     }
 
-    contentItem : Text {
+    contentItem : Row {
         id: boxText
-        verticalAlignment: Text.AlignVCenter
-        font {
-            family: Style.fontawesome.name
-            pointSize : Style.dialog.fontSize * Style.pt
-            bold: root.down
-        }
-        elide: Text.ElideRight
-        textFormat: Text.StyledText
 
-        text : root.displayText
+        Text {
-        color: !root.enabled ? Style.main.textDisabled : ( root.down ? Style.main.background : Style.main.text )
+            anchors.verticalCenter: parent.verticalCenter
+            font {
+                pointSize: Style.dialog.fontSize * Style.pt
+                family: Style.fontawesome.name
+            }
+            text: {
+                if (view.currentIndex >= 0) {
+                    if (!root.isFolderType) {
+                        return Style.fa.tags + " "
+                    }
+                    var tgtIcon = view.currentItem.folderIcon
+                    var tgtColor = view.currentItem.folderColor
+                    if (tgtIcon != Style.fa.folder_open) {
+                        return tgtIcon + " "
+                    }
+                    return '<font color="'+tgtColor+'">'+ tgtIcon + "</font> "
+                }
+                return ""
+            }
+            color: !root.enabled ? Style.main.textDisabled : ( root.down ? Style.main.background : Style.main.text )
+        }
+
+        Text {
+            anchors.verticalCenter: parent.verticalCenter
+            font {
+                pointSize : Style.dialog.fontSize * Style.pt
+                bold: root.down
+            }
+            elide: Text.ElideRight
+            textFormat: Text.StyledText
+
+            text : root.displayText
+            color: !root.enabled ? Style.main.textDisabled : ( root.down ? Style.main.background : Style.main.text )
+        }
     }
 
     displayText: {
         if (view.currentIndex >= 0) {
-            if (!root.isFolderType) return Style.fa.tags + " " + qsTr("Add/Remove labels")
+            if (!root.isFolderType) return qsTr("Add/Remove labels")
-
+            return view.currentItem.folderName
-            var tgtName = view.currentItem.folderName
-            var tgtIcon = view.currentItem.folderIcon
-            var tgtColor = view.currentItem.folderColor
-
-            if (tgtIcon != Style.fa.folder_open) {
-                return tgtIcon + " " + tgtName
-            }
-
-            return '<font color="'+tgtColor+'">'+ tgtIcon + "</font> " + tgtName
         }
         if (root.isFolderType) return qsTr("No folder selected")
         return qsTr("No labels selected")

internal/frontend/qml/ImportExportUI/SettingsView.qml

@@ -44,6 +44,7 @@ Item {
                     text : qsTr("Clear")
                     color: Style.main.text
                     font {
+                        family : cacheKeychain.font.family // use default font, not font-awesome
                         pointSize : Style.settings.fontSize * Style.pt
                         underline : true
                     }

internal/frontend/qml/ProtonUI/BugReportWindow.qml

@@ -84,7 +84,7 @@ Window {
             height: content.height - (
                 (clientVersion.visible ?  clientVersion.height + Style.dialog.fontSize : 0) +
                 userAddress.height  + Style.dialog.fontSize +
-                securityNote.contentHeight + Style.dialog.fontSize +
+                securityNoteText.contentHeight + Style.dialog.fontSize +
                 cancelButton.height + Style.dialog.fontSize
             )
             clip: true
@@ -215,7 +215,7 @@ Window {
         }
 
         // Note
-        AccessibleText {
+        Row {
             id: securityNote
             anchors {
                 left: parent.left
@@ -223,14 +223,32 @@ Window {
                 top: userAddress.bottom
                 topMargin: Style.dialog.fontSize
             }
-            wrapMode: Text.Wrap
+
-            color: Style.dialog.text
+            Text {
-            font.pointSize : Style.dialog.fontSize * Style.pt
+                id: securityNoteIcon
-            text:
+                font {
-            "<span style='font-family: " + Style.fontawesome.name + "'>" + Style.fa.exclamation_triangle + "</span> " +
+                    pointSize : Style.dialog.fontSize * Style.pt
-            qsTr("Bug reports are not end-to-end encrypted!", "The first part of warning in bug report form") + " " +
+                    family    : Style.fontawesome.name
-            qsTr("Please do not send any sensitive information.", "The second part of warning in bug report form") + " " +
+                }
-            qsTr("Contact us at security@protonmail.com for critical security issues.", "The third part of warning in bug report form")
+                color: Style.dialog.text
+                text  : Style.fa.exclamation_triangle
+            }
+
+            AccessibleText {
+                id: securityNoteText
+                anchors {
+                    left: securityNoteIcon.right
+                    leftMargin: 5 * Style.pt
+                    right: parent.right
+                }
+                wrapMode: Text.Wrap
+                color: Style.dialog.text
+                font.pointSize : Style.dialog.fontSize * Style.pt
+                text:
+                qsTr("Bug reports are not end-to-end encrypted!", "The first part of warning in bug report form") + " " +
+                qsTr("Please do not send any sensitive information.", "The second part of warning in bug report form") + " " +
+                qsTr("Contact us at security@protonmail.com for critical security issues.", "The third part of warning in bug report form")
+            }
         }
 
         // buttons

internal/frontend/qt-ie/error_list.go

@@ -72,6 +72,9 @@ func (e *ErrorListModel) data(index *core.QModelIndex, role int) *core.QVariant
 	case MailSubject:
 		return qtcommon.NewQVariantString(r.Subject)
 	case MailDate:
+		if r.Time.IsZero() {
+			return qtcommon.NewQVariantString("Unavailable")
+		}
 		return qtcommon.NewQVariantString(r.Time.String())
 	case MailFrom:
 		return qtcommon.NewQVariantString(r.From)

internal/frontend/qt-ie/frontend.go

@@ -349,6 +349,9 @@ func (f *FrontendQt) setProgressManager(progress *transfer.Progress) {
 				}
 			}
 		}
+		// Counts will add lost messages only once the progress is completeled.
+		failed, _, _, _, _ := progress.GetCounts()
+		f.Qml.SetProgressFails(int(failed))
 
 		if err := progress.GetFatalError(); err != nil {
 			f.Qml.SetProgressDescription(err.Error())

internal/imap/bridge.go

@@ -76,5 +76,9 @@ func newBridgeUserWrap(bridgeUser *users.User) *bridgeUserWrap {
 }
 
 func (u *bridgeUserWrap) GetStore() storeUserProvider {
-	return newStoreUserWrap(u.User.GetStore())
+	store := u.User.GetStore()
+	if store == nil {
+		return nil
+	}
+	return newStoreUserWrap(store)
 }

internal/imap/mailbox.go

@@ -80,7 +80,10 @@ func (im *imapMailbox) Info() (*imap.MailboxInfo, error) {
 }
 
 func (im *imapMailbox) getFlags() []string {
-	flags := []string{imap.NoInferiorsAttr} // Subfolders are not yet supported by API.
+	flags := []string{}
+	if !im.storeMailbox.IsFolder() || im.storeMailbox.IsSystem() {
+		flags = append(flags, imap.NoInferiorsAttr) // Subfolders are not supported for System or Label
+	}
 	switch im.storeMailbox.LabelID() {
 	case pmapi.SentLabel:
 		flags = append(flags, specialuse.Sent)

internal/imap/mailbox_message.go

@@ -24,7 +24,6 @@ import (
 	"mime/multipart"
 	"net/mail"
 	"net/textproto"
-	"regexp"
 	"sort"
 	"strings"
 	"time"
@@ -141,18 +140,19 @@ func (im *imapMailbox) CreateMessage(flags []string, date time.Time, body imap.L
 	references := m.Header.Get("References")
 	referenceList := strings.Fields(references)
 
-	if len(referenceList) > 0 {
+	// In case there is a mail client which corrupts headers, try
+	// "References" too.
+	if internalID == "" && len(referenceList) > 0 {
 		lastReference := referenceList[len(referenceList)-1]
-		// In case we are using a mail client which corrupts headers, try "References" too.
+		match := pmapi.RxInternalReferenceFormat.FindStringSubmatch(lastReference)
-		re := regexp.MustCompile(pmapi.InternalReferenceFormat)
+		if len(match) == 2 {
-		match := re.FindStringSubmatch(lastReference)
+			internalID = match[1]
-		if len(match) > 0 {
-			internalID = match[0]
 		}
 	}
 
-	// Avoid appending a message which is already on the server. Apply the new
+	// Avoid appending a message which is already on the server. Apply the
-	// label instead. This sometimes happens which Outlook (it uses APPEND instead of COPY).
+	// new label instead. This always happens with Outlook (it uses APPEND
+	// instead of COPY).
 	if internalID != "" {
 		// Check to see if this belongs to a different address in split mode or another ProtonMail account.
 		msg, err := im.storeMailbox.GetMessage(internalID)

internal/imap/mailbox_messages.go

@@ -57,6 +57,10 @@ func (im *imapMailbox) UpdateMessagesFlags(uid bool, seqSet *imap.SeqSet, operat
 	return im.addOrRemoveFlags(operation, messageIDs, flags)
 }
 
+// setFlags is used for FLAGS command (not +FLAGS or -FLAGS), which means
+// to set flags passed as an argument and unset the rest. For example,
+// if message is not read, is flagged and is not deleted, call FLAGS \Seen
+// should flag message as read, unflagged and keep undeleted.
 func (im *imapMailbox) setFlags(messageIDs, flags []string) error { //nolint
 	seen := false
 	flagged := false
@@ -106,16 +110,17 @@ func (im *imapMailbox) setFlags(messageIDs, flags []string) error { //nolint
 		}
 	}
 
-	spamMailbox, err := im.storeAddress.GetMailbox("Spam")
+	// Spam should not be taken into action here as Outlook is using FLAGS
-	if err != nil {
+	// without preserving junk flag. Probably it's because junk is not standard
-		return err
+	// in the rfc3501 and thus Outlook expects calling FLAGS \Seen will not
-	}
+	// change the state of junk or other non-standard flags.
+	// Still, its safe to label as spam once any client sends the request.
 	if spam {
-		if err := spamMailbox.LabelMessages(messageIDs); err != nil {
+		spamMailbox, err := im.storeAddress.GetMailbox("Spam")
+		if err != nil {
 			return err
 		}
-	} else {
+		if err := spamMailbox.LabelMessages(messageIDs); err != nil {
-		if err := spamMailbox.UnlabelMessages(messageIDs); err != nil {
 			return err
 		}
 	}

internal/importexport/credits.go

@@ -15,8 +15,8 @@
 // You should have received a copy of the GNU General Public License
 // along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
 
-// Code generated by ./credits.sh at Wed Sep 23 01:34:10 PM CEST 2020. DO NOT EDIT.
+// Code generated by ./credits.sh at Wed Nov 4 12:24:36 PM CET 2020. DO NOT EDIT.
 
 package importexport
 
-const Credits = "github.com/0xAX/notificator;github.com/abiosoft/ishell;github.com/abiosoft/readline;github.com/allan-simon/go-singleinstance;github.com/certifi/gocertifi;github.com/chzyer/logex;github.com/chzyer/test;github.com/cucumber/godog;github.com/docker/docker-credential-helpers;github.com/emersion/go-imap;github.com/emersion/go-imap-appendlimit;github.com/emersion/go-imap-idle;github.com/emersion/go-imap-move;github.com/emersion/go-imap-quota;github.com/emersion/go-imap-specialuse;github.com/emersion/go-imap-unselect;github.com/emersion/go-mbox;github.com/emersion/go-message;github.com/emersion/go-sasl;github.com/emersion/go-smtp;github.com/emersion/go-textwrapper;github.com/emersion/go-vcard;github.com/fatih/color;github.com/flynn-archive/go-shlex;github.com/getsentry/raven-go;github.com/golang/mock;github.com/google/go-cmp;github.com/google/uuid;github.com/gopherjs/gopherjs;github.com/go-resty/resty/v2;github.com/hashicorp/go-multierror;github.com/jameskeane/bcrypt;github.com/jaytaylor/html2text;github.com/kardianos/osext;github.com/keybase/go-keychain;github.com/logrusorgru/aurora;github.com/Masterminds/semver/v3;github.com/mattn/go-runewidth;github.com/miekg/dns;github.com/myesui/uuid;github.com/nsf/jsondiff;github.com/olekukonko/tablewriter;github.com/pkg/errors;github.com/ProtonMail/bcrypt;github.com/ProtonMail/crypto;github.com/ProtonMail/docker-credential-helpers;github.com/ProtonMail/go-appdir;github.com/ProtonMail/go-apple-mobileconfig;github.com/ProtonMail/go-autostart;github.com/ProtonMail/go-imap;github.com/ProtonMail/go-imap-id;github.com/ProtonMail/gopenpgp/v2;github.com/ProtonMail/go-smtp;github.com/ProtonMail/go-vcard;github.com/ProtonMail/mbox;github.com/PuerkitoBio/goquery;github.com/sirupsen/logrus;github.com/skratchdot/open-golang;github.com/ssor/bom;github.com/stretchr/testify;github.com/therecipe/qt;github.com/twinj/uuid;github.com/urfave/cli;go.etcd.io/bbolt;golang.org/x/crypto;golang.org/x/net;golang.org/x/text;gopkg.in/stretchr/testify.v1;;Font Awesome 4.7.0;;Qt 5.13 by Qt group;"
+const Credits = "github.com/0xAX/notificator;github.com/abiosoft/ishell;github.com/abiosoft/readline;github.com/allan-simon/go-singleinstance;github.com/antlr/antlr4;github.com/certifi/gocertifi;github.com/chzyer/logex;github.com/chzyer/test;github.com/cucumber/godog;github.com/docker/docker-credential-helpers;github.com/emersion/go-imap;github.com/emersion/go-imap-appendlimit;github.com/emersion/go-imap-idle;github.com/emersion/go-imap-move;github.com/emersion/go-imap-quota;github.com/emersion/go-imap-specialuse;github.com/emersion/go-imap-unselect;github.com/emersion/go-mbox;github.com/emersion/go-message;github.com/emersion/go-sasl;github.com/emersion/go-smtp;github.com/emersion/go-textwrapper;github.com/emersion/go-vcard;github.com/fatih/color;github.com/flynn-archive/go-shlex;github.com/getsentry/raven-go;github.com/golang/mock;github.com/google/go-cmp;github.com/google/uuid;github.com/gopherjs/gopherjs;github.com/go-resty/resty/v2;github.com/hashicorp/go-multierror;github.com/jameskeane/bcrypt;github.com/jaytaylor/html2text;github.com/kardianos/osext;github.com/keybase/go-keychain;github.com/logrusorgru/aurora;github.com/Masterminds/semver/v3;github.com/mattn/go-runewidth;github.com/miekg/dns;github.com/myesui/uuid;github.com/nsf/jsondiff;github.com/olekukonko/tablewriter;github.com/pkg/errors;github.com/ProtonMail/bcrypt;github.com/ProtonMail/crypto;github.com/ProtonMail/docker-credential-helpers;github.com/ProtonMail/go-appdir;github.com/ProtonMail/go-apple-mobileconfig;github.com/ProtonMail/go-autostart;github.com/ProtonMail/go-imap;github.com/ProtonMail/go-imap-id;github.com/ProtonMail/gopenpgp/v2;github.com/ProtonMail/go-smtp;github.com/ProtonMail/go-vcard;github.com/PuerkitoBio/goquery;github.com/sirupsen/logrus;github.com/skratchdot/open-golang;github.com/ssor/bom;github.com/stretchr/testify;github.com/therecipe/qt;github.com/twinj/uuid;github.com/urfave/cli;go.etcd.io/bbolt;golang.org/x/crypto;golang.org/x/net;golang.org/x/text;gopkg.in/stretchr/testify.v1;;Font Awesome 4.7.0;;Qt 5.13 by Qt group;"

internal/importexport/release_notes.go

@@ -15,17 +15,18 @@
 // You should have received a copy of the GNU General Public License
 // along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
 
-// Code generated by ./release-notes.sh at 'Wed Sep 23 01:31:53 PM CEST 2020'. DO NOT EDIT.
+// Code generated by ./release-notes.sh at 'Wed Nov  4 12:24:35 PM CET 2020'. DO NOT EDIT.
 
 package importexport
 
-const ReleaseNotes = `• Speed up import by implementing parallel processing (parallel fetch, encrypt and upload of messages)
+const ReleaseNotes = `• Improvements to the import from large mbox files with multiple labels
-• Optimising the initial fetch of messages from external accounts
+• Not allow to run multiple instances of the app or transfers at the same time
-• Better handling of attachments and non-standard formatting
+• Various enhancements of the import process related to parsing
-• Improved stability of the message parser
+• Cosmetic GUI changes
-• Added persistent anonymous API cookies
+• Better error handling
 `
 
-const ReleaseFixedBugs = `• Import from mbox files with long lines
+const ReleaseFixedBugs = `• Linux font issues - Fedora specific
-• Improvements to import from Yahoo accounts
+• App response to the user pausing and canceling import or export
+• Handling errors during update
 `

internal/smtp/preferences.go

@@ -503,13 +503,6 @@ func (b *sendPreferencesBuilder) setEncryptionPreferences(mailSettings pmapi.Mai
 }
 
 func (b *sendPreferencesBuilder) setMIMEPreferences(composerMIMEType string) {
-	// If the sign flag (that we just determined above) is true we use the scheme
-	// in the encryption preferences, unless the plain text format has been
-	// selected in the composer, in which case we must enforce PGP/INLINE.
-	if !b.isInternal() && b.shouldSign() && composerMIMEType == "text/plain" {
-		b.withScheme(pgpInline)
-	}
-
 	// If the sign flag (that we just determined above) is true, then the MIME
 	// type is determined by the PGP scheme (also determined above): we should
 	// use 'text/plain' for a PGP/Inline scheme, and 'multipart/mixed' otherwise.

internal/smtp/preferences_test.go

@@ -254,6 +254,20 @@ func TestPreferencesBuilder(t *testing.T) {
 			wantMIMEType: "multipart/mixed",
 		},
 
+		{
+			name: "external with sign enabled, sending plaintext, should still send as ClearMIME",
+
+			contactMeta:  &ContactMetadata{Sign: true, SignIsSet: true},
+			receivedKeys: []pmapi.PublicKey{},
+			isInternal:   false,
+			mailSettings: pmapi.MailSettings{PGPScheme: pmapi.PGPMIMEPackage, DraftMIMEType: "text/plain"},
+
+			wantEncrypt:  false,
+			wantSign:     true,
+			wantScheme:   pmapi.ClearMIMEPackage,
+			wantMIMEType: "multipart/mixed",
+		},
+
 		{
 			name: "external with pinned contact public key but no intention to encrypt/sign",
 

internal/smtp/send_recorder.go

@@ -20,6 +20,7 @@ package smtp
 import (
 	"crypto/sha256"
 	"fmt"
+	"strings"
 	"sync"
 	"time"
 
@@ -48,6 +49,15 @@ func newSendRecorder() *sendRecorder {
 }
 
 func (q *sendRecorder) getMessageHash(message *pmapi.Message) string {
+	// Outlook Calendar updates has only headers (no body) and thus have always
+	// the same hash. If the message is type of calendar, the "is sending"
+	// check to avoid potential duplicates is skipped. Duplicates should not
+	// be a problem in this case as calendar updates are small.
+	contentType := message.Header.Get("Content-Type")
+	if strings.HasPrefix(contentType, "text/calendar") {
+		return ""
+	}
+
 	h := sha256.New()
 	_, _ = h.Write([]byte(message.AddressID + message.Subject))
 	if message.Sender != nil {
@@ -101,6 +111,10 @@ func (q *sendRecorder) isSendingOrSent(client messageGetter, hash string) (isSen
 	q.lock.Lock()
 	defer q.lock.Unlock()
 
+	if hash == "" {
+		return false, false
+	}
+
 	q.deleteExpiredKeys()
 	value, ok := q.hashes[hash]
 	if !ok {

internal/smtp/send_recorder_test.go

@@ -349,6 +349,32 @@ func TestSendRecorder_getMessageHash(t *testing.T) {
 			},
 			false,
 		},
+		{ // Different content type - calendar
+			&pmapi.Message{
+				Header: mail.Header{
+					"Content-Type": []string{"text/calendar"},
+				},
+				AddressID: "address123",
+				Subject:   "Subject #1",
+				Sender: &mail.Address{
+					Address: "from@pm.me",
+				},
+				ToList: []*mail.Address{
+					{Address: "to@pm.me"},
+				},
+				CCList:  []*mail.Address{},
+				BCCList: []*mail.Address{},
+				Body:    "body",
+				Attachments: []*pmapi.Attachment{
+					{
+						Name:     "att1",
+						MIMEType: "image/png",
+						Size:     12345,
+					},
+				},
+			},
+			false,
+		},
 	}
 	for i, tc := range testCases {
 		tc := tc // bind
@@ -382,12 +408,13 @@ func TestSendRecorder_isSendingOrSent(t *testing.T) {
 		{"hash", &pmapi.Message{Type: pmapi.MessageTypeDraft, Time: time.Now().Unix()}, nil, true, false},
 		{"hash", &pmapi.Message{Type: pmapi.MessageTypeSent}, nil, false, true},
 		{"hash", &pmapi.Message{Type: pmapi.MessageTypeInboxAndSent}, nil, false, true},
+		{"", &pmapi.Message{Type: pmapi.MessageTypeInboxAndSent}, nil, false, false},
 	}
 	for i, tc := range testCases {
 		tc := tc // bind
 		t.Run(fmt.Sprintf("%d / %v / %v / %v", i, tc.hash, tc.message, tc.err), func(t *testing.T) {
 			messageGetter := &testSendRecorderGetMessageMock{message: tc.message, err: tc.err}
-			isSending, wasSent := q.isSendingOrSent(messageGetter, "hash")
+			isSending, wasSent := q.isSendingOrSent(messageGetter, tc.hash)
 			assert.Equal(t, tc.wantIsSending, isSending, "isSending does not match")
 			assert.Equal(t, tc.wantWasSent, wasSent, "wasSent does not match")
 		})

internal/smtp/user.go

@@ -25,7 +25,6 @@ import (
 	"io"
 	"mime"
 	"net/mail"
-	"regexp"
 	"strings"
 	"time"
 
@@ -408,9 +407,9 @@ func (su *smtpUser) handleReferencesHeader(m *pmapi.Message) (draftID, parentID
 		if !strings.Contains(reference, "@"+pmapi.InternalIDDomain) {
 			newReferences = append(newReferences, reference)
 		} else { // internalid is the parentID.
-			idMatch := regexp.MustCompile(pmapi.InternalReferenceFormat).FindStringSubmatch(reference)
+			idMatch := pmapi.RxInternalReferenceFormat.FindStringSubmatch(reference)
-			if len(idMatch) > 0 {
+			if len(idMatch) == 2 {
-				lastID := strings.TrimSuffix(strings.Trim(idMatch[0], "<>"), "@protonmail.internalid")
+				lastID := idMatch[1]
 				filter := &pmapi.MessagesFilter{ID: []string{lastID}}
 				if su.addressID != "" {
 					filter.AddressID = su.addressID

internal/store/address.go

@@ -69,7 +69,7 @@ func (storeAddress *Address) init(foldersAndLabels []*pmapi.Label) (err error) {
 			prefix := getLabelPrefix(label)
 
 			var mailbox *Mailbox
-			if mailbox, err = txNewMailbox(tx, storeAddress, label.ID, prefix, label.Name, label.Color); err != nil {
+			if mailbox, err = txNewMailbox(tx, storeAddress, label.ID, prefix, label.Path, label.Color); err != nil {
 				storeAddress.log.
 					WithError(err).
 					WithField("labelID", label.ID).

internal/store/address_mailbox.go

@@ -73,14 +73,14 @@ func (storeAddress *Address) createOrUpdateMailboxEvent(label *pmapi.Label) erro
 	prefix := getLabelPrefix(label)
 	mailbox, ok := storeAddress.mailboxes[label.ID]
 	if !ok {
-		mailbox, err := newMailbox(storeAddress, label.ID, prefix, label.Name, label.Color)
+		mailbox, err := newMailbox(storeAddress, label.ID, prefix, label.Path, label.Color)
 		if err != nil {
 			return err
 		}
 		storeAddress.mailboxes[label.ID] = mailbox
 		mailbox.store.imapMailboxCreated(storeAddress.address, mailbox.labelName)
 	} else {
-		mailbox.labelName = prefix + label.Name
+		mailbox.labelName = prefix + label.Path
 		mailbox.color = label.Color
 	}
 	return nil

internal/store/change.go

@@ -37,7 +37,7 @@ func (store *Store) SetIMAPUpdateChannel(updates chan imapBackend.Update) {
 	}
 }
 
-func (store *Store) imapNotice(address, notice string) {
+func (store *Store) imapNotice(address, notice string) *imapBackend.StatusUpdate {
 	update := new(imapBackend.StatusUpdate)
 	update.Update = imapBackend.NewUpdate(address, "")
 	update.StatusResp = &imap.StatusResp{
@@ -46,13 +46,14 @@ func (store *Store) imapNotice(address, notice string) {
 		Info: notice,
 	}
 	store.imapSendUpdate(update)
+	return update
 }
 
 func (store *Store) imapUpdateMessage(
 	address, mailboxName string,
 	uid, sequenceNumber uint32,
 	msg *pmapi.Message, hasDeletedFlag bool,
-) {
+) *imapBackend.MessageUpdate {
 	store.log.WithFields(logrus.Fields{
 		"address": address,
 		"mailbox": mailboxName,
@@ -70,9 +71,10 @@ func (store *Store) imapUpdateMessage(
 	}
 	update.Message.Uid = uid
 	store.imapSendUpdate(update)
+	return update
 }
 
-func (store *Store) imapDeleteMessage(address, mailboxName string, sequenceNumber uint32) {
+func (store *Store) imapDeleteMessage(address, mailboxName string, sequenceNumber uint32) *imapBackend.ExpungeUpdate {
 	store.log.WithFields(logrus.Fields{
 		"address": address,
 		"mailbox": mailboxName,
@@ -82,9 +84,10 @@ func (store *Store) imapDeleteMessage(address, mailboxName string, sequenceNumbe
 	update.Update = imapBackend.NewUpdate(address, mailboxName)
 	update.SeqNum = sequenceNumber
 	store.imapSendUpdate(update)
+	return update
 }
 
-func (store *Store) imapMailboxCreated(address, mailboxName string) {
+func (store *Store) imapMailboxCreated(address, mailboxName string) *imapBackend.MailboxInfoUpdate {
 	store.log.WithFields(logrus.Fields{
 		"address": address,
 		"mailbox": mailboxName,
@@ -97,9 +100,10 @@ func (store *Store) imapMailboxCreated(address, mailboxName string) {
 		Name:       mailboxName,
 	}
 	store.imapSendUpdate(update)
+	return update
 }
 
-func (store *Store) imapMailboxStatus(address, mailboxName string, total, unread, unreadSeqNum uint) {
+func (store *Store) imapMailboxStatus(address, mailboxName string, total, unread, unreadSeqNum uint) *imapBackend.MailboxUpdate {
 	store.log.WithFields(logrus.Fields{
 		"address":      address,
 		"mailbox":      mailboxName,
@@ -114,6 +118,7 @@ func (store *Store) imapMailboxStatus(address, mailboxName string, total, unread
 	update.MailboxStatus.Unseen = uint32(unread)
 	update.MailboxStatus.UnseenSeqNum = uint32(unreadSeqNum)
 	store.imapSendUpdate(update)
+	return update
 }
 
 func (store *Store) imapSendUpdate(update imapBackend.Update) {
@@ -122,22 +127,10 @@ func (store *Store) imapSendUpdate(update imapBackend.Update) {
 		return
 	}
 
-	done := update.Done()
-	go func() {
-		// This timeout is to not keep running many blocked goroutines.
-		// In case nothing listens to this channel, this thread should stop.
-		select {
-		case store.imapUpdates <- update:
-		case <-time.After(1 * time.Second):
-			store.log.Warn("IMAP update could not be sent (timeout).")
-		}
-	}()
-
-	// This timeout is to not block IMAP backend by wait for IMAP client.
 	select {
-	case <-done:
 	case <-time.After(1 * time.Second):
-		store.log.Warn("IMAP update could not be delivered (timeout).")
+		store.log.Warn("IMAP update could not be sent (timeout)")
 		return
+	case store.imapUpdates <- update:
 	}
 }

internal/store/mailbox.go

@@ -142,6 +142,9 @@ func initMailboxBucket(tx *bolt.Tx, bucketName []byte) error {
 	if _, err := bucket.CreateBucketIfNotExists(apiIDsBucket); err != nil {
 		return err
 	}
+	if _, err := bucket.CreateBucketIfNotExists(deletedIDsBucket); err != nil {
+		return err
+	}
 
 	return nil
 }
@@ -240,13 +243,7 @@ func (storeMailbox *Mailbox) txGetAPIIDsBucket(tx *bolt.Tx) *bolt.Bucket {
 
 // txGetDeletedIDsBucket returns the bucket with messagesID marked as deleted
 func (storeMailbox *Mailbox) txGetDeletedIDsBucket(tx *bolt.Tx) *bolt.Bucket {
-	// There should be no error since it _...returns an error if the bucket
+	return storeMailbox.txGetBucket(tx).Bucket(deletedIDsBucket)
-	// name is blank, or if the bucket name is too long._
-	bucket, err := storeMailbox.txGetBucket(tx).CreateBucketIfNotExists(deletedIDsBucket)
-	if err != nil || bucket == nil {
-		storeMailbox.log.WithError(err).Error("Cannot create or get bucket with deleted IDs.")
-	}
-	return bucket
 }
 
 // txGetBucket returns the bucket of mailbox containing mapping buckets.

internal/store/mailbox_counts.go

@@ -125,6 +125,7 @@ func (mc *mailboxCounts) getPMLabel() *pmapi.Label {
 	return &pmapi.Label{
 		ID:        mc.LabelID,
 		Name:      mc.LabelName,
+		Path:      mc.LabelName,
 		Color:     mc.Color,
 		Order:     mc.Order,
 		Type:      pmapi.LabelTypeMailbox,
@@ -158,7 +159,7 @@ func (store *Store) createOrUpdateMailboxCountsBuckets(labels []*pmapi.Label) er
 			}
 
 			// Update mailbox info, but dont change on-API-counts.
-			mailbox.LabelName = label.Name
+			mailbox.LabelName = label.Path
 			mailbox.Color = label.Color
 			mailbox.Order = label.Order
 			mailbox.IsFolder = label.Exclusive == 1

internal/store/mailbox_message.go

@@ -18,6 +18,8 @@
 package store
 
 import (
+	"time"
+
 	"github.com/ProtonMail/proton-bridge/pkg/pmapi"
 	"github.com/pkg/errors"
 	"github.com/sirupsen/logrus"
@@ -501,7 +503,7 @@ func (storeMailbox *Mailbox) txMarkMessagesAsDeleted(tx *bolt.Tx, apiIDs []strin
 
 		// In order to send flags in format
 		// S: * 2 FETCH (FLAGS (\Deleted \Seen))
-		storeMailbox.store.imapUpdateMessage(
+		update := storeMailbox.store.imapUpdateMessage(
 			storeMailbox.storeAddress.address,
 			storeMailbox.labelName,
 			uid,
@@ -509,6 +511,14 @@ func (storeMailbox *Mailbox) txMarkMessagesAsDeleted(tx *bolt.Tx, apiIDs []strin
 			msg,
 			markAsDeleted,
 		)
+
+		// txMarkMessagesAsDeleted is called only during processing request
+		// from IMAP call (i.e., not from event loop) and in such cases we
+		// have to wait to propagate update back before closing the response.
+		select {
+		case <-time.After(1 * time.Second):
+		case <-update.Done():
+		}
 	}
 
 	return nil

internal/store/message.go

@@ -66,7 +66,7 @@ func (message *Message) Message() *pmapi.Message {
 // mailbox
 func (message *Message) IsMarkedDeleted() bool {
 	isMarkedAsDeleted := false
-	err := message.storeMailbox.db().Update(func(tx *bolt.Tx) error {
+	err := message.storeMailbox.db().View(func(tx *bolt.Tx) error {
 		isMarkedAsDeleted = message.storeMailbox.txGetDeletedIDsBucket(tx).Get([]byte(message.msg.ID)) != nil
 		return nil
 	})

internal/transfer/message.go

@@ -29,17 +29,34 @@ type Message struct {
 	ID      string
 	Unread  bool
 	Body    []byte
-	Source  Mailbox
+	Sources []Mailbox
 	Targets []Mailbox
 }
 
+// sourceNames returns array of source mailbox names.
+func (msg Message) sourceNames() (names []string) {
+	for _, mailbox := range msg.Sources {
+		names = append(names, mailbox.Name)
+	}
+	return
+}
+
+// targetNames returns array of target mailbox names.
+func (msg Message) targetNames() (names []string) {
+	for _, mailbox := range msg.Targets {
+		names = append(names, mailbox.Name)
+	}
+	return
+}
+
 // MessageStatus holds status for message used by progress manager.
 type MessageStatus struct {
-	eventTime time.Time // Time of adding message to the process.
+	eventTime   time.Time // Time of adding message to the process.
-	rule      *Rule     // Rule with source and target mailboxes.
+	sourceNames []string  // Source mailbox names message is in.
-	SourceID  string    // Message ID at the source.
+	SourceID    string    // Message ID at the source.
-	targetID  string    // Message ID at the target (if any).
+	targetNames []string  // Target mailbox names message is in.
-	bodyHash  string    // Hash of the message body.
+	targetID    string    // Message ID at the target (if any).
+	bodyHash    string    // Hash of the message body.
 
 	exported  bool
 	imported  bool

internal/transfer/progress.go

@@ -93,7 +93,7 @@ func (p *Progress) fatal(err error) {
 	defer p.lock.Unlock()
 
 	log.WithError(err).Error("Progress finished")
-	p.isStopped = true
+	p.setStop()
 	p.fatalError = err
 	p.cleanUpdateCh()
 }
@@ -126,16 +126,17 @@ func (p *Progress) updateCount(mailbox string, count uint) {
 }
 
 // addMessage should be called as soon as there is ID of the message.
-func (p *Progress) addMessage(messageID string, rule *Rule) {
+func (p *Progress) addMessage(messageID string, sourceNames, targetNames []string) {
 	p.lock.Lock()
 	defer p.lock.Unlock()
 	defer p.update()
 
 	p.log.WithField("id", messageID).Trace("Message added")
 	p.messageStatuses[messageID] = &MessageStatus{
-		eventTime: time.Now(),
+		eventTime:   time.Now(),
-		rule:      rule,
+		sourceNames: sourceNames,
-		SourceID:  messageID,
+		SourceID:    messageID,
+		targetNames: targetNames,
 	}
 }
 
@@ -282,6 +283,15 @@ func (p *Progress) Stop() {
 	defer p.update()
 
 	p.log.Info("Progress stopped")
+	p.setStop()
+
+	// Once progress is stopped, some calls might be in progress. Results from
+	// those calls are irrelevant so we can close update channel sooner to not
+	// propagate any progress to user interface anymore.
+	p.cleanUpdateCh()
+}
+
+func (p *Progress) setStop() {
 	p.isStopped = true
 	p.pauseReason = "" // Clear pause to run paused code and stop it.
 }

internal/transfer/progress_test.go

@@ -19,6 +19,7 @@ package transfer
 
 import (
 	"testing"
+	"time"
 
 	"github.com/pkg/errors"
 	a "github.com/stretchr/testify/assert"
@@ -47,21 +48,21 @@ func TestProgressAddingMessages(t *testing.T) {
 	drainProgressUpdateChannel(&progress)
 
 	// msg1 has no problem.
-	progress.addMessage("msg1", nil)
+	progress.addMessage("msg1", []string{}, []string{})
 	progress.messageExported("msg1", []byte(""), nil)
 	progress.messageImported("msg1", "", nil)
 
 	// msg2 has an import problem.
-	progress.addMessage("msg2", nil)
+	progress.addMessage("msg2", []string{}, []string{})
 	progress.messageExported("msg2", []byte(""), nil)
 	progress.messageImported("msg2", "", errors.New("failed import"))
 
 	// msg3 has an export problem.
-	progress.addMessage("msg3", nil)
+	progress.addMessage("msg3", []string{}, []string{})
 	progress.messageExported("msg3", []byte(""), errors.New("failed export"))
 
 	// msg4 has an export problem and import is also called.
-	progress.addMessage("msg4", nil)
+	progress.addMessage("msg4", []string{}, []string{})
 	progress.messageExported("msg4", []byte(""), errors.New("failed export"))
 	progress.messageImported("msg4", "", nil)
 
@@ -91,7 +92,7 @@ func TestProgressFinish(t *testing.T) {
 	progress.finish()
 	r.Nil(t, progress.updateCh)
 
-	r.NotPanics(t, func() { progress.addMessage("msg", nil) })
+	r.NotPanics(t, func() { progress.addMessage("msg", []string{}, []string{}) })
 }
 
 func TestProgressFatalError(t *testing.T) {
@@ -101,7 +102,29 @@ func TestProgressFatalError(t *testing.T) {
 	progress.fatal(errors.New("fatal error"))
 	r.Nil(t, progress.updateCh)
 
-	r.NotPanics(t, func() { progress.addMessage("msg", nil) })
+	r.NotPanics(t, func() { progress.addMessage("msg", []string{}, []string{}) })
+}
+
+func TestFailUnpauseAndStops(t *testing.T) {
+	progress := newProgress(log, nil)
+	drainProgressUpdateChannel(&progress)
+
+	progress.Pause("pausing")
+	progress.fatal(errors.New("fatal error"))
+
+	r.Nil(t, progress.updateCh)
+	r.True(t, progress.isStopped)
+	r.False(t, progress.IsPaused())
+	r.Eventually(t, progress.shouldStop, time.Second, 10*time.Millisecond)
+}
+
+func TestStopClosesUpdates(t *testing.T) {
+	progress := newProgress(log, nil)
+	ch := progress.updateCh
+
+	progress.Stop()
+	r.Nil(t, progress.updateCh)
+	r.PanicsWithError(t, "send on closed channel", func() { ch <- struct{}{} })
 }
 
 func drainProgressUpdateChannel(progress *Progress) {

internal/transfer/provider_eml_source.go

@@ -109,7 +109,7 @@ func (p *EMLProvider) exportMessages(rule *Rule, filePaths []string, progress *P
 		// addMessage is called after time check to not report message
 		// which should not be exported but any error from reading body
 		// or parsing time is reported as an error.
-		progress.addMessage(filePath, rule)
+		progress.addMessage(filePath, msg.sourceNames(), msg.targetNames())
 		progress.messageExported(filePath, msg.Body, err)
 		if err == nil {
 			ch <- msg
@@ -134,7 +134,7 @@ func (p *EMLProvider) exportMessage(rule *Rule, filePath string) (Message, error
 		ID:      filePath,
 		Unread:  false,
 		Body:    body,
-		Source:  rule.SourceMailbox,
+		Sources: []Mailbox{rule.SourceMailbox},
 		Targets: rule.TargetMailboxes,
 	}, nil
 }

internal/transfer/provider_eml_target.go

@@ -61,7 +61,7 @@ func (p *EMLProvider) TransferFrom(rules transferRules, progress *Progress, ch <
 func (p *EMLProvider) createFolders(rules transferRules) error {
 	for rule := range rules.iterateActiveRules() {
 		for _, mailbox := range rule.TargetMailboxes {
-			path := filepath.Join(p.root, mailbox.Name)
+			path := filepath.Join(p.root, sanitizeFileName(mailbox.Name))
 			if err := os.MkdirAll(path, os.ModePerm); err != nil {
 				return err
 			}
@@ -71,7 +71,7 @@ func (p *EMLProvider) createFolders(rules transferRules) error {
 }
 
 func (p *EMLProvider) writeFile(msg Message) error {
-	fileName := filepath.Base(msg.ID)
+	fileName := sanitizeFileName(filepath.Base(msg.ID))
 	if filepath.Ext(fileName) != ".eml" {
 		fileName += ".eml"
 	}

internal/transfer/provider_imap_source.go

@@ -124,7 +124,7 @@ func (p *IMAPProvider) loadMessagesInfo(rule *Rule, progress *Progress, uidValid
 				uid:  imapMessage.Uid,
 				size: imapMessage.Size,
 			}
-			progress.addMessage(id, rule)
+			progress.addMessage(id, []string{rule.SourceMailbox.Name}, rule.TargetMailboxNames())
 		}
 
 		progress.callWrap(func() error {
@@ -231,7 +231,7 @@ func (p *IMAPProvider) exportMessage(rule *Rule, id string, imapMessage *imap.Me
 		ID:      id,
 		Unread:  unread,
 		Body:    body,
-		Source:  rule.SourceMailbox,
+		Sources: []Mailbox{rule.SourceMailbox},
 		Targets: rule.TargetMailboxes,
 	}
 }

internal/transfer/provider_mbox.go

@@ -18,8 +18,11 @@
 package transfer
 
 import (
+	"os"
 	"path/filepath"
 	"strings"
+
+	"github.com/pkg/errors"
 )
 
 // MBOXProvider implements import and export to/from MBOX structure.
@@ -44,16 +47,35 @@ func (p *MBOXProvider) ID() string {
 // In case the same folder name is used more than once (for example root/a/foo
 // and root/b/foo), it's treated as the same folder.
 func (p *MBOXProvider) Mailboxes(includeEmpty, includeAllMail bool) ([]Mailbox, error) {
-	filePaths, err := getFilePathsWithSuffix(p.root, "mbox")
+	filePaths, err := getAllPathsWithSuffix(p.root, ".mbox")
 	if err != nil {
 		return nil, err
 	}
 
-	mailboxes := []Mailbox{}
+	mailboxNames := map[string]bool{}
 	for _, filePath := range filePaths {
 		fileName := filepath.Base(filePath)
-		mailboxName := strings.TrimSuffix(fileName, ".mbox")
+		filePath, err := p.handleAppleMailMBOXStructure(filePath)
+		if err != nil {
+			log.WithError(err).Warn("Failed to handle MBOX structure")
+			continue
+		}
 
+		mailboxName := strings.TrimSuffix(fileName, ".mbox")
+		mailboxNames[mailboxName] = true
+
+		labels, err := getGmailLabelsFromMboxFile(filepath.Join(p.root, filePath))
+		if err != nil {
+			log.WithError(err).Error("Failed to get gmail labels from mbox file")
+			continue
+		}
+		for label := range labels {
+			mailboxNames[label] = true
+		}
+	}
+
+	mailboxes := []Mailbox{}
+	for mailboxName := range mailboxNames {
 		mailboxes = append(mailboxes, Mailbox{
 			ID:          "",
 			Name:        mailboxName,
@@ -61,6 +83,20 @@ func (p *MBOXProvider) Mailboxes(includeEmpty, includeAllMail bool) ([]Mailbox,
 			IsExclusive: false,
 		})
 	}
-
 	return mailboxes, nil
 }
+
+// handleAppleMailMBOXStructure changes the path of mailbox directory to
+// the path of mbox file. Apple Mail MBOX exports has this structure:
+// `Folder.mbox` directory with `mbox` file inside.
+// Example: `Folder.mbox/mbox` (and this function converts `Folder.mbox`
+// to `Folder.mbox/mbox`).
+func (p *MBOXProvider) handleAppleMailMBOXStructure(filePath string) (string, error) {
+	if info, err := os.Stat(filepath.Join(p.root, filePath)); err == nil && info.IsDir() {
+		if _, err := os.Stat(filepath.Join(p.root, filePath, "mbox")); err != nil {
+			return "", errors.Wrap(err, "wrong mbox structure")
+		}
+		return filepath.Join(filePath, "mbox"), nil
+	}
+	return filePath, nil
+}

internal/transfer/provider_mbox_gmail_labels.go

@@ -0,0 +1,118 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package transfer
+
+import (
+	"bufio"
+	"bytes"
+	"io"
+	"mime"
+	"os"
+	"strings"
+)
+
+type stringSet map[string]bool
+
+const xGmailLabelsHeader = "X-Gmail-Labels"
+
+// filteredOutGmailLabels is set of labels which we don't want to show to users
+// as they might be auto-generated by Gmail and unwanted.
+var filteredOutGmailLabels = []string{ //nolint[gochecknoglobals]
+	"Unread",
+	"Opened",
+	"IMAP_Junk",
+	"IMAP_NonJunk",
+	"IMAP_NotJunk",
+	"IMAP_$NotJunk",
+}
+
+func getGmailLabelsFromMboxFile(filePath string) (stringSet, error) {
+	f, err := os.Open(filePath) //nolint[gosec]
+	if err != nil {
+		return nil, err
+	}
+	return getGmailLabelsFromMboxReader(f)
+}
+
+func getGmailLabelsFromMboxReader(f io.Reader) (stringSet, error) {
+	allLabels := stringSet{}
+
+	// Scanner is not used as it does not support long lines and some mbox
+	// files contain very long lines even though that should not be happening.
+	r := bufio.NewReader(f)
+	for {
+		b, isPrefix, err := r.ReadLine()
+		if err == io.EOF {
+			break
+		}
+		if err != nil {
+			return nil, err
+		}
+		for isPrefix {
+			_, isPrefix, err = r.ReadLine()
+			if err != nil {
+				break
+			}
+		}
+		if bytes.HasPrefix(b, []byte(xGmailLabelsHeader)) {
+			for label := range getGmailLabelsFromValue(string(b)) {
+				allLabels[label] = true
+			}
+		}
+	}
+
+	return allLabels, nil
+}
+
+func getGmailLabelsFromMessage(body []byte) (stringSet, error) {
+	header, err := getMessageHeader(body)
+	if err != nil {
+		return nil, err
+	}
+	labels := header.Get(xGmailLabelsHeader)
+	return getGmailLabelsFromValue(labels), nil
+}
+
+func getGmailLabelsFromValue(value string) stringSet {
+	value = strings.TrimPrefix(value, xGmailLabelsHeader+":")
+	if decoded, err := new(mime.WordDecoder).DecodeHeader(value); err != nil {
+		log.WithError(err).Error("Failed to decode header")
+	} else {
+		value = decoded
+	}
+
+	labels := stringSet{}
+	for _, label := range strings.Split(value, ",") {
+		label = strings.TrimSpace(label)
+		if label == "" {
+			continue
+		}
+		skip := false
+		for _, filteredOutLabel := range filteredOutGmailLabels {
+			if label == filteredOutLabel {
+				skip = true
+				break
+			}
+		}
+		if skip {
+			continue
+		}
+		labels[label] = true
+	}
+	return labels
+}

internal/transfer/provider_mbox_gmail_labels_test.go

@@ -0,0 +1,135 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package transfer
+
+import (
+	"fmt"
+	"strings"
+	"testing"
+
+	r "github.com/stretchr/testify/require"
+)
+
+func TestGetGmailLabelsFromMboxReader(t *testing.T) {
+	mboxFile := `From - Mon May  4 16:40:31 2020
+Subject: Test 1
+X-Gmail-Labels: Foo,Bar
+
+hello
+
+From - Mon May  4 16:40:31 2020
+Subject: Test 2
+X-Gmail-Labels: Foo , Baz
+
+hello
+
+From - Mon May  4 16:40:31 2020
+Subject: Test 3
+X-Gmail-Labels: ,
+
+hello
+
+From - Mon May  4 16:40:31 2020
+Subject: Test 4
+X-Gmail-Labels:
+
+hello
+
+From - Mon May  4 16:40:31 2020
+Subject: Test 5
+
+hello
+
+`
+	mboxReader := strings.NewReader(mboxFile)
+	labels, err := getGmailLabelsFromMboxReader(mboxReader)
+	r.NoError(t, err)
+	r.Equal(t, toSet("Foo", "Bar", "Baz"), labels)
+}
+
+func TestGetGmailLabelsFromMessage(t *testing.T) {
+	tests := []struct {
+		body       string
+		wantLabels stringSet
+	}{
+		{`Subject: One
+X-Gmail-Labels: Foo,Bar
+
+Hello
+`, toSet("Foo", "Bar")},
+		{`Subject: Two
+X-Gmail-Labels: Foo , Bar ,
+
+Hello
+`, toSet("Foo", "Bar")},
+		{`Subject: Three
+X-Gmail-Labels: ,
+
+Hello
+`, toSet()},
+		{`Subject: Four
+X-Gmail-Labels:
+
+Hello
+`, toSet()},
+		{`Subject: Five
+
+Hello
+`, toSet()},
+	}
+	for _, tc := range tests {
+		tc := tc
+		t.Run(fmt.Sprintf("%v", tc.body), func(t *testing.T) {
+			labels, err := getGmailLabelsFromMessage([]byte(tc.body))
+			r.NoError(t, err)
+			r.Equal(t, tc.wantLabels, labels)
+		})
+	}
+}
+
+func TestGetGmailLabelsFromValue(t *testing.T) {
+	tests := []struct {
+		value      string
+		wantLabels stringSet
+	}{
+		{"Foo,Bar", toSet("Foo", "Bar")},
+		{" Foo , Bar ", toSet("Foo", "Bar")},
+		{" Foo , Bar , ", toSet("Foo", "Bar")},
+		{" Foo Bar ", toSet("Foo Bar")},
+		{" , ", toSet()},
+		{" ", toSet()},
+		{"", toSet()},
+		{"=?UTF-8?Q?Archived,Category_personal,test_=F0=9F=98=80=F0=9F=99=83?=", toSet("Archived", "Category personal", "test 😀🙃")},
+		{"IMAP_NotJunk,Foo,Opened,bar,Unread", toSet("Foo", "bar")},
+	}
+	for _, tc := range tests {
+		tc := tc
+		t.Run(fmt.Sprintf("%v", tc.value), func(t *testing.T) {
+			labels := getGmailLabelsFromValue(tc.value)
+			r.Equal(t, tc.wantLabels, labels)
+		})
+	}
+}
+
+func toSet(items ...string) stringSet {
+	set := map[string]bool{}
+	for _, item := range items {
+		set[item] = true
+	}
+	return set
+}

internal/transfer/provider_mbox_source.go

@@ -34,7 +34,7 @@ func (p *MBOXProvider) TransferTo(rules transferRules, progress *Progress, ch ch
 	log.Info("Started transfer from MBOX to channel")
 	defer log.Info("Finished transfer from MBOX to channel")
 
-	filePathsPerFolder, err := p.getFilePathsPerFolder(rules)
+	filePathsPerFolder, err := p.getFilePathsPerFolder()
 	if err != nil {
 		progress.fatal(err)
 		return
@@ -45,32 +45,29 @@ func (p *MBOXProvider) TransferTo(rules transferRules, progress *Progress, ch ch
 	}
 
 	for folderName, filePaths := range filePathsPerFolder {
-		// No error guaranteed by getFilePathsPerFolder.
+		log.WithField("folder", folderName).Debug("Estimating folder counts")
-		rule, _ := rules.getRuleBySourceMailboxName(folderName)
 		for _, filePath := range filePaths {
 			if progress.shouldStop() {
 				break
 			}
-			p.updateCount(rule, progress, filePath)
+			p.updateCount(progress, filePath)
 		}
 	}
 	progress.countsFinal()
 
 	for folderName, filePaths := range filePathsPerFolder {
-		// No error guaranteed by getFilePathsPerFolder.
+		log.WithField("folder", folderName).Debug("Processing folder")
-		rule, _ := rules.getRuleBySourceMailboxName(folderName)
-		log.WithField("rule", rule).Debug("Processing rule")
 		for _, filePath := range filePaths {
 			if progress.shouldStop() {
 				break
 			}
-			p.transferTo(rule, progress, ch, filePath)
+			p.transferTo(rules, progress, ch, folderName, filePath)
 		}
 	}
 }
 
-func (p *MBOXProvider) getFilePathsPerFolder(rules transferRules) (map[string][]string, error) {
+func (p *MBOXProvider) getFilePathsPerFolder() (map[string][]string, error) {
-	filePaths, err := getFilePathsWithSuffix(p.root, ".mbox")
+	filePaths, err := getAllPathsWithSuffix(p.root, ".mbox")
 	if err != nil {
 		return nil, err
 	}
@@ -78,19 +75,19 @@ func (p *MBOXProvider) getFilePathsPerFolder(rules transferRules) (map[string][]
 	filePathsMap := map[string][]string{}
 	for _, filePath := range filePaths {
 		fileName := filepath.Base(filePath)
-		folder := strings.TrimSuffix(fileName, ".mbox")
+		filePath, err := p.handleAppleMailMBOXStructure(filePath)
-		_, err := rules.getRuleBySourceMailboxName(folder)
+		// Skip unsupported MBOX structures. It was already filtered out in configuration step.
 		if err != nil {
-			log.WithField("msg", filePath).Trace("Mailbox skipped due to folder name")
 			continue
 		}
 
+		folder := strings.TrimSuffix(fileName, ".mbox")
 		filePathsMap[folder] = append(filePathsMap[folder], filePath)
 	}
 	return filePathsMap, nil
 }
 
-func (p *MBOXProvider) updateCount(rule *Rule, progress *Progress, filePath string) {
+func (p *MBOXProvider) updateCount(progress *Progress, filePath string) {
 	mboxReader := p.openMbox(progress, filePath)
 	if mboxReader == nil {
 		return
@@ -107,10 +104,10 @@ func (p *MBOXProvider) updateCount(rule *Rule, progress *Progress, filePath stri
 		}
 		count++
 	}
-	progress.updateCount(rule.SourceMailbox.Name, uint(count))
+	progress.updateCount(filePath, uint(count))
 }
 
-func (p *MBOXProvider) transferTo(rule *Rule, progress *Progress, ch chan<- Message, filePath string) {
+func (p *MBOXProvider) transferTo(rules transferRules, progress *Progress, ch chan<- Message, folderName, filePath string) {
 	mboxReader := p.openMbox(progress, filePath)
 	if mboxReader == nil {
 		return
@@ -134,50 +131,122 @@ func (p *MBOXProvider) transferTo(rule *Rule, progress *Progress, ch chan<- Mess
 			break
 		}
 
-		msg, err := p.exportMessage(rule, id, msgReader)
+		msg, err := p.exportMessage(rules, folderName, id, msgReader)
 
-		// Read and check time in body only if the rule specifies it
+		if err == nil && len(msg.Targets) == 0 {
-		// to not waste energy.
+			// Here should be called progress.messageSkipped(id) once we have
-		if err == nil && rule.HasTimeLimit() {
+			// this feature, and following progress.updateCount can be removed.
-			msgTime, msgTimeErr := getMessageTime(msg.Body)
+			continue
-			if msgTimeErr != nil {
-				err = msgTimeErr
-			} else if !rule.isTimeInRange(msgTime) {
-				log.WithField("msg", id).Debug("Message skipped due to time")
-				continue
-			}
 		}
 
-		// Counting only messages filtered by time to update count to correct total.
 		count++
 
 		// addMessage is called after time check to not report message
 		// which should not be exported but any error from reading body
 		// or parsing time is reported as an error.
-		progress.addMessage(id, rule)
+		progress.addMessage(id, msg.sourceNames(), msg.targetNames())
 		progress.messageExported(id, msg.Body, err)
 		if err == nil {
 			ch <- msg
 		}
 	}
-	progress.updateCount(rule.SourceMailbox.Name, uint(count))
+	progress.updateCount(filePath, uint(count))
 }
 
-func (p *MBOXProvider) exportMessage(rule *Rule, id string, msgReader io.Reader) (Message, error) {
+func (p *MBOXProvider) exportMessage(rules transferRules, folderName, id string, msgReader io.Reader) (Message, error) {
 	body, err := ioutil.ReadAll(msgReader)
 	if err != nil {
 		return Message{}, errors.Wrap(err, "failed to read message")
 	}
 
+	msgRules := p.getMessageRules(rules, folderName, id, body)
+	sources := p.getMessageSources(msgRules)
+	targets := p.getMessageTargets(msgRules, id, body)
 	return Message{
 		ID:      id,
 		Unread:  false,
 		Body:    body,
-		Source:  rule.SourceMailbox,
+		Sources: sources,
-		Targets: rule.TargetMailboxes,
+		Targets: targets,
 	}, nil
 }
 
+func (p *MBOXProvider) getMessageRules(rules transferRules, folderName, id string, body []byte) []*Rule {
+	msgRules := []*Rule{}
+
+	folderRule, err := rules.getRuleBySourceMailboxName(folderName)
+	if err != nil {
+		log.WithField("msg", id).WithField("source", folderName).Debug("Message source doesn't have a rule")
+	} else {
+		msgRules = append(msgRules, folderRule)
+	}
+
+	gmailLabels, err := getGmailLabelsFromMessage(body)
+	if err != nil {
+		log.WithError(err).Error("Failed to get gmail labels, ")
+	} else {
+		for label := range gmailLabels {
+			rule, err := rules.getRuleBySourceMailboxName(label)
+			if err != nil {
+				log.WithField("msg", id).WithField("source", label).Debug("Message source doesn't have a rule")
+				continue
+			}
+			msgRules = append(msgRules, rule)
+		}
+	}
+
+	return msgRules
+}
+
+func (p *MBOXProvider) getMessageSources(msgRules []*Rule) []Mailbox {
+	sources := []Mailbox{}
+	for _, rule := range msgRules {
+		sources = append(sources, rule.SourceMailbox)
+	}
+	return sources
+}
+
+func (p *MBOXProvider) getMessageTargets(msgRules []*Rule, id string, body []byte) []Mailbox {
+	targets := []Mailbox{}
+	haveExclusiveMailbox := false
+	for _, rule := range msgRules {
+		// Read and check time in body only if the rule specifies it
+		// to not waste energy.
+		if rule.HasTimeLimit() {
+			msgTime, err := getMessageTime(body)
+			if err != nil {
+				log.WithError(err).Error("Failed to parse time, time check skipped")
+			} else if !rule.isTimeInRange(msgTime) {
+				log.WithField("msg", id).WithField("source", rule.SourceMailbox.Name).Debug("Message skipped due to time")
+				continue
+			}
+		}
+		for _, newTarget := range rule.TargetMailboxes {
+			// msgRules is sorted. The first rule is based on the folder name,
+			// followed by the order from X-Gmail-Labels. The rule based on
+			// the folder name should have priority for exclusive target.
+			if newTarget.IsExclusive && haveExclusiveMailbox {
+				continue
+			}
+			found := false
+			for _, target := range targets {
+				if target.Hash() == newTarget.Hash() {
+					found = true
+					break
+				}
+			}
+			if found {
+				continue
+			}
+			if newTarget.IsExclusive {
+				haveExclusiveMailbox = true
+			}
+			targets = append(targets, newTarget)
+		}
+	}
+	return targets
+}
+
 func (p *MBOXProvider) openMbox(progress *Progress, mboxPath string) *mbox.Reader {
 	mboxPath = filepath.Join(p.root, mboxPath)
 	mboxFile, err := os.Open(mboxPath) //nolint[gosec]

internal/transfer/provider_mbox_target.go

@@ -57,7 +57,7 @@ func (p *MBOXProvider) TransferFrom(rules transferRules, progress *Progress, ch
 func (p *MBOXProvider) writeMessage(msg Message) error {
 	var multiErr error
 	for _, mailbox := range msg.Targets {
-		mboxName := filepath.Base(mailbox.Name)
+		mboxName := sanitizeFileName(mailbox.Name)
 		if !strings.HasSuffix(mboxName, ".mbox") {
 			mboxName += ".mbox"
 		}

internal/transfer/provider_mbox_test.go

@@ -35,27 +35,35 @@ func newTestMBOXProvider(path string) *MBOXProvider {
 }
 
 func TestMBOXProviderMailboxes(t *testing.T) {
-	provider := newTestMBOXProvider("")
-
 	tests := []struct {
+		provider      *MBOXProvider
 		includeEmpty  bool
 		wantMailboxes []Mailbox
 	}{
-		{true, []Mailbox{
+		{newTestMBOXProvider(""), true, []Mailbox{
+			{Name: "All Mail"},
 			{Name: "Foo"},
+			{Name: "Bar"},
 			{Name: "Inbox"},
 		}},
-		{false, []Mailbox{
+		{newTestMBOXProvider(""), false, []Mailbox{
+			{Name: "All Mail"},
 			{Name: "Foo"},
+			{Name: "Bar"},
 			{Name: "Inbox"},
 		}},
+		{newTestMBOXProvider("testdata/mbox-applemail"), true, []Mailbox{
+			{Name: "All Mail"},
+			{Name: "Foo"},
+			{Name: "Bar"},
+		}},
 	}
 	for _, tc := range tests {
 		tc := tc
 		t.Run(fmt.Sprintf("%v", tc.includeEmpty), func(t *testing.T) {
-			mailboxes, err := provider.Mailboxes(tc.includeEmpty, false)
+			mailboxes, err := tc.provider.Mailboxes(tc.includeEmpty, false)
 			r.NoError(t, err)
-			r.Equal(t, tc.wantMailboxes, mailboxes)
+			r.ElementsMatch(t, tc.wantMailboxes, mailboxes)
 		})
 	}
 }
@@ -67,14 +75,47 @@ func TestMBOXProviderTransferTo(t *testing.T) {
 	defer rulesClose()
 	setupMBOXRules(rules)
 
-	testTransferTo(t, rules, provider, []string{
+	msgs := testTransferTo(t, rules, provider, []string{
+		"All Mail.mbox:1",
+		"All Mail.mbox:2",
 		"Foo.mbox:1",
 		"Inbox.mbox:1",
 	})
+	got := map[string][]string{}
+	for _, msg := range msgs {
+		got[msg.ID] = msg.targetNames()
+	}
+	r.Equal(t, map[string][]string{
+		"All Mail.mbox:1": {"Archive", "Foo"}, // Bar is not in rules.
+		"All Mail.mbox:2": {"Archive", "Foo"},
+		"Foo.mbox:1":      {"Foo"},
+		"Inbox.mbox:1":    {"Inbox"},
+	}, got)
+}
+
+func TestMBOXProviderTransferToAppleMail(t *testing.T) {
+	provider := newTestMBOXProvider("testdata/mbox-applemail")
+
+	rules, rulesClose := newTestRules(t)
+	defer rulesClose()
+	setupMBOXRules(rules)
+
+	msgs := testTransferTo(t, rules, provider, []string{
+		"All Mail.mbox/mbox:1",
+		"All Mail.mbox/mbox:2",
+	})
+	got := map[string][]string{}
+	for _, msg := range msgs {
+		got[msg.ID] = msg.targetNames()
+	}
+	r.Equal(t, map[string][]string{
+		"All Mail.mbox/mbox:1": {"Archive", "Foo"}, // Bar is not in rules.
+		"All Mail.mbox/mbox:2": {"Archive", "Foo"},
+	}, got)
 }
 
 func TestMBOXProviderTransferFrom(t *testing.T) {
-	dir, err := ioutil.TempDir("", "eml")
+	dir, err := ioutil.TempDir("", "mbox")
 	r.NoError(t, err)
 	defer os.RemoveAll(dir) //nolint[errcheck]
 
@@ -94,7 +135,7 @@ func TestMBOXProviderTransferFrom(t *testing.T) {
 }
 
 func TestMBOXProviderTransferFromTo(t *testing.T) {
-	dir, err := ioutil.TempDir("", "eml")
+	dir, err := ioutil.TempDir("", "mbox")
 	r.NoError(t, err)
 	defer os.RemoveAll(dir) //nolint[errcheck]
 
@@ -103,23 +144,57 @@ func TestMBOXProviderTransferFromTo(t *testing.T) {
 
 	rules, rulesClose := newTestRules(t)
 	defer rulesClose()
-	setupEMLRules(rules)
+	setupMBOXRules(rules)
 
 	testTransferFromTo(t, rules, source, target, 5*time.Second)
 
 	checkMBOXFileStructure(t, dir, []string{
+		"Archive.mbox",
 		"Foo.mbox",
 		"Inbox.mbox",
 	})
 }
 
+func TestMBOXProviderGetMessageTargetsReturnsOnlyOneFolder(t *testing.T) {
+	provider := newTestMBOXProvider("")
+
+	folderA := Mailbox{Name: "Folder A", IsExclusive: true}
+	folderB := Mailbox{Name: "Folder B", IsExclusive: true}
+	labelA := Mailbox{Name: "Label A", IsExclusive: false}
+	labelB := Mailbox{Name: "Label B", IsExclusive: false}
+	labelC := Mailbox{Name: "Label C", IsExclusive: false}
+
+	rule1 := &Rule{TargetMailboxes: []Mailbox{folderA, labelA, labelB}}
+	rule2 := &Rule{TargetMailboxes: []Mailbox{folderB, labelC}}
+	rule3 := &Rule{TargetMailboxes: []Mailbox{folderB}}
+
+	tests := []struct {
+		rules         []*Rule
+		wantMailboxes []Mailbox
+	}{
+		{[]*Rule{}, []Mailbox{}},
+		{[]*Rule{rule1}, []Mailbox{folderA, labelA, labelB}},
+		{[]*Rule{rule1, rule2}, []Mailbox{folderA, labelA, labelB, labelC}},
+		{[]*Rule{rule1, rule3}, []Mailbox{folderA, labelA, labelB}},
+		{[]*Rule{rule3, rule1}, []Mailbox{folderB, labelA, labelB}},
+	}
+	for _, tc := range tests {
+		tc := tc
+		t.Run(fmt.Sprintf("%v", tc.rules), func(t *testing.T) {
+			mailboxes := provider.getMessageTargets(tc.rules, "", []byte(""))
+			r.Equal(t, tc.wantMailboxes, mailboxes)
+		})
+	}
+}
+
 func setupMBOXRules(rules transferRules) {
+	_ = rules.setRule(Mailbox{Name: "All Mail"}, []Mailbox{{Name: "Archive"}}, 0, 0)
 	_ = rules.setRule(Mailbox{Name: "Inbox"}, []Mailbox{{Name: "Inbox"}}, 0, 0)
 	_ = rules.setRule(Mailbox{Name: "Foo"}, []Mailbox{{Name: "Foo"}}, 0, 0)
 }
 
 func checkMBOXFileStructure(t *testing.T, root string, expectedFiles []string) {
-	files, err := getFilePathsWithSuffix(root, ".mbox")
+	files, err := getAllPathsWithSuffix(root, ".mbox")
 	r.NoError(t, err)
 	r.Equal(t, expectedFiles, files)
 }

internal/transfer/provider_pmapi_source.go

@@ -123,7 +123,7 @@ func (p *PMAPIProvider) transferTo(rule *Rule, progress *Progress, ch chan<- Mes
 				}
 
 				msgID := fmt.Sprintf("%s_%s", rule.SourceMailbox.ID, pmapiMessage.ID)
-				progress.addMessage(msgID, rule)
+				progress.addMessage(msgID, []string{rule.SourceMailbox.Name}, rule.TargetMailboxNames())
 				msg, err := p.exportMessage(rule, progress, pmapiMessage.ID, msgID, skipEncryptedMessages)
 				progress.messageExported(msgID, msg.Body, err)
 				if err == nil {
@@ -177,7 +177,7 @@ func (p *PMAPIProvider) exportMessage(rule *Rule, progress *Progress, pmapiMsgID
 		ID:      msgID,
 		Unread:  unread,
 		Body:    body,
-		Source:  rule.SourceMailbox,
+		Sources: []Mailbox{rule.SourceMailbox},
 		Targets: rule.TargetMailboxes,
 	}, nil
 }

internal/transfer/provider_pmapi_target.go

@@ -177,6 +177,10 @@ func (p *PMAPIProvider) transferMessage(rules transferRules, progress *Progress,
 		return
 	}
 
+	if progress.shouldStop() {
+		return
+	}
+
 	importMsgReqSize := len(importMsgReq.Body)
 	if p.nextImportRequestsSize+importMsgReqSize > pmapiImportBatchMaxSize || len(p.nextImportRequests) == pmapiImportBatchMaxItems {
 		preparedImportRequestsCh <- p.nextImportRequests

internal/transfer/provider_pmapi_utils.go

@@ -72,7 +72,8 @@ func (p *PMAPIProvider) tryReconnect() error {
 
 func (p *PMAPIProvider) listMessages(filter *pmapi.MessagesFilter) (messages []*pmapi.Message, count int, err error) {
 	err = p.ensureConnection(func() error {
-		key := fmt.Sprintf("%s_%d", filter.LabelID, filter.Page)
+		// Sort is used in the key so the filter is different for estimating and real fetching.
+		key := fmt.Sprintf("%s_%s_%d", filter.LabelID, filter.Sort, filter.Page)
 		p.timeIt.start("listing", key)
 		defer p.timeIt.stop("listing", key)
 
@@ -117,8 +118,10 @@ func (p *PMAPIProvider) createDraft(msgSourceID string, message *pmapi.Message,
 
 func (p *PMAPIProvider) createAttachment(msgSourceID string, att *pmapi.Attachment, r io.Reader, sig io.Reader) (created *pmapi.Attachment, err error) {
 	err = p.ensureConnection(func() error {
-		p.timeIt.start("upload", msgSourceID)
+		// Use some attributes from attachment to have unique key for each call.
-		defer p.timeIt.stop("upload", msgSourceID)
+		key := fmt.Sprintf("%s_%s_%d", msgSourceID, att.Name, att.Size)
+		p.timeIt.start("upload", key)
+		defer p.timeIt.stop("upload", key)
 
 		created, err = p.client().CreateAttachment(att, r, sig)
 		return err

internal/transfer/provider_test.go

@@ -43,7 +43,7 @@ hello
 `, subject))
 }
 
-func testTransferTo(t *testing.T, rules transferRules, provider SourceProvider, expectedMessageIDs []string) {
+func testTransferTo(t *testing.T, rules transferRules, provider SourceProvider, expectedMessageIDs []string) []Message {
 	progress := newProgress(log, nil)
 	drainProgressUpdateChannel(&progress)
 
@@ -53,13 +53,17 @@ func testTransferTo(t *testing.T, rules transferRules, provider SourceProvider,
 		close(ch)
 	}()
 
+	msgs := []Message{}
 	gotMessageIDs := []string{}
 	for msg := range ch {
+		msgs = append(msgs, msg)
 		gotMessageIDs = append(gotMessageIDs, msg.ID)
 	}
 	r.ElementsMatch(t, expectedMessageIDs, gotMessageIDs)
 
 	r.Empty(t, progress.GetFailedMessages())
+
+	return msgs
 }
 
 func testTransferFrom(t *testing.T, rules transferRules, provider TargetProvider, messages []Message) {
@@ -69,7 +73,7 @@ func testTransferFrom(t *testing.T, rules transferRules, provider TargetProvider
 	ch := make(chan Message)
 	go func() {
 		for _, message := range messages {
-			progress.addMessage(message.ID, nil)
+			progress.addMessage(message.ID, []string{}, []string{})
 			progress.messageExported(message.ID, []byte(""), nil)
 			ch <- message
 		}

internal/transfer/report.go

@@ -114,7 +114,7 @@ type messageReport struct {
 	SourceID        string
 	TargetID        string
 	BodyHash        string
-	SourceMailbox   string
+	SourceMailboxes []string
 	TargetMailboxes []string
 	Error           string
 
@@ -130,8 +130,8 @@ func newMessageReportFromMessageStatus(messageStatus *MessageStatus, includePriv
 		SourceID:        messageStatus.SourceID,
 		TargetID:        messageStatus.targetID,
 		BodyHash:        messageStatus.bodyHash,
-		SourceMailbox:   messageStatus.rule.SourceMailbox.Name,
+		SourceMailboxes: messageStatus.sourceNames,
-		TargetMailboxes: messageStatus.rule.TargetMailboxNames(),
+		TargetMailboxes: messageStatus.targetNames,
 		Error:           messageStatus.GetErrorMessage(),
 	}
 

internal/transfer/testdata/mbox-applemail/All Mail.mbox/mbox

@@ -0,0 +1,16 @@
+From - Mon May  4 16:40:31 2020
+From: Bridge Test <bridgetest@pm.test>
+To: Bridge Test <bridgetest@protonmail.com>
+Subject: Test 1
+X-Gmail-Labels: Foo,Bar
+
+hello
+
+
+From - Mon May  4 16:40:31 2020
+From: Bridge Test <bridgetest@pm.test>
+To: Bridge Test <bridgetest@protonmail.com>
+Subject: Test 2
+X-Gmail-Labels: Foo
+
+hello

internal/transfer/testdata/mbox/All Mail.mbox

@@ -0,0 +1,16 @@
+From - Mon May  4 16:40:31 2020
+From: Bridge Test <bridgetest@pm.test>
+To: Bridge Test <bridgetest@protonmail.com>
+Subject: Test 1
+X-Gmail-Labels: Foo,Bar
+
+hello
+
+
+From - Mon May  4 16:40:31 2020
+From: Bridge Test <bridgetest@pm.test>
+To: Bridge Test <bridgetest@protonmail.com>
+Subject: Test 2
+X-Gmail-Labels: Foo
+
+hello

internal/transfer/utils.go

@@ -24,9 +24,11 @@ import (
 	"net/mail"
 	"net/textproto"
 	"path/filepath"
+	"runtime"
 	"sort"
 	"strings"
 
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322"
 	"github.com/pkg/errors"
 )
 
@@ -81,7 +83,7 @@ func getFolderNamesWithFileSuffix(root, fileSuffix string) ([]string, error) {
 // getFilePathsWithSuffix collects all file names with `suffix` under `root`.
 // File names will be with relative path based to `root`.
 func getFilePathsWithSuffix(root, suffix string) ([]string, error) {
-	fileNames, err := getFilePathsWithSuffixInner("", root, suffix)
+	fileNames, err := getFilePathsWithSuffixInner("", root, suffix, false)
 	if err != nil {
 		return nil, err
 	}
@@ -89,7 +91,18 @@ func getFilePathsWithSuffix(root, suffix string) ([]string, error) {
 	return fileNames, err
 }
 
-func getFilePathsWithSuffixInner(prefix, root, suffix string) ([]string, error) {
+// getAllPathsWithSuffix is the same as getFilePathsWithSuffix but includes
+// also directories.
+func getAllPathsWithSuffix(root, suffix string) ([]string, error) {
+	fileNames, err := getFilePathsWithSuffixInner("", root, suffix, true)
+	if err != nil {
+		return nil, err
+	}
+	sort.Strings(fileNames)
+	return fileNames, err
+}
+
+func getFilePathsWithSuffixInner(prefix, root, suffix string, includeDir bool) ([]string, error) {
 	fileNames := []string{}
 
 	files, err := ioutil.ReadDir(root)
@@ -103,10 +116,14 @@ func getFilePathsWithSuffixInner(prefix, root, suffix string) ([]string, error)
 				fileNames = append(fileNames, filepath.Join(prefix, file.Name()))
 			}
 		} else {
+			if includeDir && strings.HasSuffix(file.Name(), suffix) {
+				fileNames = append(fileNames, filepath.Join(prefix, file.Name()))
+			}
 			subfolderFileNames, err := getFilePathsWithSuffixInner(
 				filepath.Join(prefix, file.Name()),
 				filepath.Join(root, file.Name()),
 				suffix,
+				includeDir,
 			)
 			if err != nil {
 				return nil, err
@@ -120,14 +137,21 @@ func getFilePathsWithSuffixInner(prefix, root, suffix string) ([]string, error)
 
 // getMessageTime returns time of the message specified in the message header.
 func getMessageTime(body []byte) (int64, error) {
-	mailHeader, err := getMessageHeader(body)
+	hdr, err := getMessageHeader(body)
 	if err != nil {
 		return 0, err
 	}
-	if t, err := mailHeader.Date(); err == nil && !t.IsZero() {
+
-		return t.Unix(), nil
+	t, err := rfc5322.ParseDateTime(hdr.Get("Date"))
+	if err != nil {
+		return 0, err
 	}
-	return 0, nil
+
+	if t.IsZero() {
+		return 0, nil
+	}
+
+	return t.Unix(), nil
 }
 
 // getMessageHeader returns headers of the message body.
@@ -139,3 +163,24 @@ func getMessageHeader(body []byte) (mail.Header, error) {
 	}
 	return mail.Header(header), nil
 }
+
+// sanitizeFileName replaces problematic special characters with underscore.
+func sanitizeFileName(fileName string) string {
+	if len(fileName) == 0 {
+		return fileName
+	}
+	if runtime.GOOS != "windows" && (fileName[0] == '-' || fileName[0] == '.') { //nolint[goconst]
+		fileName = "_" + fileName[1:]
+	}
+	return strings.Map(func(r rune) rune {
+		switch r {
+		case '\\', '/', ':', '*', '?', '"', '<', '>', '|':
+			return '_'
+		case '[', ']', '(', ')', '{', '}', '^', '#', '%', '&', '!', '@', '+', '=', '\'', '~':
+			if runtime.GOOS != "windows" {
+				return '_'
+			}
+		}
+		return r
+	}, fileName)
+}

internal/transfer/utils_test.go

@@ -21,6 +21,7 @@ import (
 	"io/ioutil"
 	"os"
 	"path/filepath"
+	"runtime"
 	"testing"
 
 	r "github.com/stretchr/testify/require"
@@ -38,6 +39,7 @@ func TestGetFolderNames(t *testing.T) {
 			"",
 			[]string{
 				"bar",
+				"bar.mbox",
 				"baz",
 				filepath.Base(root),
 				"foo",
@@ -94,6 +96,13 @@ func TestGetFilePathsWithSuffix(t *testing.T) {
 				"test/foo/msg9.eml",
 			},
 		},
+		{
+			".mbox",
+			[]string{
+				"bar.mbox",
+				"foo.mbox",
+			},
+		},
 		{
 			".txt",
 			[]string{
@@ -108,7 +117,7 @@ func TestGetFilePathsWithSuffix(t *testing.T) {
 	for _, tc := range tests {
 		tc := tc
 		t.Run(tc.suffix, func(t *testing.T) {
-			paths, err := getFilePathsWithSuffix(root, tc.suffix)
+			paths, err := getAllPathsWithSuffix(root, tc.suffix)
 			r.NoError(t, err)
 			r.Equal(t, tc.wantPaths, paths)
 		})
@@ -124,6 +133,7 @@ func createTestingFolderStructure(t *testing.T) (string, func()) {
 		"foo/baz",
 		"test/foo",
 		"qwerty",
+		"bar.mbox",
 	} {
 		err = os.MkdirAll(filepath.Join(root, path), os.ModePerm)
 		r.NoError(t, err)
@@ -141,6 +151,8 @@ func createTestingFolderStructure(t *testing.T) (string, func()) {
 		"test/foo/msg9.eml",
 		"msg10.eml",
 		"info.txt",
+		"foo.mbox",
+		"bar.mbox/mbox", // Apple Mail mbox export format.
 	} {
 		f, err := os.Create(filepath.Join(root, path))
 		r.NoError(t, err)
@@ -188,3 +200,26 @@ Body
 	r.Equal(t, header.Get("subject"), "Hello")
 	r.Equal(t, header.Get("from"), "user@example.com")
 }
+
+func TestSanitizeFileName(t *testing.T) {
+	tests := map[string]string{
+		"hello":            "hello",
+		"a\\b/c:*?d\"<>|e": "a_b_c___d____e",
+	}
+	if runtime.GOOS == "darwin" || runtime.GOOS == "linux" {
+		tests[".hello"] = "_hello"
+		tests["-hello"] = "_hello"
+	}
+	if runtime.GOOS == "windows" {
+		tests["[hello]&@=~~"] = "_hello______"
+	}
+
+	for path, wantPath := range tests {
+		path := path
+		wantPath := wantPath
+		t.Run(path, func(t *testing.T) {
+			gotPath := sanitizeFileName(path)
+			r.Equal(t, wantPath, gotPath)
+		})
+	}
+}

internal/updates/sync.go

@@ -45,7 +45,7 @@ func syncFolders(localPath, updatePath string) (err error) {
 }
 
 func removeMissing(folderToCleanPath, itemsToKeepPath string) (err error) {
-	log.Debug("remove missing")
+	log.WithField("from", folderToCleanPath).Debug("Remove missing.")
 	// Create list of files.
 	existingRelPaths := map[string]bool{}
 	err = filepath.Walk(itemsToKeepPath, func(keepThis string, _ os.FileInfo, walkErr error) error {
@@ -56,7 +56,7 @@ func removeMissing(folderToCleanPath, itemsToKeepPath string) (err error) {
 		if walkErr != nil {
 			return walkErr
 		}
-		log.Debug("path to keep ", relPath)
+		log.WithField("path", relPath).Trace("Keep the path.")
 		existingRelPaths[relPath] = true
 		return nil
 	})
@@ -95,12 +95,18 @@ func removeMissing(folderToCleanPath, itemsToKeepPath string) (err error) {
 }
 
 func restoreFromBackup(backupDir, localPath string) {
-	log.Error("recovering from ", backupDir, " to ", localPath)
+	log.WithField("from", backupDir).
-	_ = copyRecursively(backupDir, localPath)
+		WithField("to", localPath).
+		Error("recovering")
+	if err := copyRecursively(backupDir, localPath); err != nil {
+		log.WithField("from", backupDir).
+			WithField("to", localPath).
+			Error("Not able to recover.")
+	}
 }
 
 func createBackup(srcFile, dstDir string) (err error) {
-	log.Debug("backup ", srcFile, " in ", dstDir)
+	log.WithField("from", srcFile).WithField("to", dstDir).Debug("Create backup")
 	if err = mkdirAllClear(dstDir); err != nil {
 		return
 	}

internal/updates/updates.go

@@ -107,7 +107,7 @@ func NewImportExport(updateTempDir string) *Updates {
 		versionFileBaseName: "current_version_ie",
 		updateFileBaseName:  "ie/ie_upgrade",
 		linuxFileBaseName:   "ie/protonmail-import-export-app",
-		macAppBundleName:    "Import-Export app.app",
+		macAppBundleName:    "ProtonMail Import-Export app.app",
 	}
 }
 
@@ -310,7 +310,9 @@ func (u *Updates) StartUpgrade(currentStatus chan<- Progress) { // nolint[funlen
 	status.UpdateDescription(InfoUpgrading)
 	switch runtime.GOOS {
 	case "windows": //nolint[goconst]
-		installerFile := strings.Split(u.winInstallerFile, "/")[1]
+		// Cannot use filepath.Base on windows it has different delimiter
+		split := strings.Split(u.winInstallerFile, "/")
+		installerFile := split[len(split)-1]
 		cmd := exec.Command("./" + installerFile) // nolint[gosec]
 		cmd.Dir = u.updateTempDir
 		status.Err = cmd.Start()
@@ -326,10 +328,15 @@ func (u *Updates) StartUpgrade(currentStatus chan<- Progress) { // nolint[funlen
 		localPath = filepath.Dir(localPath) // .app
 
 		updatePath := filepath.Join(u.updateTempDir, u.macAppBundleName)
-		log.Warn("localPath ", localPath)
+		log.WithField("local", localPath).
-		log.Warn("updatePath ", updatePath)
+			WithField("update", updatePath).
+			Info("Syncing folders..")
 		status.Err = syncFolders(localPath, updatePath)
 		if status.Err != nil {
+			log.WithField("from", localPath).
+				WithField("to", updatePath).
+				WithError(status.Err).
+				Error("Sync failed.")
 			return
 		}
 		status.UpdateDescription(InfoRestartApp)

pkg/message/header.go

@@ -19,9 +19,7 @@ package message
 
 import (
 	"mime"
-	"net/mail"
 	"net/textproto"
-	"regexp"
 	"strings"
 	"time"
 
@@ -86,10 +84,6 @@ func GetHeader(msg *pmapi.Message) textproto.MIMEHeader { //nolint[funlen]
 	}
 	if msg.ConversationID != "" {
 		h.Set("X-Pm-ConversationID-Id", msg.ConversationID)
-		if references := h.Get("References"); !strings.Contains(references, msg.ConversationID) {
-			references += " <" + msg.ConversationID + "@" + pmapi.ConversationIDDomain + ">"
-			h.Set("References", references)
-		}
 	}
 
 	return h
@@ -141,46 +135,3 @@ func GetAttachmentHeader(att *pmapi.Attachment) textproto.MIMEHeader {
 
 	return h
 }
-
-var reEmailComment = regexp.MustCompile("[(][^)]*[)]") // nolint[gochecknoglobals]
-
-// parseAddressComment removes the comments completely even though they should be allowed
-// http://tools.wordtothewise.com/rfc/822
-// NOTE: This should be supported in go>1.10 but it seems it's not ¯\_(ツ)_/¯
-func parseAddressComment(raw string) string {
-	return reEmailComment.ReplaceAllString(raw, "")
-}
-
-func parseAddressList(val string) (addrs []*mail.Address, err error) {
-	if val == "" || val == "<>" {
-		return
-	}
-
-	addrs, err = mail.ParseAddressList(parseAddressComment(val))
-	if err == nil {
-		if addrs == nil {
-			addrs = []*mail.Address{}
-		}
-		return
-	}
-
-	// Probably missing encoding error -- try to at least parse addresses in brackets.
-	first := strings.Index(val, "<")
-	last := strings.LastIndex(val, ">")
-	if first < 0 || last < 0 || first >= last {
-		return
-	}
-	var addrList []string
-	open := first
-	for open < last && 0 <= open {
-		val = val[open:]
-		close := strings.Index(val, ">")
-		addrList = append(addrList, val[:close+1])
-		val = val[close:]
-		open = strings.Index(val, "<")
-		last = strings.LastIndex(val, ">")
-	}
-	val = strings.Join(addrList, ", ")
-
-	return mail.ParseAddressList(val)
-}

pkg/message/parser.go

@@ -27,6 +27,7 @@ import (
 	"strings"
 
 	"github.com/ProtonMail/proton-bridge/pkg/message/parser"
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322"
 	pmmime "github.com/ProtonMail/proton-bridge/pkg/mime"
 	"github.com/ProtonMail/proton-bridge/pkg/pmapi"
 	"github.com/emersion/go-message"
@@ -365,7 +366,6 @@ func attachPublicKey(p *parser.Part, key, keyName string) {
 	})
 }
 
-// NOTE: We should use our own ParseAddressList here.
 func parseMessageHeader(m *pmapi.Message, h message.Header) error { // nolint[funlen]
 	mimeHeader, err := toMailHeader(h)
 	if err != nil {
@@ -373,59 +373,64 @@ func parseMessageHeader(m *pmapi.Message, h message.Header) error { // nolint[fu
 	}
 	m.Header = mimeHeader
 
-	if err := forEachDecodedHeaderField(h, func(key, val string) error {
+	fields := h.Fields()
-		switch strings.ToLower(key) {
+
+	for fields.Next() {
+		switch strings.ToLower(fields.Key()) {
 		case "subject":
-			m.Subject = val
+			s, err := fields.Text()
+			if err != nil {
+				if s, err = pmmime.DecodeHeader(fields.Value()); err != nil {
+					return errors.Wrap(err, "failed to parse subject")
+				}
+			}
+
+			m.Subject = s
 
 		case "from":
-			sender, err := parseAddressList(val)
+			sender, err := rfc5322.ParseAddressList(fields.Value())
 			if err != nil {
-				return err
+				return errors.Wrap(err, "failed to parse from")
 			}
 			if len(sender) > 0 {
 				m.Sender = sender[0]
 			}
 
 		case "to":
-			toList, err := parseAddressList(val)
+			toList, err := rfc5322.ParseAddressList(fields.Value())
 			if err != nil {
-				return err
+				return errors.Wrap(err, "failed to parse to")
 			}
 			m.ToList = toList
 
 		case "reply-to":
-			replyTos, err := parseAddressList(val)
+			replyTos, err := rfc5322.ParseAddressList(fields.Value())
 			if err != nil {
-				return err
+				return errors.Wrap(err, "failed to parse reply-to")
 			}
 			m.ReplyTos = replyTos
 
 		case "cc":
-			ccList, err := parseAddressList(val)
+			ccList, err := rfc5322.ParseAddressList(fields.Value())
 			if err != nil {
-				return err
+				return errors.Wrap(err, "failed to parse cc")
 			}
 			m.CCList = ccList
 
 		case "bcc":
-			bccList, err := parseAddressList(val)
+			bccList, err := rfc5322.ParseAddressList(fields.Value())
 			if err != nil {
-				return err
+				return errors.Wrap(err, "failed to parse bcc")
 			}
 			m.BCCList = bccList
 
 		case "date":
-			date, err := mail.ParseDate(val)
+			date, err := rfc5322.ParseDateTime(fields.Value())
 			if err != nil {
-				return err
+				return errors.Wrap(err, "failed to parse date")
 			}
 			m.Time = date.Unix()
 		}
-
-		return nil
-	}); err != nil {
-		return err
 	}
 
 	return nil
@@ -469,29 +474,6 @@ func parseAttachment(h message.Header) (*pmapi.Attachment, error) {
 	return att, nil
 }
 
-func forEachDecodedHeaderField(h message.Header, fn func(string, string) error) error {
-	fields := h.Fields()
-
-	for fields.Next() {
-		text, err := fields.Text()
-		if err != nil {
-			if !message.IsUnknownCharset(err) {
-				return err
-			}
-
-			if text, err = pmmime.DecodeHeader(fields.Value()); err != nil {
-				return err
-			}
-		}
-
-		if err := fn(fields.Key(), text); err != nil {
-			return err
-		}
-	}
-
-	return nil
-}
-
 func toMailHeader(h message.Header) (mail.Header, error) {
 	mimeHeader := make(mail.Header)
 
@@ -517,3 +499,26 @@ func toMIMEHeader(h message.Header) (textproto.MIMEHeader, error) {
 
 	return mimeHeader, nil
 }
+
+func forEachDecodedHeaderField(h message.Header, fn func(string, string) error) error {
+	fields := h.Fields()
+
+	for fields.Next() {
+		text, err := fields.Text()
+		if err != nil {
+			if !message.IsUnknownCharset(err) {
+				return err
+			}
+
+			if text, err = pmmime.DecodeHeader(fields.Value()); err != nil {
+				return err
+			}
+		}
+
+		if err := fn(fields.Key(), text); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}

pkg/message/parser/writer.go

@@ -35,7 +35,7 @@ func newWriter(root *Part) *Writer {
 
 func (w *Writer) Write(ww io.Writer) error {
 	if !w.root.is7BitClean() {
-		w.root.Header.Add("Content-Transfer-Encoding", "base64")
+		w.root.Header.Set("Content-Transfer-Encoding", "base64")
 	}
 
 	msgWriter, err := message.CreateWriter(ww, w.root.Header)
@@ -68,7 +68,7 @@ func (w *Writer) write(writer *message.Writer, p *Part) error {
 
 func (w *Writer) writeAsChild(writer *message.Writer, p *Part) error {
 	if !p.is7BitClean() {
-		p.Header.Add("Content-Transfer-Encoding", "base64")
+		p.Header.Set("Content-Transfer-Encoding", "base64")
 	}
 
 	childWriter, err := writer.CreatePart(p.Header)

pkg/message/parser_test.go

@@ -498,80 +498,3 @@ func readerToString(r io.Reader) string {
 
 	return string(b)
 }
-
-func TestRFC822AddressFormat(t *testing.T) { //nolint[funlen]
-	tests := []struct {
-		address  string
-		expected []string
-	}{
-		{
-			" normal name  <username@server.com>",
-			[]string{
-				"\"normal name\" <username@server.com>",
-			},
-		},
-		{
-			" \"comma, name\"  <username@server.com>",
-			[]string{
-				"\"comma, name\" <username@server.com>",
-			},
-		},
-		{
-			" name  <username@server.com> (ignore comment)",
-			[]string{
-				"\"name\" <username@server.com>",
-			},
-		},
-		{
-			" name (ignore comment)  <username@server.com>,  (Comment as name) username2@server.com",
-			[]string{
-				"\"name\" <username@server.com>",
-				"<username2@server.com>",
-			},
-		},
-		{
-			" normal name  <username@server.com>, (comment)All.(around)address@(the)server.com",
-			[]string{
-				"\"normal name\" <username@server.com>",
-				"<All.address@server.com>",
-			},
-		},
-		{
-			" normal name  <username@server.com>, All.(\"comma, in comment\")address@(the)server.com",
-			[]string{
-				"\"normal name\" <username@server.com>",
-				"<All.address@server.com>",
-			},
-		},
-		{
-			" \"normal name\"  <username@server.com>, \"comma, name\" <address@server.com>",
-			[]string{
-				"\"normal name\" <username@server.com>",
-				"\"comma, name\" <address@server.com>",
-			},
-		},
-		{
-			" \"comma, one\"  <username@server.com>, \"comma, two\" <address@server.com>",
-			[]string{
-				"\"comma, one\" <username@server.com>",
-				"\"comma, two\" <address@server.com>",
-			},
-		},
-		{
-			" \"comma, name\"  <username@server.com>, another, name <address@server.com>",
-			[]string{
-				"\"comma, name\" <username@server.com>",
-				"\"another, name\" <address@server.com>",
-			},
-		},
-	}
-
-	for _, data := range tests {
-		result, err := parseAddressList(data.address)
-		assert.NoError(t, err)
-		assert.Len(t, result, len(data.expected))
-		for i, result := range result {
-			assert.Equal(t, data.expected[i], result.String())
-		}
-	}
-}

pkg/message/rfc5322/Makefile

@@ -0,0 +1,2 @@
+antlr: RFC5322Parser.g4 RFC5322Lexer.g4
+	antlr4 -Dlanguage=Go -o parser $^

pkg/message/rfc5322/README.md

@@ -0,0 +1,160 @@
+# Outline
+The `rfc5322` package implements a parser for `address-list` and `date-time` strings, as defined in RFC5322.
+It also supports encoded words (RFC2047) and has international tokens (RFC6532).
+
+# `rfc5322/parser` directory
+The lexer and parser are generated using ANTLR4.
+The grammar is defined in the g4 files:
+- RFC5322Parser.g4 defines the parser grammar,
+- RFC5322Lexer.g4 defines the lexer grammar.
+
+These grammars are derived from the ABNF grammar provided in the RFCs above, 
+albeit with some relaxations added to support "nonstandard" (bad) input.
+
+Running `antlr4` on these g4 files generates a parser which recognises strings conforming to the grammar:
+- rfc5322_lexer.go
+- rfc5322parser_base_listener.go
+- rfc5322_parser.go
+- rfc5322parser_listener.go
+
+The generated parser can then be used to convert a valid address/date into an abstract syntax tree.
+
+# `rfc5322` directory
+Once we have an abstract syntax tree, we must turn it into something usable, namely a `mail.Address` or `time.Time`.
+
+The generated code in the `rfc5322/parser` directory implements a walker.
+This walker walks over the abstract syntax tree, 
+calling a callback when entering and another when when exiting each node.
+By default, the callbacks are no-ops, unless they are overridden.
+
+## `walker.go`
+The `walker` type extends the base walker, overriding the default no-op callbacks
+to do something specific when entering and exiting certain nodes. 
+
+The goal of the walker is to traverse the syntax tree, picking out relevant information from each node's text.
+For example, when parsing a `mailbox` node, the relevant information to pick out from the parse tree is the
+name and address of the mailbox. This information can appear in a number of different ways, e.g. it might be
+RFC2047 word-encoded, it might be a string with escaped chars that need to be handled, it might have comments
+that should be ignored, and so on.
+
+So while walking the syntax tree, each node needs to ask its children what their "value" is.
+The `mailbox` needs to ask its child nodes (either a `nameAddr` node or an `addrSpec` node)
+what the name and address are.
+If the child node is a `nameAddr`, it needs to ask its `displayName` child what the name is
+and the `angleAddr` what the address is; these in turn ask `word` nodes, `addrSpec` nodes, etc.
+
+Each child node is responsible for telling its parent what its own value is.
+The parent is responsible for assembling the children into something useful.
+
+Ideally, this would be done with the visitor pattern. But unfortunately, the generated parser only
+provides a walker interface. So we need to make use of a stack, pushing on nodes when we enter them
+and popping off nodes when we exit them, to turn the walker into a kind of visitor.
+
+## `parser.go`
+This file implements two methods, 
+`ParseAddressList(string) ([]*mail.Address, error)` 
+and
+`ParseDateTime(string) (time.Time, error)`.
+
+These methods set up a parser from the raw input, start the walker, and convert the walker result
+into an object of the correct type.
+
+
+# Example: Parsing `dateTime`
+Parsing a date-time is rather simple. The implementation begins in `date_time.go`. The abridged code is below:
+
+```
+type dateTime struct {
+	year   int
+	...
+}
+
+func (dt *dateTime) withYear(year *year) {
+	dt.year = year.value
+}
+
+...
+
+func (w *walker) EnterDateTime(ctx *parser.DateTimeContext) {
+	w.enter(&dateTime{
+		loc: time.UTC,
+	})
+}
+
+func (w *walker) ExitDateTime(ctx *parser.DateTimeContext) {
+	dt := w.exit().(*dateTime)
+	w.res = time.Date(dt.year, ...)
+}
+```
+
+As you can see, when the walker reaches a `dateTime` node, it pushes a `dateTime` object onto the stack:
+```
+w.enter(&dateTime{
+	loc: time.UTC,
+})
+```
+
+and when it leaves a `dateTime` node, it pops it off the stack, 
+converting it from `interface{}` to the concrete type,
+and uses the parsed `dateTime` values like day, month, year etc 
+to construct a go `time.Time` object to set the walker result:
+```
+dt := w.exit().(*dateTime)
+w.res = time.Date(dt.year, ...)
+```
+
+These parsed values were discovered while the walker continued to walk across the date-time node.
+
+Let's see how the walker discovers the `year`.
+Here is the abridged code of what happens when the walker enters a `year` node:
+```
+type year struct {
+	value int
+}
+
+func (w *walker) EnterYear(ctx *parser.YearContext) {
+	var text string
+
+	for _, digit := range ctx.AllDigit() {
+		text += digit.GetText()
+	}
+
+	val, err := strconv.Atoi(text)
+	if err != nil {
+		w.err = err
+	}
+
+	w.enter(&year{
+		value: val,
+	})
+}
+```
+
+When entering the `year` node, it collects all the raw digits, which are strings, then
+converts them to an integer, and sets that as the year's integer value while pushing it onto the stack.
+
+When exiting, it pops the year off the stack and gives itself to the parent (now on the top of the stack).
+It doesn't know what type of object the parent is, it just checks to see if anything above it on the stack
+is expecting a `year` node:
+```
+func (w *walker) ExitYear(ctx *parser.YearContext) {
+	type withYear interface {
+		withYear(*year)
+	}
+
+	res := w.exit().(*year)
+
+	if parent, ok := w.parent().(withYear); ok {
+		parent.withYear(res)
+	}
+}
+```
+
+In our case, the `date` is expecting a `year` node because it implements `withYear`,
+```
+func (dt *dateTime) withYear(year *year) {
+	dt.year = year.value
+}
+```
+and that is how the `dateTime` data members are collected.
+

pkg/message/rfc5322/RFC5322Lexer.g4

@@ -0,0 +1,98 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+lexer grammar RFC5322Lexer;
+
+U_00:             '\u0000';
+U_01_08:          '\u0001'..'\u0008';
+TAB:              '\t';     // \u0009
+LF:               '\n';     // \u000A
+U_0B:             '\u000B';
+U_0C:             '\u000C';
+CR:               '\r';     // \u000D
+U_0E_1F:          '\u000E'..'\u001F';
+
+// Printable (0x20-0x7E)
+SP:               ' ';      // \u0020
+Exclamation:      '!';      // \u0021
+DQuote:           '"';      // \u0022
+Hash:             '#';      // \u0023
+Dollar:           '$';      // \u0024
+Percent:          '%';      // \u0025
+Ampersand:        '&';      // \u0026
+SQuote:           '\'';     // \u0027
+LParens:          '(';      // \u0028
+RParens:          ')';      // \u0029
+Asterisk:         '*';      // \u002A
+Plus:             '+';      // \u002B
+Comma:            ',';      // \u002C
+Minus:            '-';      // \u002D
+Period:           '.';      // \u002E
+Slash:            '/';      // \u002F
+Digit:            [0-9];    // \u0030 -- \u0039
+Colon:            ':';      // \u003A
+Semicolon:        ';';      // \u003B
+Less:             '<';      // \u003C
+Equal:            '=';      // \u003D
+Greater:          '>';      // \u003E
+Question:         '?';      // \u003F
+At:               '@';      // \u0040
+// alphaUpper
+LBracket:         '[';      // \u005B
+Backslash:        '\\';     // \u005C
+RBracket:         ']';      // \u005D
+Caret:            '^';      // \u005E
+Underscore:       '_';      // \u005F
+Backtick:         '`';      // \u0060
+// alphaLower
+LCurly:           '{';      // \u007B
+Pipe:             '|';      // \u007C
+RCurly:           '}';      // \u007D
+Tilde:            '~';      // \u007E
+
+// Other
+Delete: '\u007F';
+
+// RFC6532 Extension
+UTF8NonAscii: '\u0080'..'\uFFFF';
+
+A: 'A'|'a';
+B: 'B'|'b';
+C: 'C'|'c';
+D: 'D'|'d';
+E: 'E'|'e';
+F: 'F'|'f';
+G: 'G'|'g';
+H: 'H'|'h';
+I: 'I'|'i';
+J: 'J'|'j';
+K: 'K'|'k';
+L: 'L'|'l';
+M: 'M'|'m';
+N: 'N'|'n';
+O: 'O'|'o';
+P: 'P'|'p';
+Q: 'Q'|'q';
+R: 'R'|'r';
+S: 'S'|'s';
+T: 'T'|'t';
+U: 'U'|'u';
+V: 'V'|'v';
+W: 'W'|'w';
+X: 'X'|'x';
+Y: 'Y'|'y';
+Z: 'Z'|'z';

pkg/message/rfc5322/RFC5322Parser.g4

@@ -0,0 +1,530 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+parser grammar RFC5322Parser;
+
+options { tokenVocab=RFC5322Lexer; }
+
+
+// -------------------
+// 3.2. Lexical tokens
+// -------------------
+
+quotedChar: vchar | wsp;
+
+quotedPair
+	: Backslash quotedChar
+	| obsQP
+	;
+
+fws 
+	: (wsp* crlf)? wsp+
+	| obsFWS
+	;
+
+ctext
+	: alpha
+	| Exclamation
+	| DQuote
+	| Hash
+	| Dollar
+	| Percent
+	| Ampersand
+	| SQuote
+	| Asterisk
+	| Plus
+	| Comma
+	| Minus
+	| Period
+	| Slash
+	| Digit
+	| Colon
+	| Semicolon
+	| Less
+	| Equal
+	| Greater
+	| Question
+	| At
+	| LBracket
+	| RBracket
+	| Caret
+	| Underscore
+	| Backtick
+	| LCurly
+	| Pipe
+	| RCurly
+	| Tilde
+	| obsCtext
+	| UTF8NonAscii
+	;
+
+ccontent
+	: ctext
+	| quotedPair
+	| comment
+	;
+
+comment: LParens (fws? ccontent)* fws? RParens;
+
+cfws
+	: (fws? comment)+ fws?
+	| fws
+	;
+
+atext
+	: alpha
+	| Digit
+	| Exclamation 
+	| Hash
+	| Dollar 
+	| Percent
+	| Ampersand 
+	| SQuote
+	| Asterisk 
+	| Plus
+	| Minus 
+	| Slash
+	| Equal 
+	| Question
+	| Caret 
+	| Underscore
+	| Backtick 
+	| LCurly
+	| Pipe 
+	| RCurly
+	| Tilde
+	| UTF8NonAscii
+	;
+
+atom: atext+;
+
+// Allow dotAtom to have a trailing period; some messages in the wild look like this.
+dotAtom: atext+ (Period atext+)* Period?;
+
+qtext
+	: alpha
+	| Exclamation
+	| Hash
+	| Dollar
+	| Percent
+	| Ampersand
+	| SQuote
+	| LParens
+	| RParens
+	| Asterisk
+	| Plus
+	| Comma
+	| Minus
+	| Period
+	| Slash
+	| Digit
+	| Colon
+	| Semicolon
+	| Less
+	| Equal
+	| Greater
+	| Question
+	| At
+	| LBracket
+	| RBracket
+	| Caret
+	| Underscore
+	| Backtick
+	| LCurly
+	| Pipe
+	| RCurly
+	| Tilde
+	| obsQtext
+	| UTF8NonAscii
+	;
+
+quotedContent
+	: qtext
+	| quotedPair
+	;
+
+quotedValue: (fws? quotedContent)*;
+
+quotedString: DQuote quotedValue fws? DQuote;
+
+// Allow word to consist of the @ token.
+word
+	: cfws? encodedWord cfws?
+	| cfws? atom cfws?
+	| cfws? quotedString cfws?
+	| At
+	;
+
+
+// --------------------------------
+// 3.3. Date and Time Specification
+// --------------------------------
+
+dateTime: (dayOfweek Comma)? day month year hour Colon minute (Colon second)? zone? cfws? EOF;
+
+dayOfweek
+	: fws? dayName 
+	| cfws? dayName cfws?
+	;
+
+dayName
+	: M O N 
+	| T U E 
+	| W E D 
+	| T H U 
+	| F R I 
+	| S A T 
+	| S U N
+	;
+
+day
+	: fws? Digit Digit? fws 
+	| cfws? Digit Digit? cfws?
+	;
+
+month
+	: J A N 
+	| F E B 
+	| M A R 
+	| A P R 
+	| M A Y 
+	| J U N 
+	| J U L 
+	| A U G 
+	| S E P 
+	| O C T 
+	| N O V 
+	| D E C
+	;
+
+year
+	: fws Digit Digit Digit Digit fws 
+	| cfws? Digit Digit cfws?
+	;
+
+// NOTE: RFC5322 requires two digits for the hour, but we 
+// relax that requirement a bit, allowing single digits.
+hour
+	: Digit? Digit 
+	| cfws? Digit? Digit cfws?
+	;
+
+minute
+	: Digit Digit 
+	| cfws? Digit Digit cfws?
+	;
+
+second
+	: Digit Digit 
+	| cfws? Digit Digit cfws?
+	;
+
+offset: (Plus | Minus)? Digit Digit Digit Digit;
+
+zone
+	: fws offset
+	| obsZone
+	;
+
+
+// --------------------------
+// 3.4. Address Specification
+// --------------------------
+
+address
+	: mailbox
+	| group
+	;
+
+mailbox
+	: nameAddr
+	| addrSpec
+	;
+
+nameAddr: displayName? angleAddr;
+
+angleAddr
+	: cfws? Less addrSpec? Greater cfws?
+	| obsAngleAddr
+	;
+
+group: displayName Colon groupList? Semicolon cfws?;
+
+displayName
+	: word+
+	| word (word | Period | cfws)*
+	;
+
+mailboxList
+	: mailbox (Comma mailbox)*
+	| obsMboxList
+	;
+
+addressList
+	: address (Comma address)* EOF
+	| obsAddrList EOF
+	;
+
+groupList
+	: mailboxList
+	| cfws
+	| obsGroupList
+	;
+
+// Allow addrSpec contain a port.
+addrSpec: localPart At domain (Colon port)?;
+
+localPart
+	: cfws? dotAtom cfws?
+	| cfws? quotedString cfws?
+	| obsLocalPart
+	;
+
+port: Digit+;
+
+domain
+	: cfws? dotAtom cfws? 
+	| cfws? domainLiteral cfws?
+	| cfws? obsDomain cfws?
+	;
+
+domainLiteral: LBracket (fws? dtext)* fws? RBracket;
+
+dtext
+	: alpha
+	| Exclamation
+	| DQuote
+	| Hash
+	| Dollar
+	| Percent
+	| Ampersand
+	| SQuote
+	| LParens
+	| RParens
+	| Asterisk
+	| Plus
+	| Comma
+	| Minus
+	| Period
+	| Slash
+	| Digit
+	| Colon
+	| Semicolon
+	| Less
+	| Equal
+	| Greater
+	| Question
+	| At
+	| Caret
+	| Underscore
+	| Backtick
+	| LCurly
+	| Pipe
+	| RCurly
+	| Tilde
+//| obsDtext
+	| UTF8NonAscii
+	;
+
+
+// ----------------------------------
+// 4.1. Miscellaneous Obsolete Tokens
+// ----------------------------------
+
+obsNoWSCTL
+	: U_01_08
+	| U_0B
+	| U_0C
+	| U_0E_1F
+	| Delete
+	;
+
+obsCtext: obsNoWSCTL;
+
+obsQtext: obsNoWSCTL;
+
+obsQP: Backslash (U_00 | obsNoWSCTL | LF | CR);
+
+
+// ---------------------------------
+// 4.2. Obsolete Folding White Space
+// ---------------------------------
+
+obsFWS: wsp+ (crlf wsp+);
+
+
+// ---------------------------
+// 4.3. Obsolete Date and Time
+// ---------------------------
+
+obsZone
+	: U T 
+	| U T C
+	| G M T
+	| E S T
+	| E D T
+	| C S T
+	| C D T
+	| M S T
+	| M D T
+	| P S T
+	| P D T
+//| obsZoneMilitary
+	;
+
+
+// ------------------------
+// 4.4. Obsolete Addressing
+// ------------------------
+
+obsAngleAddr: cfws? Less obsRoute addrSpec Greater cfws?;
+
+obsRoute: obsDomainList Colon;
+
+obsDomainList: (cfws | Comma)* At domain (Comma cfws? (At domain)?)*;
+
+obsMboxList: (cfws? Comma)* mailbox (Comma (mailbox | cfws)?)*;
+
+obsAddrList: (cfws? Comma)* address (Comma (address | cfws)?)*;
+
+obsGroupList: (cfws? Comma)+ cfws?;
+
+obsLocalPart: word (Period word)*;
+
+obsDomain: atom (Period atom)*;
+
+
+// ------------------------------------
+// 2. Syntax of encoded-words (RFC2047)
+// ------------------------------------
+
+encodedWord: Equal Question charset Question encoding Question encodedText Question Equal;
+
+charset: token;
+
+encoding: token;
+
+token: tokenChar+;
+
+tokenChar
+	: alpha
+	| Exclamation
+	| Hash
+	| Dollar
+	| Percent
+	| Ampersand
+	| SQuote
+	| Asterisk
+	| Plus
+	| Minus
+	| Digit
+	| Backslash
+	| Caret
+	| Underscore
+	| Backtick
+	| LCurly
+	| Pipe
+	| RCurly
+	| Tilde
+	;
+
+encodedText: encodedChar+;
+
+encodedChar
+	: alpha
+	| Exclamation
+	| DQuote
+	| Hash
+	| Dollar
+	| Percent
+	| Ampersand
+	| SQuote
+	| LParens
+	| RParens
+	| Asterisk
+	| Plus
+	| Comma
+	| Minus
+	| Period
+	| Slash
+	| Digit
+	| Colon
+	| Semicolon
+	| Less
+	| Equal
+	| Greater
+	| At
+	| LBracket
+	| Backslash
+	| RBracket
+	| Caret
+	| Underscore
+	| Backtick
+	| LCurly
+	| Pipe
+	| RCurly
+	| Tilde
+	;
+
+
+// -------------------------
+// B.1. Core Rules (RFC5234)
+// -------------------------
+
+crlf: CR LF;
+
+wsp: SP | TAB;
+
+vchar
+	: alpha
+	| Exclamation
+	| DQuote
+	| Hash
+	| Dollar
+	| Percent
+	| Ampersand
+	| SQuote
+	| LParens
+	| RParens
+	| Asterisk
+	| Plus
+	| Comma
+	| Minus
+	| Period
+	| Slash
+	| Digit
+	| Colon
+	| Semicolon
+	| Less
+	| Equal
+	| Greater
+	| Question
+	| At
+	| LBracket
+	| Backslash
+	| RBracket
+	| Caret
+	| Underscore
+	| Backtick
+	| LCurly
+	| Pipe
+	| RCurly
+	| Tilde
+	| UTF8NonAscii
+	;
+
+alpha: A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z ;

pkg/message/rfc5322/addr_spec.go

@@ -0,0 +1,58 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type addrSpec struct {
+	localPart, domain string
+}
+
+func (a *addrSpec) withLocalPart(localPart *localPart) {
+	a.localPart = localPart.value
+}
+
+func (a *addrSpec) withDomain(domain *domain) {
+	a.domain = domain.value
+}
+
+func (a *addrSpec) withPort(port *port) {
+	a.domain += ":" + port.value
+}
+
+func (w *walker) EnterAddrSpec(ctx *parser.AddrSpecContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering addrSpec")
+	w.enter(&addrSpec{})
+}
+
+func (w *walker) ExitAddrSpec(ctx *parser.AddrSpecContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting addrSpec")
+
+	type withAddrSpec interface {
+		withAddrSpec(*addrSpec)
+	}
+
+	res := w.exit().(*addrSpec)
+
+	if parent, ok := w.parent().(withAddrSpec); ok {
+		parent.withAddrSpec(res)
+	}
+}

pkg/message/rfc5322/address.go

@@ -0,0 +1,59 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"net/mail"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type address struct {
+	addresses []*mail.Address
+}
+
+func (a *address) withMailbox(mailbox *mailbox) {
+	a.addresses = append(a.addresses, &mail.Address{
+		Name:    mailbox.name,
+		Address: mailbox.address,
+	})
+}
+
+func (a *address) withGroup(group *group) {
+	a.addresses = append(a.addresses, group.addresses...)
+}
+
+func (w *walker) EnterAddress(ctx *parser.AddressContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering address")
+	w.enter(&address{})
+}
+
+func (w *walker) ExitAddress(ctx *parser.AddressContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting address")
+
+	type withAddress interface {
+		withAddress(*address)
+	}
+
+	res := w.exit().(*address)
+
+	if parent, ok := w.parent().(withAddress); ok {
+		parent.withAddress(res)
+	}
+}

pkg/message/rfc5322/address_list.go

@@ -0,0 +1,43 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"net/mail"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type addressList struct {
+	addresses []*mail.Address
+}
+
+func (a *addressList) withAddress(address *address) {
+	a.addresses = append(a.addresses, address.addresses...)
+}
+
+func (w *walker) EnterAddressList(ctx *parser.AddressListContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering addressList")
+	w.enter(&addressList{})
+}
+
+func (w *walker) ExitAddressList(ctx *parser.AddressListContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting addressList")
+	w.res = w.exit().(*addressList).addresses
+}

pkg/message/rfc5322/angle_addr.go

@@ -0,0 +1,56 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"fmt"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type angleAddr struct {
+	address string
+}
+
+func (a *angleAddr) withAddrSpec(addrSpec *addrSpec) {
+	a.address = fmt.Sprintf("%v@%v", addrSpec.localPart, addrSpec.domain)
+}
+
+func (a *angleAddr) withObsAngleAddr(obsAngleAddr *obsAngleAddr) {
+	a.address = obsAngleAddr.address
+}
+
+func (w *walker) EnterAngleAddr(ctx *parser.AngleAddrContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering angleAddr")
+	w.enter(&angleAddr{})
+}
+
+func (w *walker) ExitAngleAddr(ctx *parser.AngleAddrContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting angleAddr")
+
+	type withAngleAddr interface {
+		withAngleAddr(*angleAddr)
+	}
+
+	res := w.exit().(*angleAddr)
+
+	if parent, ok := w.parent().(withAngleAddr); ok {
+		parent.withAngleAddr(res)
+	}
+}

pkg/message/rfc5322/atom.go

@@ -0,0 +1,49 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type atom struct {
+	value string
+}
+
+func (w *walker) EnterAtom(ctx *parser.AtomContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering atom")
+
+	w.enter(&atom{
+		value: ctx.GetText(),
+	})
+}
+
+func (w *walker) ExitAtom(ctx *parser.AtomContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting atom")
+
+	type withAtom interface {
+		withAtom(*atom)
+	}
+
+	res := w.exit().(*atom)
+
+	if parent, ok := w.parent().(withAtom); ok {
+		parent.withAtom(res)
+	}
+}

pkg/message/rfc5322/date_time.go

@@ -0,0 +1,79 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"time"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type dateTime struct {
+	day   int
+	month time.Month
+	year  int
+
+	hour, min, sec int
+
+	loc *time.Location
+}
+
+func (dt *dateTime) withDay(day *day) {
+	dt.day = day.value
+}
+
+func (dt *dateTime) withMonth(month *month) {
+	dt.month = month.value
+}
+
+func (dt *dateTime) withYear(year *year) {
+	dt.year = year.value
+}
+
+func (dt *dateTime) withHour(hour *hour) {
+	dt.hour = hour.value
+}
+
+func (dt *dateTime) withMinute(minute *minute) {
+	dt.min = minute.value
+}
+
+func (dt *dateTime) withSecond(second *second) {
+	dt.sec = second.value
+}
+
+func (dt *dateTime) withZone(zone *zone) {
+	dt.loc = zone.location
+}
+
+func (w *walker) EnterDateTime(ctx *parser.DateTimeContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering dateTime")
+
+	w.enter(&dateTime{
+		loc: time.UTC,
+	})
+}
+
+func (w *walker) ExitDateTime(ctx *parser.DateTimeContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting dateTime")
+
+	dt := w.exit().(*dateTime)
+
+	w.res = time.Date(dt.year, dt.month, dt.day, dt.hour, dt.min, dt.sec, 0, dt.loc)
+}

pkg/message/rfc5322/day.go

@@ -0,0 +1,62 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"strconv"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type day struct {
+	value int
+}
+
+func (w *walker) EnterDay(ctx *parser.DayContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering day")
+
+	var text string
+
+	for _, digit := range ctx.AllDigit() {
+		text += digit.GetText()
+	}
+
+	val, err := strconv.Atoi(text)
+	if err != nil {
+		w.err = err
+	}
+
+	w.enter(&day{
+		value: val,
+	})
+}
+
+func (w *walker) ExitDay(ctx *parser.DayContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting day")
+
+	type withDay interface {
+		withDay(*day)
+	}
+
+	res := w.exit().(*day)
+
+	if parent, ok := w.parent().(withDay); ok {
+		parent.withDay(res)
+	}
+}

pkg/message/rfc5322/display_name.go

@@ -0,0 +1,50 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type displayName struct {
+	words []string
+}
+
+func (n *displayName) withWord(word *word) {
+	n.words = append(n.words, word.value)
+}
+
+func (w *walker) EnterDisplayName(ctx *parser.DisplayNameContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering displayName")
+	w.enter(&displayName{})
+}
+
+func (w *walker) ExitDisplayName(ctx *parser.DisplayNameContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting displayName")
+
+	type withDisplayName interface {
+		withDisplayName(*displayName)
+	}
+
+	res := w.exit().(*displayName)
+
+	if parent, ok := w.parent().(withDisplayName); ok {
+		parent.withDisplayName(res)
+	}
+}

pkg/message/rfc5322/domain.go

@@ -0,0 +1,60 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"strings"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type domain struct {
+	value string
+}
+
+func (d *domain) withDotAtom(dotAtom *dotAtom) {
+	d.value = dotAtom.value
+}
+
+func (d *domain) withDomainLiteral(domainLiteral *domainLiteral) {
+	d.value = domainLiteral.value
+}
+
+func (d *domain) withObsDomain(obsDomain *obsDomain) {
+	d.value = strings.Join(obsDomain.atoms, ".")
+}
+
+func (w *walker) EnterDomain(ctx *parser.DomainContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering domain")
+	w.enter(&domain{})
+}
+
+func (w *walker) ExitDomain(ctx *parser.DomainContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting domain")
+
+	type withDomain interface {
+		withDomain(*domain)
+	}
+
+	res := w.exit().(*domain)
+
+	if parent, ok := w.parent().(withDomain); ok {
+		parent.withDomain(res)
+	}
+}

pkg/message/rfc5322/domain_literal.go

@@ -0,0 +1,49 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type domainLiteral struct {
+	value string
+}
+
+func (w *walker) EnterDomainLiteral(ctx *parser.DomainLiteralContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering domainLiteral")
+
+	w.enter(&domainLiteral{
+		value: ctx.GetText(),
+	})
+}
+
+func (w *walker) ExitDomainLiteral(ctx *parser.DomainLiteralContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting domainLiteral")
+
+	type withDomainLiteral interface {
+		withDomainLiteral(*domainLiteral)
+	}
+
+	res := w.exit().(*domainLiteral)
+
+	if parent, ok := w.parent().(withDomainLiteral); ok {
+		parent.withDomainLiteral(res)
+	}
+}

pkg/message/rfc5322/dot_atom.go

@@ -0,0 +1,49 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type dotAtom struct {
+	value string
+}
+
+func (w *walker) EnterDotAtom(ctx *parser.DotAtomContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering dotAtom")
+
+	w.enter(&dotAtom{
+		value: ctx.GetText(),
+	})
+}
+
+func (w *walker) ExitDotAtom(ctx *parser.DotAtomContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting dotAtom")
+
+	type withDotAtom interface {
+		withDotAtom(*dotAtom)
+	}
+
+	res := w.exit().(*dotAtom)
+
+	if parent, ok := w.parent().(withDotAtom); ok {
+		parent.withDotAtom(res)
+	}
+}

pkg/message/rfc5322/encoded_word.go

@@ -0,0 +1,55 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	pmmime "github.com/ProtonMail/proton-bridge/pkg/mime"
+	"github.com/sirupsen/logrus"
+)
+
+type encodedWord struct {
+	value string
+}
+
+func (w *walker) EnterEncodedWord(ctx *parser.EncodedWordContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering encodedWord")
+
+	word, err := pmmime.WordDec.Decode(ctx.GetText())
+	if err != nil {
+		word = ctx.GetText()
+	}
+
+	w.enter(&encodedWord{
+		value: word,
+	})
+}
+
+func (w *walker) ExitEncodedWord(ctx *parser.EncodedWordContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting encodedWord")
+
+	type withEncodedWord interface {
+		withEncodedWord(*encodedWord)
+	}
+
+	res := w.exit().(*encodedWord)
+
+	if parent, ok := w.parent().(withEncodedWord); ok {
+		parent.withEncodedWord(res)
+	}
+}

pkg/message/rfc5322/fws.go

@@ -0,0 +1,49 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type fws struct {
+	value string
+}
+
+func (w *walker) EnterFws(ctx *parser.FwsContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering fws")
+
+	w.enter(&fws{
+		value: ctx.GetText(),
+	})
+}
+
+func (w *walker) ExitFws(ctx *parser.FwsContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting fws")
+
+	type withFws interface {
+		withFws(*fws)
+	}
+
+	res := w.exit().(*fws)
+
+	if parent, ok := w.parent().(withFws); ok {
+		parent.withFws(res)
+	}
+}

pkg/message/rfc5322/gen.go

@@ -0,0 +1,20 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+//go:generate make antlr

pkg/message/rfc5322/group.go

@@ -0,0 +1,52 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"net/mail"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type group struct {
+	addresses []*mail.Address
+}
+
+func (g *group) withGroupList(groupList *groupList) {
+	g.addresses = append(g.addresses, groupList.addresses...)
+}
+
+func (w *walker) EnterGroup(ctx *parser.GroupContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering group")
+	w.enter(&group{})
+}
+
+func (w *walker) ExitGroup(ctx *parser.GroupContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting group")
+
+	type withGroup interface {
+		withGroup(*group)
+	}
+
+	res := w.exit().(*group)
+
+	if parent, ok := w.parent().(withGroup); ok {
+		parent.withGroup(res)
+	}
+}

pkg/message/rfc5322/group_list.go

@@ -0,0 +1,52 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"net/mail"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type groupList struct {
+	addresses []*mail.Address
+}
+
+func (gl *groupList) withMailboxList(mailboxList *mailboxList) {
+	gl.addresses = append(gl.addresses, mailboxList.addresses...)
+}
+
+func (w *walker) EnterGroupList(ctx *parser.GroupListContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering groupList")
+	w.enter(&groupList{})
+}
+
+func (w *walker) ExitGroupList(ctx *parser.GroupListContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting groupList")
+
+	type withGroupList interface {
+		withGroupList(*groupList)
+	}
+
+	res := w.exit().(*groupList)
+
+	if parent, ok := w.parent().(withGroupList); ok {
+		parent.withGroupList(res)
+	}
+}

pkg/message/rfc5322/hour.go

@@ -0,0 +1,62 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"strconv"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type hour struct {
+	value int
+}
+
+func (w *walker) EnterHour(ctx *parser.HourContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering hour")
+
+	var text string
+
+	for _, digit := range ctx.AllDigit() {
+		text += digit.GetText()
+	}
+
+	val, err := strconv.Atoi(text)
+	if err != nil {
+		w.err = err
+	}
+
+	w.enter(&hour{
+		value: val,
+	})
+}
+
+func (w *walker) ExitHour(ctx *parser.HourContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting hour")
+
+	type withHour interface {
+		withHour(*hour)
+	}
+
+	res := w.exit().(*hour)
+
+	if parent, ok := w.parent().(withHour); ok {
+		parent.withHour(res)
+	}
+}

pkg/message/rfc5322/local_part.go

@@ -0,0 +1,60 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"strings"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type localPart struct {
+	value string
+}
+
+func (p *localPart) withDotAtom(dotAtom *dotAtom) {
+	p.value = dotAtom.value
+}
+
+func (p *localPart) withQuotedString(quotedString *quotedString) {
+	p.value = quotedString.value
+}
+
+func (p *localPart) withObsLocalPart(obsLocalPart *obsLocalPart) {
+	p.value = strings.Join(obsLocalPart.words, ".")
+}
+
+func (w *walker) EnterLocalPart(ctx *parser.LocalPartContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering localPart")
+	w.enter(&localPart{})
+}
+
+func (w *walker) ExitLocalPart(ctx *parser.LocalPartContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting localPart")
+
+	type withLocalPart interface {
+		withLocalPart(*localPart)
+	}
+
+	res := w.exit().(*localPart)
+
+	if parent, ok := w.parent().(withLocalPart); ok {
+		parent.withLocalPart(res)
+	}
+}

pkg/message/rfc5322/mailbox.go

@@ -0,0 +1,57 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"fmt"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type mailbox struct {
+	name, address string
+}
+
+func (m *mailbox) withNameAddr(nameAddr *nameAddr) {
+	m.name = nameAddr.name
+	m.address = nameAddr.address
+}
+
+func (m *mailbox) withAddrSpec(addrSpec *addrSpec) {
+	m.address = fmt.Sprintf("%v@%v", addrSpec.localPart, addrSpec.domain)
+}
+
+func (w *walker) EnterMailbox(ctx *parser.MailboxContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering mailbox")
+	w.enter(&mailbox{})
+}
+
+func (w *walker) ExitMailbox(ctx *parser.MailboxContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting mailbox")
+
+	type withMailbox interface {
+		withMailbox(*mailbox)
+	}
+
+	res := w.exit().(*mailbox)
+
+	if parent, ok := w.parent().(withMailbox); ok {
+		parent.withMailbox(res)
+	}
+}

pkg/message/rfc5322/mailbox_list.go

@@ -0,0 +1,59 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"net/mail"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type mailboxList struct {
+	addresses []*mail.Address
+}
+
+func (ml *mailboxList) withMailbox(mailbox *mailbox) {
+	ml.addresses = append(ml.addresses, &mail.Address{
+		Name:    mailbox.name,
+		Address: mailbox.address,
+	})
+}
+
+func (ml *mailboxList) withObsMboxList(obsMboxList *obsMboxList) {
+	ml.addresses = append(ml.addresses, obsMboxList.addresses...)
+}
+
+func (w *walker) EnterMailboxList(ctx *parser.MailboxListContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering mailboxList")
+	w.enter(&mailboxList{})
+}
+
+func (w *walker) ExitMailboxList(ctx *parser.MailboxListContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting mailboxList")
+
+	type withMailboxList interface {
+		withMailboxList(*mailboxList)
+	}
+
+	res := w.exit().(*mailboxList)
+
+	if parent, ok := w.parent().(withMailboxList); ok {
+		parent.withMailboxList(res)
+	}
+}

pkg/message/rfc5322/minute.go

@@ -0,0 +1,62 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"strconv"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type minute struct {
+	value int
+}
+
+func (w *walker) EnterMinute(ctx *parser.MinuteContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering minute")
+
+	var text string
+
+	for _, digit := range ctx.AllDigit() {
+		text += digit.GetText()
+	}
+
+	val, err := strconv.Atoi(text)
+	if err != nil {
+		w.err = err
+	}
+
+	w.enter(&minute{
+		value: val,
+	})
+}
+
+func (w *walker) ExitMinute(ctx *parser.MinuteContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting minute")
+
+	type withMinute interface {
+		withMinute(*minute)
+	}
+
+	res := w.exit().(*minute)
+
+	if parent, ok := w.parent().(withMinute); ok {
+		parent.withMinute(res)
+	}
+}

pkg/message/rfc5322/month.go

@@ -0,0 +1,84 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"errors"
+	"strings"
+	"time"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type month struct {
+	value time.Month
+}
+
+func (w *walker) EnterMonth(ctx *parser.MonthContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering month")
+
+	var m time.Month
+
+	switch strings.ToLower(ctx.GetText()) {
+	case "jan":
+		m = time.January
+	case "feb":
+		m = time.February
+	case "mar":
+		m = time.March
+	case "apr":
+		m = time.April
+	case "may":
+		m = time.May
+	case "jun":
+		m = time.June
+	case "jul":
+		m = time.July
+	case "aug":
+		m = time.August
+	case "sep":
+		m = time.September
+	case "oct":
+		m = time.October
+	case "nov":
+		m = time.November
+	case "dec":
+		m = time.December
+	default:
+		w.err = errors.New("no such month")
+	}
+
+	w.enter(&month{
+		value: m,
+	})
+}
+
+func (w *walker) ExitMonth(ctx *parser.MonthContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting month")
+
+	type withMonth interface {
+		withMonth(*month)
+	}
+
+	res := w.exit().(*month)
+
+	if parent, ok := w.parent().(withMonth); ok {
+		parent.withMonth(res)
+	}
+}

pkg/message/rfc5322/name_addr.go

@@ -0,0 +1,56 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"strings"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type nameAddr struct {
+	name, address string
+}
+
+func (a *nameAddr) withDisplayName(displayName *displayName) {
+	a.name = strings.Join(displayName.words, " ")
+}
+
+func (a *nameAddr) withAngleAddr(angleAddr *angleAddr) {
+	a.address = angleAddr.address
+}
+
+func (w *walker) EnterNameAddr(ctx *parser.NameAddrContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering nameAddr")
+	w.enter(&nameAddr{})
+}
+
+func (w *walker) ExitNameAddr(ctx *parser.NameAddrContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting nameAddr")
+
+	type withNameAddr interface {
+		withNameAddr(*nameAddr)
+	}
+
+	res := w.exit().(*nameAddr)
+
+	if parent, ok := w.parent().(withNameAddr); ok {
+		parent.withNameAddr(res)
+	}
+}

pkg/message/rfc5322/obs_angle_addr.go

@@ -0,0 +1,54 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"fmt"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+// When interpreting addresses, the route portion SHOULD be ignored.
+
+type obsAngleAddr struct {
+	address string
+}
+
+func (a *obsAngleAddr) withAddrSpec(addrSpec *addrSpec) {
+	a.address = fmt.Sprintf("%v@%v", addrSpec.localPart, addrSpec.domain)
+}
+
+func (w *walker) EnterObsAngleAddr(ctx *parser.ObsAngleAddrContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering obsAngleAddr")
+	w.enter(&obsAngleAddr{})
+}
+
+func (w *walker) ExitObsAngleAddr(ctx *parser.ObsAngleAddrContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting obsAngleAddr")
+
+	type withObsAngleAddr interface {
+		withObsAngleAddr(*obsAngleAddr)
+	}
+
+	res := w.exit().(*obsAngleAddr)
+
+	if parent, ok := w.parent().(withObsAngleAddr); ok {
+		parent.withObsAngleAddr(res)
+	}
+}

pkg/message/rfc5322/obs_domain.go

@@ -0,0 +1,50 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type obsDomain struct {
+	atoms []string
+}
+
+func (p *obsDomain) withAtom(atom *atom) {
+	p.atoms = append(p.atoms, atom.value)
+}
+
+func (w *walker) EnterObsDomain(ctx *parser.ObsDomainContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering obsDomain")
+	w.enter(&obsDomain{})
+}
+
+func (w *walker) ExitObsDomain(ctx *parser.ObsDomainContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting obsDomain")
+
+	type withObsDomain interface {
+		withObsDomain(*obsDomain)
+	}
+
+	res := w.exit().(*obsDomain)
+
+	if parent, ok := w.parent().(withObsDomain); ok {
+		parent.withObsDomain(res)
+	}
+}

pkg/message/rfc5322/obs_local_part.go

@@ -0,0 +1,50 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type obsLocalPart struct {
+	words []string
+}
+
+func (p *obsLocalPart) withWord(word *word) {
+	p.words = append(p.words, word.value)
+}
+
+func (w *walker) EnterObsLocalPart(ctx *parser.ObsLocalPartContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering obsLocalPart")
+	w.enter(&obsLocalPart{})
+}
+
+func (w *walker) ExitObsLocalPart(ctx *parser.ObsLocalPartContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting obsLocalPart")
+
+	type withObsLocalPart interface {
+		withObsLocalPart(*obsLocalPart)
+	}
+
+	res := w.exit().(*obsLocalPart)
+
+	if parent, ok := w.parent().(withObsLocalPart); ok {
+		parent.withObsLocalPart(res)
+	}
+}

pkg/message/rfc5322/obs_mbox_list.go

@@ -0,0 +1,55 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"net/mail"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type obsMboxList struct {
+	addresses []*mail.Address
+}
+
+func (ml *obsMboxList) withMailbox(mailbox *mailbox) {
+	ml.addresses = append(ml.addresses, &mail.Address{
+		Name:    mailbox.name,
+		Address: mailbox.address,
+	})
+}
+
+func (w *walker) EnterObsMboxList(ctx *parser.ObsMboxListContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering obsMboxList")
+	w.enter(&obsMboxList{})
+}
+
+func (w *walker) ExitObsMboxList(ctx *parser.ObsMboxListContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting obsMboxList")
+
+	type withObsMboxList interface {
+		withObsMboxList(*obsMboxList)
+	}
+
+	res := w.exit().(*obsMboxList)
+
+	if parent, ok := w.parent().(withObsMboxList); ok {
+		parent.withObsMboxList(res)
+	}
+}

pkg/message/rfc5322/obs_zone.go

@@ -0,0 +1,82 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"errors"
+	"strings"
+	"time"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type obsZone struct {
+	location *time.Location
+}
+
+func (w *walker) EnterObsZone(ctx *parser.ObsZoneContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering obsZone")
+
+	loc := time.UTC
+
+	switch strings.ToLower(ctx.GetText()) {
+	case "ut":
+		loc = time.FixedZone(ctx.GetText(), 0)
+	case "utc":
+		loc = time.FixedZone(ctx.GetText(), 0)
+	case "gmt":
+		loc = time.FixedZone(ctx.GetText(), 0)
+	case "est":
+		loc = time.FixedZone(ctx.GetText(), -5*60*60)
+	case "edt":
+		loc = time.FixedZone(ctx.GetText(), -4*60*60)
+	case "cst":
+		loc = time.FixedZone(ctx.GetText(), -6*60*60)
+	case "cdt":
+		loc = time.FixedZone(ctx.GetText(), -5*60*60)
+	case "mst":
+		loc = time.FixedZone(ctx.GetText(), -7*60*60)
+	case "mdt":
+		loc = time.FixedZone(ctx.GetText(), -6*60*60)
+	case "pst":
+		loc = time.FixedZone(ctx.GetText(), -8*60*60)
+	case "pdt":
+		loc = time.FixedZone(ctx.GetText(), -7*60*60)
+	default:
+		w.err = errors.New("bad timezone")
+	}
+
+	w.enter(&obsZone{
+		location: loc,
+	})
+}
+
+func (w *walker) ExitObsZone(ctx *parser.ObsZoneContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting obsZone")
+
+	type withObsZone interface {
+		withObsZone(*obsZone)
+	}
+
+	res := w.exit().(*obsZone)
+
+	if parent, ok := w.parent().(withObsZone); ok {
+		parent.withObsZone(res)
+	}
+}

pkg/message/rfc5322/offset.go

@@ -0,0 +1,73 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"fmt"
+	"strings"
+	"time"
+
+	"github.com/ProtonMail/proton-bridge/pkg/message/rfc5322/parser"
+	"github.com/sirupsen/logrus"
+)
+
+type offset struct {
+	rep   string
+	value int
+}
+
+func (w *walker) EnterOffset(ctx *parser.OffsetContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Entering offset")
+
+	text := ctx.GetText()
+
+	// NOTE: RFC5322 date-time should always begin with + or -
+	// but we relax that requirement a bit due to many messages
+	// in the wild that skip the +; we add the "+" if missing.
+	if !strings.HasPrefix(text, "+") && !strings.HasPrefix(text, "-") {
+		text = "+" + text
+	}
+
+	sgn := text[0:1]
+	hrs := text[1:3]
+	min := text[3:5]
+
+	dur, err := time.ParseDuration(fmt.Sprintf("%v%vh%vm", sgn, hrs, min))
+	if err != nil {
+		w.err = err
+	}
+
+	w.enter(&offset{
+		rep:   text,
+		value: int(dur.Seconds()),
+	})
+}
+
+func (w *walker) ExitOffset(ctx *parser.OffsetContext) {
+	logrus.WithField("text", ctx.GetText()).Trace("Exiting offset")
+
+	type withOffset interface {
+		withOffset(*offset)
+	}
+
+	res := w.exit().(*offset)
+
+	if parent, ok := w.parent().(withOffset); ok {
+		parent.withOffset(res)
+	}
+}

pkg/message/rfc5322/parse_address_list_test.go

@@ -0,0 +1,701 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"encoding/xml"
+	"io"
+	"net/mail"
+	"os"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+func TestParseSingleAddress(t *testing.T) {
+	tests := []struct {
+		input string
+		addrs []*mail.Address
+	}{
+		{
+			input: `user@example.com`,
+			addrs: []*mail.Address{{
+				Address: `user@example.com`,
+			}},
+		},
+		{
+			input: `John Doe <jdoe@machine.example>`,
+			addrs: []*mail.Address{{
+				Name:    `John Doe`,
+				Address: `jdoe@machine.example`,
+			}},
+		},
+		{
+			input: `Mary Smith <mary@example.net>`,
+			addrs: []*mail.Address{{
+				Name:    `Mary Smith`,
+				Address: `mary@example.net`,
+			}},
+		},
+		{
+			input: `"Joe Q. Public" <john.q.public@example.com>`,
+			addrs: []*mail.Address{{
+				Name:    `Joe Q. Public`,
+				Address: `john.q.public@example.com`,
+			}},
+		},
+		{
+			input: `Mary Smith <mary@x.test>`,
+			addrs: []*mail.Address{{
+				Name:    `Mary Smith`,
+				Address: `mary@x.test`,
+			}},
+		},
+		{
+			input: `jdoe@example.org`,
+			addrs: []*mail.Address{{
+				Address: `jdoe@example.org`,
+			}},
+		},
+		{
+			input: `Who? <one@y.test>`,
+			addrs: []*mail.Address{{
+				Name:    `Who?`,
+				Address: `one@y.test`,
+			}},
+		},
+		{
+			input: `<boss@nil.test>`,
+			addrs: []*mail.Address{{
+				Address: `boss@nil.test`,
+			}},
+		},
+		{
+			input: `"Giant; \"Big\" Box" <sysservices@example.net>`,
+			addrs: []*mail.Address{{
+				Name:    `Giant; "Big" Box`,
+				Address: `sysservices@example.net`,
+			}},
+		},
+		{
+			input: `Pete <pete@silly.example>`,
+			addrs: []*mail.Address{{
+				Name:    `Pete`,
+				Address: `pete@silly.example`,
+			}},
+		},
+		{
+			input: `"Mary Smith: Personal Account" <smith@home.example>`,
+			addrs: []*mail.Address{{
+				Name:    `Mary Smith: Personal Account`,
+				Address: `smith@home.example`,
+			}},
+		},
+		{
+			input: `Pete(A nice \) chap) <pete(his account)@silly.test(his host)>`,
+			addrs: []*mail.Address{{
+				Name:    `Pete`,
+				Address: `pete@silly.test`,
+			}},
+		},
+		{
+			input: `Gogh Fir <gf@example.com>`,
+			addrs: []*mail.Address{{
+				Name:    `Gogh Fir`,
+				Address: `gf@example.com`,
+			}},
+		},
+		{
+			input: `normal name  <username@server.com>`,
+			addrs: []*mail.Address{{
+				Name:    `normal name`,
+				Address: `username@server.com`,
+			}},
+		},
+		{
+			input: `"comma, name"  <username@server.com>`,
+			addrs: []*mail.Address{{
+				Name:    `comma, name`,
+				Address: `username@server.com`,
+			}},
+		},
+		{
+			input: `name  <username@server.com> (ignore comment)`,
+			addrs: []*mail.Address{{
+				Name:    `name`,
+				Address: `username@server.com`,
+			}},
+		},
+		{
+			input: `"Mail Robot" <>`,
+			addrs: []*mail.Address{{
+				Name: `Mail Robot`,
+			}},
+		},
+		{
+			input: `Michal Hořejšek <hořejšek@mail.com>`,
+			addrs: []*mail.Address{{
+				Name:    `Michal Hořejšek`,
+				Address: `hořejšek@mail.com`, // Not his real address.
+			}},
+		},
+		{
+			input: `First Last <user@domain.com >`,
+			addrs: []*mail.Address{{
+				Name:    `First Last`,
+				Address: `user@domain.com`,
+			}},
+		},
+		{
+			input: `First Last <user@domain.com. >`,
+			addrs: []*mail.Address{{
+				Name:    `First Last`,
+				Address: `user@domain.com.`,
+			}},
+		},
+		{
+			input: `First Last <user@domain.com.>`,
+			addrs: []*mail.Address{{
+				Name:    `First Last`,
+				Address: `user@domain.com.`,
+			}},
+		},
+		{
+			input: `First Last <user@domain.com:25>`,
+			addrs: []*mail.Address{{
+				Name:    `First Last`,
+				Address: `user@domain.com:25`,
+			}},
+		},
+		{
+			input: `First Last <user@[10.0.0.1]>`,
+			addrs: []*mail.Address{{
+				Name:    `First Last`,
+				Address: `user@[10.0.0.1]`,
+			}},
+		},
+		{
+			input: `<postmaster@[10.10.10.10]>`,
+			addrs: []*mail.Address{
+				{
+					Address: `postmaster@[10.10.10.10]`,
+				},
+			},
+		},
+		{
+			input: `user@domain <user@domain.com>`,
+			addrs: []*mail.Address{{
+				// Name:    `user@domain`,
+				Name:    `user @ domain`,
+				Address: `user@domain.com`,
+			}},
+		},
+		{
+			input: `First Last < user@domain.com>`,
+			addrs: []*mail.Address{{
+				Name:    `First Last`,
+				Address: `user@domain.com`,
+			}},
+		},
+		{
+			input: `First Middle @ Last <user@domain.com>`,
+			addrs: []*mail.Address{{
+				Name:    `First Middle @ Last`,
+				Address: `user@domain.com`,
+			}},
+		},
+		{
+			input: `user@domain.com,`,
+			addrs: []*mail.Address{
+				{
+					Address: `user@domain.com`,
+				},
+			},
+		},
+		{
+			input: `First Middle "Last" <user@domain.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `First Middle Last`,
+					Address: `user@domain.com`,
+				},
+			},
+		},
+		{
+			input: `First Middle Last <user@domain.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `First Middle Last`,
+					Address: `user@domain.com`,
+				},
+			},
+		},
+		{
+			input: `First Middle"Last" <user@domain.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `First Middle Last`,
+					Address: `user@domain.com`,
+				},
+			},
+		},
+		{
+			input: `First Middle "Last"<user@domain.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `First Middle Last`,
+					Address: `user@domain.com`,
+				},
+			},
+		},
+		{
+			input: `First "Middle" "Last" <user@domain.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `First Middle Last`,
+					Address: `user@domain.com`,
+				},
+			},
+		},
+		{
+			input: `First "Middle""Last" <user@domain.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `First Middle Last`,
+					Address: `user@domain.com`,
+				},
+			},
+		},
+	}
+	for _, test := range tests {
+		test := test
+
+		t.Run(test.input, func(t *testing.T) {
+			addrs, err := ParseAddressList(test.input)
+			assert.NoError(t, err)
+			assert.ElementsMatch(t, test.addrs, addrs)
+		})
+	}
+}
+
+func TestParseSingleAddressEncodedWord(t *testing.T) {
+	tests := []struct {
+		input string
+		addrs []*mail.Address
+	}{
+		{
+			input: `=?US-ASCII?Q?Keith_Moore?= <moore@cs.utk.edu>`,
+			addrs: []*mail.Address{{
+				Name:    `Keith Moore`,
+				Address: `moore@cs.utk.edu`,
+			}},
+		},
+		{
+			input: `=?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <keld@dkuug.dk>`,
+			addrs: []*mail.Address{{
+				Name:    `Keld Jørn Simonsen`,
+				Address: `keld@dkuug.dk`,
+			}},
+		},
+		{
+			input: `=?ISO-8859-1?Q?Andr=E9?= Pirard <PIRARD@vm1.ulg.ac.be>`,
+			addrs: []*mail.Address{{
+				Name:    `André Pirard`,
+				Address: `PIRARD@vm1.ulg.ac.be`,
+			}},
+		},
+		{
+			input: `=?ISO-8859-1?Q?Olle_J=E4rnefors?= <ojarnef@admin.kth.se>`,
+			addrs: []*mail.Address{{
+				Name:    `Olle Järnefors`,
+				Address: `ojarnef@admin.kth.se`,
+			}},
+		},
+		{
+			input: `=?ISO-8859-1?Q?Patrik_F=E4ltstr=F6m?= <paf@nada.kth.se>`,
+			addrs: []*mail.Address{{
+				Name:    `Patrik Fältström`,
+				Address: `paf@nada.kth.se`,
+			}},
+		},
+		{
+			input: `Nathaniel Borenstein <nsb@thumper.bellcore.com> (=?iso-8859-8?b?7eXs+SDv4SDp7Oj08A==?=)`,
+			addrs: []*mail.Address{{
+				Name:    `Nathaniel Borenstein`,
+				Address: `nsb@thumper.bellcore.com`,
+			}},
+		},
+		{
+			input: `=?UTF-8?B?PEJlemUgam3DqW5hPg==?= <user@domain.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `<Beze jména>`,
+					Address: `user@domain.com`,
+				},
+			},
+		},
+		{
+			input: `First Middle =?utf-8?Q?Last?= <user@domain.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `First Middle Last`,
+					Address: `user@domain.com`,
+				},
+			},
+		},
+		/*
+			{
+				input: `First Middle=?utf-8?Q?Last?= <user@domain.com>`,
+				addrs: []*mail.Address{
+					{
+						Name:    `First Middle Last`,
+						Address: `user@domain.com`,
+					},
+				},
+			},
+		*/
+		{
+			input: `First Middle =?utf-8?Q?Last?=<user@domain.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `First Middle Last`,
+					Address: `user@domain.com`,
+				},
+			},
+		},
+		{
+			input: `First =?utf-8?Q?Middle?= =?utf-8?Q?Last?= <user@domain.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `First Middle Last`,
+					Address: `user@domain.com`,
+				},
+			},
+		},
+		{
+			input: `First =?utf-8?Q?Middle?==?utf-8?Q?Last?= <user@domain.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `First Middle Last`,
+					Address: `user@domain.com`,
+				},
+			},
+		},
+		{
+			input: `First "Middle"=?utf-8?Q?Last?= <user@domain.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `First Middle Last`,
+					Address: `user@domain.com`,
+				},
+			},
+		},
+		{
+			input: `First "Middle" =?utf-8?Q?Last?= <user@domain.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `First Middle Last`,
+					Address: `user@domain.com`,
+				},
+			},
+		},
+		{
+			input: `First "Middle" =?utf-8?Q?Last?=<user@domain.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `First Middle Last`,
+					Address: `user@domain.com`,
+				},
+			},
+		},
+		{
+			input: `=?UTF-8?B?PEJlemUgam3DqW5hPg==?= <user@domain.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `<Beze jména>`,
+					Address: `user@domain.com`,
+				},
+			},
+		},
+	}
+	for _, test := range tests {
+		test := test
+
+		t.Run(test.input, func(t *testing.T) {
+			addrs, err := ParseAddressList(test.input)
+			assert.NoError(t, err)
+			assert.ElementsMatch(t, test.addrs, addrs)
+		})
+	}
+}
+
+func TestParseAddressList(t *testing.T) {
+	tests := []struct {
+		input string
+		addrs []*mail.Address
+	}{
+		{
+			input: `Alice <alice@example.com>, Bob <bob@example.com>, Eve <eve@example.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `Alice`,
+					Address: `alice@example.com`,
+				},
+				{
+					Name:    `Bob`,
+					Address: `bob@example.com`,
+				},
+				{
+					Name:    `Eve`,
+					Address: `eve@example.com`,
+				},
+			},
+		},
+		{
+			input: `Ed Jones <c@a.test>,joe@where.test,John <jdoe@one.test>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `Ed Jones`,
+					Address: `c@a.test`,
+				},
+				{
+					Address: `joe@where.test`,
+				},
+				{
+					Name:    `John`,
+					Address: `jdoe@one.test`,
+				},
+			},
+		},
+		{
+			input: `name (ignore comment)  <username@server.com>,  (Comment as name) username2@server.com`,
+			addrs: []*mail.Address{
+				{
+					Name:    `name`,
+					Address: `username@server.com`,
+				},
+				{
+					Address: `username2@server.com`,
+				},
+			},
+		},
+		{
+			input: `"normal name"  <username@server.com>, "comma, name" <address@server.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `normal name`,
+					Address: `username@server.com`,
+				},
+				{
+					Name:    `comma, name`,
+					Address: `address@server.com`,
+				},
+			},
+		},
+		{
+			input: `"comma, one"  <username@server.com>, "comma, two" <address@server.com>`,
+			addrs: []*mail.Address{
+				{
+					Name:    `comma, one`,
+					Address: `username@server.com`,
+				},
+				{
+					Name:    `comma, two`,
+					Address: `address@server.com`,
+				},
+			},
+		},
+		{
+			input: `normal name  <username@server.com>, (comment)All.(around)address@(the)server.com`,
+			addrs: []*mail.Address{
+				{
+					Name:    `normal name`,
+					Address: `username@server.com`,
+				},
+				{
+					Address: `All.address@server.com`,
+				},
+			},
+		},
+		{
+			input: `normal name  <username@server.com>, All.("comma, in comment")address@(the)server.com`,
+			addrs: []*mail.Address{
+				{
+					Name:    `normal name`,
+					Address: `username@server.com`,
+				},
+				{
+					Address: `All.address@server.com`,
+				},
+			},
+		},
+	}
+	for _, test := range tests {
+		test := test
+
+		t.Run(test.input, func(t *testing.T) {
+			addrs, err := ParseAddressList(test.input)
+			assert.NoError(t, err)
+			assert.ElementsMatch(t, test.addrs, addrs)
+		})
+	}
+}
+
+func TestParseGroup(t *testing.T) {
+	tests := []struct {
+		input string
+		addrs []*mail.Address
+	}{
+		{
+			input: `A Group:Ed Jones <c@a.test>,joe@where.test,John <jdoe@one.test>;`,
+			addrs: []*mail.Address{
+				{
+					Name:    `Ed Jones`,
+					Address: `c@a.test`,
+				},
+				{
+					Address: `joe@where.test`,
+				},
+				{
+					Name:    `John`,
+					Address: `jdoe@one.test`,
+				},
+			},
+		},
+		{
+			input: `Undisclosed recipients:;`,
+			addrs: []*mail.Address{},
+		},
+		{
+			input: `(Empty list)(start)Hidden recipients  :(nobody(that I know))  ;`,
+			addrs: []*mail.Address{},
+		},
+	}
+	for _, test := range tests {
+		test := test
+
+		t.Run(test.input, func(t *testing.T) {
+			addrs, err := ParseAddressList(test.input)
+			assert.NoError(t, err)
+			assert.ElementsMatch(t, test.addrs, addrs)
+		})
+	}
+}
+
+// TestParseRejectedAddresses tests that weird addresses that are rejected by
+// serverside are also rejected by us. If for some reason we end up being able
+// to parse these malformed addresses, great! For now let's collect them here.
+func TestParseRejectedAddresses(t *testing.T) {
+	tests := []struct {
+		input string
+		addrs []*mail.Address
+	}{
+		{input: `"comma, name"  <username@server.com>, another, name <address@server.com>`},
+		{input: `username`},
+		{input: `undisclosed-recipients:`},
+		{input: `=?ISO-8859-2?Q?First_Last?= <user@domain.com>, <user@domain.com,First/AAA/BBB/CCC,>`},
+		{input: `user@domain...com`},
+		{input: `=?windows-1250?Q?Spr=E1vce_syst=E9mu?=`},
+		{input: `"'user@domain.com.'"`},
+		{input: `<this is not an email address>`},
+	}
+	for _, test := range tests {
+		test := test
+
+		t.Run(test.input, func(t *testing.T) {
+			_, err := ParseAddressList(test.input)
+			assert.Error(t, err)
+		})
+	}
+}
+
+// TestIsEmailValidCategory runs over the "IsEmail" standard tests,
+// ensuring it can at least recognize all emails in the "valid" category.
+// In future, we should expand these tests to run over more categories.
+func TestIsEmailValidCategory(t *testing.T) {
+	f, err := os.Open("tests.xml")
+	require.NoError(t, err)
+	defer func() { require.NoError(t, err) }()
+
+	for test := range readTestCases(f) {
+		test := test
+
+		if test.category != "ISEMAIL_VALID_CATEGORY" {
+			continue
+		}
+
+		t.Run(test.id, func(t *testing.T) {
+			_, err := ParseAddressList(test.address)
+			assert.NoError(t, err)
+		})
+	}
+}
+
+type testCase struct {
+	id        string
+	address   string
+	category  string
+	diagnosis string
+}
+
+func readTestCases(r io.Reader) chan testCase {
+	ch := make(chan testCase)
+
+	var (
+		test testCase
+		data string
+	)
+
+	go func() {
+		decoder := xml.NewDecoder(r)
+
+		for token, err := decoder.Token(); err == nil; token, err = decoder.Token() {
+			switch t := token.(type) {
+			case xml.StartElement:
+				if t.Name.Local == "test" {
+					test = testCase{
+						id: t.Attr[0].Value,
+					}
+				}
+
+			case xml.EndElement:
+				switch t.Name.Local {
+				case "test":
+					ch <- test
+
+				case "address":
+					test.address = data
+
+				case "category":
+					test.category = data
+
+				case "diagnosis":
+					test.diagnosis = data
+				}
+
+			case xml.CharData:
+				data = string(t)
+			}
+		}
+
+		close(ch)
+	}()
+
+	return ch
+}

pkg/message/rfc5322/parse_date_time_test.go

@@ -0,0 +1,248 @@
+// Copyright (c) 2020 Proton Technologies AG
+//
+// This file is part of ProtonMail Bridge.
+//
+// ProtonMail Bridge is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// ProtonMail Bridge is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with ProtonMail Bridge.  If not, see <https://www.gnu.org/licenses/>.
+
+package rfc5322
+
+import (
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/assert"
+)
+
+func TestParseDateTime(t *testing.T) {
+	tests := []struct {
+		input string
+		want  string
+	}{
+		{
+			input: `Fri, 21 Nov 1997 09:55:06`,
+			want:  `1997-11-21T09:55:06Z`,
+		},
+		{
+			input: `Fri, 21 Nov 1997 09:55:06 -0600`,
+			want:  `1997-11-21T09:55:06-06:00`,
+		},
+		{
+			input: `Tue, 1 Jul 2003 10:52:37 +0200`,
+			want:  `2003-07-01T10:52:37+02:00`,
+		},
+		{
+			input: `Thu, 13 Feb 1969 23:32:54 -0330`,
+			want:  `1969-02-13T23:32:54-03:30`,
+		},
+		{
+			input: "Thu, 13 Feb 1969 23:32 -0330 (Newfoundland Time)",
+			want:  `1969-02-13T23:32:00-03:30`,
+		},
+		{
+			input: `2 Jan 2006 15:04:05 -0700`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `2 Jan 2006 15:04:05 MST`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `2 Jan 2006 15:04 -0700`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+		{
+			input: `2 Jan 2006 15:04 MST`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+		{
+			input: `2 Jan 06 15:04:05 -0700`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `2 Jan 06 15:04:05 MST`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `2 Jan 06 15:04 -0700`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+		{
+			input: `2 Jan 06 15:04 MST`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+		{
+			input: `02 Jan 2006 15:04:05 -0700`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `02 Jan 2006 15:04:05 MST`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `02 Jan 2006 15:04 -0700`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+		{
+			input: `02 Jan 2006 15:04 MST`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+		{
+			input: `02 Jan 06 15:04:05 -0700`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `02 Jan 06 15:04:05 MST`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `02 Jan 06 15:04 -0700`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+		{
+			input: `02 Jan 06 15:04 MST`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+		{
+			input: `Mon, 2 Jan 2006 15:04:05 -0700`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `Mon, 2 Jan 2006 15:04:05 MST`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `Mon, 2 Jan 2006 15:04 -0700`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+		{
+			input: `Mon, 2 Jan 2006 15:04 MST`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+		{
+			input: `Mon, 2 Jan 06 15:04:05 -0700`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `Mon, 2 Jan 06 15:04:05 MST`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `Mon, 2 Jan 06 15:04 -0700`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+		{
+			input: `Mon, 2 Jan 06 15:04 MST`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+		{
+			input: `Mon, 02 Jan 2006 15:04:05 -0700`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `Mon, 02 Jan 2006 15:04:05 MST`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `Mon, 02 Jan 2006 15:04 -0700`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+		{
+			input: `Mon, 02 Jan 2006 15:04 MST`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+		{
+			input: `Mon, 02 Jan 06 15:04:05 -0700`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `Mon, 02 Jan 06 15:04:05 MST`,
+			want:  `2006-01-02T15:04:05-07:00`,
+		},
+		{
+			input: `Mon, 02 Jan 06 15:04 -0700`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+		{
+			input: `Mon, 02 Jan 06 15:04 MST`,
+			want:  `2006-01-02T15:04:00-07:00`,
+		},
+	}
+	for _, test := range tests {
+		test := test
+
+		t.Run(test.input, func(t *testing.T) {
+			got, err := ParseDateTime(test.input)
+			assert.NoError(t, err)
+			assert.Equal(t, test.want, got.Format(time.RFC3339))
+		})
+	}
+}
+
+func TestParseDateTimeObsolete(t *testing.T) {
+	tests := []struct {
+		input string
+		want  string
+	}{
+		{
+			input: `21 Nov 97 09:55:06 GMT`,
+			want:  `1997-11-21T09:55:06Z`,
+		},
+		{
+			input: `Wed, 01 Jan 2020 12:00:00 UTC`,
+			want:  `2020-01-01T12:00:00Z`,
+		},
+		{
+			input: `Wed, 01 Jan 2020 13:00:00 UTC`,
+			want:  `2020-01-01T13:00:00Z`,
+		},
+		{
+			input: `Wed, 01 Jan 2020 12:30:00 UTC`,
+			want:  `2020-01-01T12:30:00Z`,
+		},
+	}
+	for _, test := range tests {
+		test := test
+
+		t.Run(test.input, func(t *testing.T) {
+			got, err := ParseDateTime(test.input)
+			assert.NoError(t, err)
+			assert.Equal(t, test.want, got.Format(time.RFC3339))
+		})
+	}
+}
+
+func TestParseDateTimeRelaxed(t *testing.T) {
+	tests := []struct {
+		input string
+		want  string
+	}{
+		{
+			input: `Mon, 28 Jan 2019 20:59:01 0000`,
+			want:  `2019-01-28T20:59:01Z`,
+		},
+		{
+			input: `Mon, 25 Sep 2017 5:25:40 +0200`,
+			want:  `2017-09-25T05:25:40+02:00`,
+		},
+	}
+	for _, test := range tests {
+		test := test
+
+		t.Run(test.input, func(t *testing.T) {
+			got, err := ParseDateTime(test.input)
+			assert.NoError(t, err)
+			assert.Equal(t, test.want, got.Format(time.RFC3339))
+		})
+	}
+}