proton-bridge/test

Integration tests

This folder contains integration tests of the Bridge app.

What and how we are testing

graph LR
    S[Server]
    C[Client]
    U[User]
    Creds[Credentials store]

    subgraph "Bridge app"
        Core[Bridge core]
        Store
        Frontend["Qt / CLI"]
        IMAP
        SMTP
        API[PMAPI]

        IMAP --> Core
        SMTP --> Core
        Frontend --> Core
        Store --> Core
        Core --> API
    end

    C --> IMAP
    C --> SMTP
    U --> Frontend
    API --> S
    Core --> Creds

We want to test Bridge app from outside as much as possible. So we mock server (API), credentials store and call commands to IMAP or SMTP the same way as client would do.

## Running tests

In order to run Integration tests just go into the test folder cd test and run make test.

You can also test only specific feature (or subset of features) by using FEATURES environment variable: FEATURES=features/imap/message/create.feature make test.

Example test

BDD test in gherkin (cucumber) format (https://cucumber.io/docs/gherkin/reference/).

Feature: IMAP update messages
  Background:
    Given there is connected user "user"
    And there are messages in mailbox "INBOX" for "user"
      | from              | to         | subject | body  | read  | starred |
      | john.doe@mail.com | user@pm.me | foo     | hello | false | false   |
      | jane.doe@mail.com | name@pm.me | bar     | world | true  | true    |
    And there is IMAP client logged in as "user"
    And there is IMAP client selected in "INBOX"

  Scenario: Mark message as read
    When IMAP client marks message "1" as read
    Then IMAP response is "OK"
    And message "1" in "INBOX" for "user" is marked as read
    And message "1" in "INBOX" for "user" is marked as unstarred

Is translated into code with godog (https://github.com/cucumber/godog/).

// Registration
func FeatureContext(s *godog.Suite) {
	s.Step(`^there is connected user "([^"]*)"$`, thereIsConnectedUser)
}

// Godog step function
func thereIsConnectedUser(username string) error {
	account := ctx.GetTestAccount(username)
	if account == nil {
		return godog.ErrPending
	}
	ctx.GetPMAPIController().AddUser(account.User, account.Addresses)
	return ctx.LoginUser(account.Username(), account.Password(), account.MailboxPassword())
}

BDD

BDD has three parts:

• Given (setup),
• When (action)
• and Then (check).

Setup has to prepare context and always end without error. Action, on the other hand, needs to always end without error, but store it in the context. Check should analyze the status of the bridge, store or API and also check whether something failed before.

Therefore we cannot use a sentence such as there is user for both setup and check steps. We always begin setup steps with there is/are, while check steps are written in the form something is/has feature. Actions are written in the form something does action. By doing this we can always be sure what each steps does or should do.

In the code, we separate those parts in its own files to make sure it's clear how the function should be implemented.

In the Given phase is also generally better to setup data (as there are messages...) first, then users (there is connected user...) and then connections (there is IMAP client...). This can prevent some hitches in internal implementation of integration tests.

API faked by fakeapi or liveapi

We need to control what server returns. Instead of using raw JSONs, we fake the whole pmapi for local testing. Fake pmapi behaves as much as possible the same way as real server, but does not follow every single detail. Otherwise we would end up with writing complete server. :-)

For both -- fake local pmapi and real live server -- we use controller. Controller is available on test context and does setup like setting up internet connection, user settings, labels or messages.

Accounts for each environment are set up in accounts folder. Each test function should use TestAccount object obtained by test ID (such as user or userMultipleAddress for users, or primary or secondary for addresses) and use available functions to get real IDs (even if fake API uses the test IDs as real ones).

Testing against live is using real users and doesn't work in parallel. Only one job against live at a time can be running.

External e-mail accounts

We have some external accounts which we are using for testing:

• pm.bridge.qa@gmail.com
• bridge-qa@yandex.ru
• bridgeqa@seznam.cz

For access, ask bridge team.