Technical¶

Components¶

The email integration consists of the following components:

Web Components
Custom Endpoints
EmailEngine
Database Structure
Webhook
Taskhandler

Web Components¶

Email Composer¶

The web component lwc-email-composer enables the user to select an account, find recipients among the follower parents or create them on the fly, compose subject and body, pick attachments and send the message.

When configured as the activities composer for the conversation limetype it's rendered as part of the activity feed and the user can reply while seeing all previous messages of this conversation. This view configuration will be imported as part of the Lime Admin setup step or add the tag manually as composer web component under Views -> Conversation -> Activities.

Feed Item Factory¶

Typically, you can configure the values of a feed item in the Activity item view of a limetype. However, for conversation messages in the email integration that's not enough, so the package provides its own factory implementation. This e.g. enables the integration to choice the style of the message or construct the heading based on the source and context. Any configuration on the conversation message in Lime Admin will most likely be overwritten by the factory.

Unread Message Indicator¶

The lwc-unread-message-icon-indicator web component can be configured in the table view of a conversation parent limetype to highlight conversations with unread messages. Follow the first point here to configure it.

Administrator Tools¶

The following web components are rendered as part of the package's schema and provide the administrator with tools to manage the email integration in Lime Admin.

OAuth2 Application Management: The page "OAuth2 Applications" renders the component lwc-oauth-app-management. It allows the administrator to create, update, and delete OAuth2 applications. Furthermore, the sync of applications and accounts can be started from here.

Template Management: Under "Templates" the component lwc-template-management is rendered. It allows the administrator to create, update, delete and preview email templates. The component also provides the possibility to create a default template.

Account Management: Under "Email Accounts" the component lwc-account-management is rendered. It allows the administrator to create, update, delete and disable/ enable email accounts. The component also provides a button to sync emails of a list of accounts and with that start the recovery process manually.

Custom Endpoints¶

The email integration comes with a set of custom endpoints that are used to interact with the EmailEngine API and the web client. The following tables show an overview of available endpoints and a short description. For more detailed information you can find the "Email Integration API" as part of your application's API documentation.

The base URL for all the endpoints is <CUSTOMER DOMAIN>/<APP NAME>/limepkg-email.

Account¶

Method	Endpoint	Description
GET	/user/account/	Get all accounts connect to the current user
POST	/account/	Create new account, the form endpoint callbacks to this.
GET	/account/	Get all accounts
DELETE	/account/	Delete list of accounts
PUT	/account/	Enable or disable accounts in bulk
PUT	/account/{id}/	Update account
POST	/account/form/	Generates a link to the authentication form

Conversation¶

Method	Endpoint	Description
POST	/conversation/	Create a new conversation (used for new conversations from the client)

Message¶

Method	Endpoint	Description
GET	/messages/documents	Fetch all documents related to the given messages
POST	/message/{account_id}/	Submit message for delivery
GET	/failed-messages/{account_id}/	Fetch all failed messages for the given account
POST	/force-reload-message/{message_id}/	Update message date with a microsecond to reload state

Oauth2 Application¶

Method	Endpoint	Description
POST	/oauth_app/	Create a new Oauth2 application
GET	/oauth_app/	Get all Oauth2 applications
DELETE	/oauth_app/{oauth_app_id}	Delete OAuth2 application and connected accounts
PUT	/oauth_app/{oauth_app_id}	Update OAuth2 application

Sync¶

Method	Endpoint	Description
POST	/sync/	Start task to sync apps and accounts from EmailEngine to Lime

Template¶

Method	Endpoint	Description
POST	/template/	Create a new template
GET	/template/	Get all templates
GET	/default_template/	Get default template
GET	/template/{id}/	Get template
DELETE	/template/{id}/	Delete template
PUT	/template/{id}/	Update template
GET	/preview-template/{template_id}/	Preview a template with mocked data

Time Zone¶

Method	Endpoint	Description
GET	/time-zone/	Get all time zones

Recovery Process¶

Method	Endpoint	Description
POST	/recovery/start/	Start task for recovery process

Webhook¶

Method	Endpoint	Description
POST	/webhook/	Webhook endpoint for EmailEngine

EmailEngine¶

The email integration is built based on the EmailEngine Email API.

IDs Explained¶

EmailEngine comes with an abundance of different message identifiers. They are all stored in Lime CRM for the sake of completeness and for different use cases, such as: making API requests towards EmailEngine, find existing conversations, and for traceability.

id - Value used in EmailEngine's API requests. Stored as emailengineid on conversationmessage limetype.
uid - IMAP UID identifier (unique per folder integer). Stored as uniqueid on conversationmessage limetype.
emailId - Uniquely identifies a message entity for an email account. Stored as emailid on conversationmessage limetype if mail server supports it.
messageId - Value from Message-ID header. Stored as messageid on messagefollower limetype.
queueId - Queue identifier for scheduled email. Stored as queueid on messagefollower limetype.
threadId - Thread identifier. Stored as threadid on conversation limetype if mail server supports it.

More information about the IDs can be found here.

Database Structure¶

The UML diagram below shows the database structure of the email integration. The mentioned tables and fields can be installed through the setup page in Lime Admin.

Database structure

Webhook¶

The webhook is triggered when the monitored account receives an email. The webhook starts a task to process the email and create a conversation in Lime CRM.

Configuration¶

You can read up on the configuration details in here.

Taskhandler¶

Import Messages¶

The task limepkg_email.tasks.tasks.message_to_limeobjects imports messages from EmailEngine to Lime CRM and has two possible outcomes:

The email is successfully processed and a conversation and all related objects are created.
The email is not processed due to an error and is moved to a separate folder called Lime CRM Integration Failed for the recovery process to handle.

Automatic Recovery Process¶

The email integration offers a recovery process for failed or unprocessed messages. It runs as a scheduled task every 20 minutes which is configurable on-premise. It is possible to turn off the scheduled task through the config key handle_unprocessed_emails_enabled. The recovery process can also be triggered manually by clicking "Sync emails" from the account management in Lime Admin.

Responsibilities:

Fallback for the webhook - To ensure no message is left behind, the recovery process checks for unread messages in the inbox folder and processes them. The config key vacuum_cleaner_grace_period is used to skip messages that the webhook hasn't gotten to yet and therefore avoid duplicates. The default value is 2 minutes.
Self-heal messages - Messages which couldn't be processed due to an error are moved to a separate folder. The recovery process will attempt to process these messages again. The config key max_import_age_for_failed_messages describes how far back in time it should look for failed messages. The default value is 1 day.

Task name: limepkg_email.tasks.tasks.recovery_process

Configuration¶

Read more here for how to fine tune the recovery process in your application.

Sync Apps and Accounts¶

Data about OAuth2 Applications and Email Accounts is stored both in the specific Lime CRM application and the Lime CRM server. By running the synchronization task limepkg_email.tasks.tasks.sync_emailengine_to_lime it's ensured that the information in both sources is in sync.

This can be especially useful when you have created a new Lime CRM test application based on your production application, or for other reasons, have restored database backup files between environments.

You can trigger the sync from the OAuth application page in Lime Admin.