Amazon Simple Email Service (SES) is cloud based email service that allow to send emails from within any application. Its basically designed to support several email use cases, including transactional, marketing, or mass email communications.

Amazon SES also has support for incoming email but one of the major challenge with incoming SES emails is  we need AWS SES to forward email into an external email address.

We need to implement this because AWS SES doesn’t support IMAP/POP3 protocols and IMAP/POP3 are the services supported by most of the email clients. We also need this solution if our website has contact form and we need to get email alerts when our clients contact us using the website contact form.

So in short our goal is AWS SES to forward email into an external email address like any free gmail address which supports IMAP/POP3 protocols.  In this post we use Amazon cloud solutions like SES, Amazon S3, and AWS Lambda to create such solution and its heavily inspired from below two blog posts.

https://aws.amazon.com/blogs/messaging-and-targeting/forward-incoming-email-to-an-external-destination/
https://github.com/arithmetric/aws-lambda-ses-forwarder

Below is the Architecture diagram we are following and the work flow.

  1. A external email arrives and Amazon SES handles the incoming email for your domain.
  2. According to Amazon SES receipt rule the incoming message saves in an S3 bucket.
  3. The Amazon SES receipt rule also triggers a Lambda function.
  4. The Lambda function retrieves the message content from S3, and send it back to SES.
  5. Finally Amazon SES sends the message to the forwarding email destination server.

Prerequisites

we already need a verified domain in SES with MX record points to SES provided one. Also if the Amazon SES is in sandbox mode, submit a support request to have it removed.

Create S3 Bucket and Edit Policy

Now lets get started. First create an s3 bucket from amazon s3 bucket console. Make sure you create s3 bucket in same region as your Amazon SES exists. Once created click on s3 bucket name >> Click Permissions tab >> Click Bucket policy. Add below policy to the bucket.

Copy to Clipboard

In the policy, make the following changes:

Replace <bucketName> with the name of your S3 bucket.
Replace <awsAccountId> with your AWS account ID.

Create an IAM Policy and Role

Now we are creating IAM policy and associating it to an IAM role. We assign this newly created role to the Lambda Function we create later.

Go to the IAM console in AWS console >> Click on Policies available in left side >> Click Create Policy >> Switch to JSON tab. Copy paste below content and review and create the policy by giving a name you can identify.

Copy to Clipboard

In the preceding policy, make the following changes:

Replace <bucketName> with the name of the S3 bucket that you created earlier.
Replace <region> with the name of the AWS Region that you created the bucket in. For example us-west-2
Replace <awsAccountId> with your AWS account ID.

Now lets create a new IAM role. For that Click Roles  from IAM console >> click  Create Role >>  Select type of trusted entity as AWS Service  >> Choose Lambda from it
>> Click next  Permissions  >> select the policy that we just created to the new role >> Click review  and create by giving a Name for the Role. Make sure the role name you gave is identifiable amount other existing roles.

Create the Lambda Function

Now go to the AWS Lambda console, create a new Python 3.7 function from scratch.  For that click Create Function  >> Choose runtime as Python 3.7 >>  In Choose or create an execution role , select use an existing role >> Select the role that we created for Lambda from drop down menu >> Gave a name and click create.

In the function code editor section gave below code.

Copy to Clipboard

Now we need to create the following environment variables for the Lambda function by clicking on Edit button next to Environment variables  which is available next to function code in AWS Lambda console.

Key                                                            Value
MailS3Bucket                        The name of the S3 bucket that you created earlier.

MailS3Prefix                         The path of the folder in the S3 bucket where you will store incoming email.  This is the name of the sub-folder  where the actual emails resides. If we don’t have any such folder path and we directly keeping the incoming emails in the main bucket folder. We don’t have to define this environment variable. Also in the lambda function code search for incoming_email_prefix  variable  and set the value as  False.  In the code it will look like below.

incoming_email_prefix = False

MailSender                            The  incoming SES email address that we need to forward messages. This address has to be verified in SES. If the entire domain is verified, no need to verify the email specifically.

MailRecipient                        The destination  external forwarding email address.

Region                                   The name of the AWS Region  where the SES exists.

Also from AWS Lambda Basic settings section, we can increase the code execution time out and the memory, if we are expecting large emails.  The basic timeout and memory limit is enough for normal emails.

Create a Receipt Rule Set

Now go the AWS SES console and click on Rule Sets >>  Click Create a New Rule Set  >>  Gave A name for the rule >>

On the Recipients configuration page, add our SES email addresses from which you want to forward email.

On the Actions configuration page, add an S3 action first and then an Lambda action.

For the S3 action:  choose the existing S3 bucket we created . Optionally, add an object key prefix. Leave Encrypt Message unchecked and SNS Topic set to [none].

For the Lambda action: Choose the Lambda function we created . Leave Invocation Type set to Event and SNS Topic set to [none].

If you are asked by SES to add permissions to access lambda:InvokeFunction, agree to it.

Once created, it will list under rule set section. Select it and make it as  Active one by clicking ” Set as Active rule set ” button.

This concludes the setting up. Now as part of testing, try to send an email to SES email address . Make sure that the email arrives in the  S3 bucket. In in 5min normally , the email will forward to remote forwarded email address.

Now one of the main downside of this Lambda function is the the actual email will reach the remote destination as attachment. Not just like a regular forwarded email we usually see.

So we are going to create a new Lambda function  and use that Lambda function for forwarding emails.  For that go the AWS Lambda console, create a new function

Ensure Runtime is set to Node.js 12.x.

Ensure Handler is set to index.handler.

For Role, choose our old Lambda role itself from the drop down menu.

In the Function code section, copy/paste below Node.js code.

Copy to Clipboard

Now lets set Environment variables in AWS Lambda console for below two variables.

emailBucket : S3 bucket name where email stores.

fromEmail   :   Set a SES email address like noreply@yourdomain.com. This is the from email address which will display in the destination forwarded email box.

Also correct the variables defined in Lambda code according to our values which is under the defaultConfig section. the values we need to correct is.

fromEmail  ,   details are already mentioned above.

emailBucket , details are already mentioned above.

emailKeyPrefix , S3 key name prefix where SES stores email. Include the  trailing slash.  If we don’t have any such subfolder which stored emails in s3 bucket and we are directly keeping the emails in a single folder. Just leave it as blank and it code line will look like below.

subjectPrefix: “”,

forwardMapping:  ,   in this variable the first email address is our SES forwarding email address and the second one will be our external destination email box like our gmail address.

Now go to the SES Rule Sets we created earlier and edit it and change the lambda function into the new one we created.  Test the working again.

In my case it didn’t worked in first place and no emails forwarded from s3 bucket. So I started troubleshooting. The first thing you need to is, go the  AWS CloudWatch console

Click  CloudWatch Logs > Log groups >> Click on our AWS Lambda function name log >>  Click on latest log if there is multiple one. 

Look for errors and I found below error.

level: ‘info’, message: ‘Fetching email at s3:

level: ‘error’, message: ‘copyObject() returned error:’, error: AccessDenied: Access Denied at

INFO { level: ‘error’, message: ‘Step returned error: Error: Could not make readable copy of email.’,

Basically it means that AWS Lambda function can’t read emails from s3 bucket.  In order to resolve the issue, I go the IAM console >> Click roles >> choose our role created for SES forward >> Click attached policy >> add below entry under “ses:SendRawEmail”,  as next line.

“s3:PutObject”

Refer aws articles if you got any syntax errors.

Once that done. Try to test our email forwarding and  at this time its worked successfully and we got  email in remote email inbox  like regular forwarded email.

After a while we got a failure message at forwarding email address with subject as   “FW: Delivery Status Notification (Failure)“. On further lookup from email header, we noticed below error code.

Diagnostic-Code: Amazon SES did not send the message to this address because it is on the suppression list for your account. For more information about removing addresses from the suppression list, see the Amazon SES Developer Guide at https://docs.aws.amazon.com/ses/latest/DeveloperGuide/sending-email-suppression-list.html
Status: 5.1.1

In order to resolve this, issue below two commands. The first one show the email address listed in account level suppression list and second is to remove that email address from the list.

Copy to Clipboard