Merge pull request #55 from cjww-development/feature/email-providers

AWS SES and Mailgun email providers

README.md

@@ -24,7 +24,10 @@ However, if a service offers integration with a third party OAuth2 provider, the
     - Create clients from presets
 - User and developer registration
     - Update email and password
-    - Email verification (via AWS SES)
+    - Email verification via
+	    - AWS SES
+	    - Mailgun
+	    - More offerings to come
     - TOTP MFA
     - See list of apps accessing your account
     - Revoke sessions for apps accessing your account
@@ -108,15 +111,32 @@ Run `./docker-boot.sh` to run this process.
 ## Docker compose variables
 The following table describes what each of the gatekeeper envs means in the docker compose file.
 
-| Env Var       | Default                          | Description                                                                                              |
-|---------------|----------------------------------|----------------------------------------------------------------------------------------------------------|
-| VERSION       | dev                              | The version of Gatekeeper you're running. Appears at the bottom of pages                                 |
-| EMAIL_FROM    | test@email.com                   | The email address used to send emails from Gatekeeper                                                    |
-| MONGO_URI     | mongodb://mongo.local            | Where MongoDB lives. The database that backs Gatekeeper                                                  |
-| APP_SECRET    | 23817cc7d0e6460e9c1515aa4047b29b | The app secret scala play uses to sign session cookies and CSRF tokens. Should be changed to run in prod |
-| ENC_KEY       | 23817cc7d0e6460e9c1515aa4047b29b | The key used to secure data. Should be changed to run in prod                                            |
-| MFA_ISSUER    | Gatekeeper (docker)              | The string used to describe the TOTP Code in apps like Google Authenticator                              |
-| SMS_SENDER_ID | SmsVerify                        | The string used to say where SMS messages have come from                                                 |
+| Env Var         | Default                          | Description                                                                                                |
+|-----------------|----------------------------------|------------------------------------------------------------------------------------------------------------|
+| VERSION         | dev                              | The version of Gatekeeper you're running. Appears at the bottom of pages                                   |
+| EMAIL_FROM      | test@email.com                   | The email address used to send emails from Gatekeeper                                                      |
+| MONGO_URI       | mongodb://mongo.local            | Where MongoDB lives. The database that backs Gatekeeper                                                    |
+| APP_SECRET      | 23817cc7d0e6460e9c1515aa4047b29b | The app secret scala play uses to sign session cookies and CSRF tokens. Should be changed to run in prod   |
+| ENC_KEY         | 23817cc7d0e6460e9c1515aa4047b29b | The key used to secure data. Should be changed to run in prod                                              |
+| MFA_ISSUER      | Gatekeeper (docker)              | The string used to describe the TOTP Code in apps like Google Authenticator                                |
+| SMS_SENDER_ID   | SmsVerify                        | The string used to say where SMS messages have come from                                                   |
+| EMAIL_PROVIDER  | n/a                              | Used to determine what email provider to use. Valid options are ses or mail-gun                            |
+| AWS_REGION      | n/a                              | Should only be set if EMAIL_PROVIDER is ses. Should match the AWS region you're running SES from           |
+| MAILGUN_API_KEY | n/a                              | Should only be set if EMAIL_PROVIDER is mail-gun. Obtained from the mailgun console after account creation |
+| MAILGUN_URL     | n/a                              | Should only be set if EMAIL_PROVIDER is mail-gun. Obtained from the mailgun console after account creation |
+      
+## Choosing an email provider
+Gatekeeper currently sends emails via AWS SES or Mailgun. Both support sending emails from a proper address, or some address on a verified domain. On their respective free tiers you can only send to email addresses you've verified in SES or Mailgun.
+To lift that limitation you need to be on a paid plan. However, AWS SES lets you send 62000 emails a month for free forever, but you need to be in their production zone on SES.
+Mailgun, on their flex plan, allows 5000 emails a month for 3 months, and then you move to pay as you go ($0.80 / 1000 emails).
+
+### What if I don't want to use AWS SES or Mailgun?
+That's a fair question. Neither may suit you. If you're technically minded, look at [the adding more email providers section](#Adding-further-email-providers) to find out more about adding your preferred provided (dev work required).
+If you're not technically minded, no fear, get started with a [feature request](https://github.com/cjww-development/gatekeeper/issues/new?assignees=&labels=&template=feature_request.md&title=) and we can try to make your request a reality.
+
+Find more information about each here
+- [Mailgun](https://www.mailgun.com/)
+- [AWS SES](https://aws.amazon.com/ses/)
 
 ## Adding further client presets
 Gatekeeper supports creating client presets for 
@@ -151,6 +171,83 @@ For the time being, the service's icon needs to be hosted in Gatekeeper, find a
 
 Once a service has been added to this list, it will be available for creation in the frontend. 
 
+## Adding further email providers
+Right now Gatekeeper can send emails via AWS SES or Mailgun, however these providers can be extended. Below details how you can go about adding new email providers.
+
+1. Clone the repository and branch off as `feature/email-provider/new-provider-name`
+
+
+2. Add the new email providers config into `conf/application.conf` 
+```hocon
+email-service {
+    selected-provider = ${?EMAIL_PROVIDER}
+    message-settings {
+      from = ${?EMAIL_FROM}
+      verification-subject = "Verifying your account"
+    }
+
+    ses {
+      region = ${?AWS_REGION}
+    }
+
+    mail-gun {
+      api-key = ${?MAILGUN_API_KEY}
+      url = ${?MAILGUN_URL}
+    }
+    
+    new-provider-name {
+        ...whatever config the new provider needs. API Key, endpoints etc
+    }
+}
+```
+
+3. Create a new default class and trait inside of `app/services/comms/email`. Traits in this package should all extend the `EmailService`. Ensure appropriate tests are included.
+
+```scala
+import database.VerificationStore
+import models.{EmailResponse, Verification}
+import play.api.mvc.Request
+import services.comms.email.EmailService
+
+import scala.concurrent.{ExecutionContext, Future}
+
+class DefaultNewProvider @Inject()(val config: Configuration,
+                                   val verificationStore: VerificationStore) extends NewProvider {
+  override val emailSenderAddress: String = config.get[String]("email-service.message-settings.from")
+  override val verificationSubjectLine: String = config.get[String]("email-service.message-settings.verification-subject")
+  override val valueFromConfigNeededForNewProvider: String = config.get[String]("email-service.new-provider-name.whatever-required-config")
+}
+
+trait NewProvider extends EmailService {
+
+  val valueFromConfigNeededForNewProvider: String
+
+  override def sendEmailVerificationMessage(to: String, record: Verification)(implicit req: Request[_], ec: ExecutionContext): Future[EmailResponse] = {
+    //What ever logic is needed to send an email via the new provider
+    // Should return a Future EmailResponse
+    Future.successful(EmailResponse(
+      provider = "new-provider-name",
+      userId = record.userId,
+      messageId = "some email message id returned from the new email provider"
+    ))
+  }
+}
+```
+
+4. Add new class to Gatekeepers Service bindings in `app/global/ServiceBindings`
+```scala
+private def emailService(config: Configuration): Seq[Binding[_]] = {
+  config.get[String]("email-service.selected-provider") match {
+    case "ses"               => Seq(bind[EmailService].to[DefaultSesService].eagerly())
+    case "mailgun"           => Seq(bind[EmailService].to[DefaultMailgunService].eagerly())
+    case "new-provider-name" => Seq(bind[EmailService].to[DefaultNewProvider].eagerly())
+    case _                   => throw new RuntimeException("Invalid email provider")
+  }
+}
+```
+
+5. Create a pull request
+
 License
 =======
 This code is open sourced licensed under the Apache 2.0 License

app/global/ServiceBindings.scala

@@ -30,7 +30,8 @@ import filters.{DefaultRequestLoggingFilter, DefaultShutteringFilter, RequestLog
 import orchestrators._
 import play.api.inject.{Binding, Module}
 import play.api.{Configuration, Environment}
-import services.comms.{DefaultEmailService, DefaultPhoneService, EmailService, PhoneService}
+import services.comms.email.{DefaultMailgunService, DefaultSesService, EmailService}
+import services.comms.{DefaultPhoneService, PhoneService}
 import services.oauth2._
 import services.security.{DefaultTOTPService, TOTPService}
 import services.users._
@@ -45,7 +46,8 @@ class ServiceBindings extends Module {
     controllers() ++
     apiControllers() ++
     testControllers() ++
-    systemControllers()
+    systemControllers() ++
+    emailService(configuration)
   }
 
   private def globals(): Seq[Binding[_]] = Seq(
@@ -78,7 +80,6 @@ class ServiceBindings extends Module {
     bind[TokenService].to[DefaultTokenService].eagerly(),
     bind[ClientService].to[DefaultClientService].eagerly(),
     bind[TOTPService].to[DefaultTOTPService].eagerly(),
-    bind[EmailService].to[DefaultEmailService].eagerly(),
     bind[PhoneService].to[DefaultPhoneService].eagerly(),
     bind[JwksService].to[DefaultJwksService].eagerly()
   )
@@ -121,4 +122,12 @@ class ServiceBindings extends Module {
     bind[EmailViewTestController].to[DefaultEmailViewTestController].eagerly(),
     bind[ExceptionTestController].to[DefaultExceptionTestController].eagerly()
   )
+
+  private def emailService(config: Configuration): Seq[Binding[_]] = {
+    config.get[String]("email-service.selected-provider") match {
+      case "ses"     => Seq(bind[EmailService].to[DefaultSesService].eagerly())
+      case "mailgun" => Seq(bind[EmailService].to[DefaultMailgunService].eagerly())
+      case _         => throw new RuntimeException("Invalid email provider")
+    }
+  }
 }

app/models/EmailResponse.scala

@@ -0,0 +1,21 @@
+/*
+ * Copyright 2021 CJWW Development
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package models
+
+case class EmailResponse(provider: String,
+                         userId: String,
+                         messageId: String)

app/orchestrators/RegistrationOrchestrator.scala

@@ -21,7 +21,8 @@ import dev.cjww.mongo.responses.{MongoFailedCreate, MongoFailedUpdate, MongoSucc
 import models.{RegisteredApplication, User, Verification}
 import org.slf4j.LoggerFactory
 import play.api.mvc.Request
-import services.comms.{EmailService, PhoneService}
+import services.comms.PhoneService
+import services.comms.email.EmailService
 import services.oauth2.ClientService
 import services.users.{RegistrationService, UserService}
 import utils.StringUtils._
@@ -73,15 +74,10 @@ trait RegistrationOrchestrator {
         registrationService.createNewUser(user.copy(salt = saltToUse)) flatMap {
           case MongoSuccessCreate =>
             val emailAddress = user.digitalContact.email.address.decrypt.getOrElse(throw new Exception("Decryption error"))
-            emailService.saveVerificationRecord(user.id, user.digitalContact.email.address, user.accType) map { record =>
-              logger.info(s"[registerUser] - Registration successful; new user under ${user.id}")
-              Try(emailService.sendEmailVerificationMessage(emailAddress, record.get)) match {
-                case Success(_) =>
-                  logger.info(s"[registerUser] - Send email verification message to user ${user.id}")
-                  Registered
-                case Failure(e) =>
-                  logger.warn("[registerUser] - Problem sending email verification message", e)
-                  Registered
+            emailService.saveVerificationRecord(user.id, user.digitalContact.email.address, user.accType) flatMap { record =>
+              emailService.sendEmailVerificationMessage(emailAddress, record.get).map { resp =>
+                logger.info(s"[registerUser] - Registration successful; new user under ${user.id}")
+                Registered
               }
             }
           case MongoFailedCreate =>

app/orchestrators/UserOrchestrator.scala

@@ -21,7 +21,7 @@ import models._
 import org.slf4j.LoggerFactory
 import play.api.libs.json.{JsObject, Json}
 import play.api.mvc.Request
-import services.comms.EmailService
+import services.comms.email.EmailService
 import services.users.{RegistrationService, UserService}
 import utils.StringUtils._
 
@@ -72,15 +72,10 @@ trait UserOrchestrator {
             for {
               _ <- userService.updateUserEmailAddress(userId, obsEmail)
               Some(vRec) <- emailService.saveVerificationRecord(userId, obsEmail, user.accType)
+              resp <- emailService.sendEmailVerificationMessage(email, vRec)
             } yield {
-              Try(emailService.sendEmailVerificationMessage(email, vRec)) match {
-                case Success(_) =>
-                  logger.info(s"[updateEmailAndReVerify] - Send email verification message to user $userId")
-                  EmailUpdated
-                case Failure(e) =>
-                  logger.warn("[updateEmailAndReVerify] - Problem sending email verification message", e)
-                  EmailUpdated
-              }
+              logger.info(s"[updateEmailAndReVerify] - Reverification email sent with messageId ${resp.messageId} to userId ${resp.userId} via ${resp.provider}")
+              EmailUpdated
             }
           }
         }

app/services/comms/EmailService.scala → app/services/comms/email/EmailService.scala

@@ -14,72 +14,26 @@
  * limitations under the License.
  */
 
-package services.comms
+package services.comms.email
 
-import com.amazonaws.regions.Regions
-import com.amazonaws.services.simpleemail.model._
-import com.amazonaws.services.simpleemail.{AmazonSimpleEmailService, AmazonSimpleEmailServiceClientBuilder}
 import database.VerificationStore
 import dev.cjww.mongo.responses.{MongoDeleteResponse, MongoFailedCreate, MongoSuccessCreate}
-import dev.cjww.security.Implicits._
-import models.Verification
+import models.{EmailResponse, Verification}
 import org.joda.time.DateTime
 import org.mongodb.scala.model.Filters.{and, equal}
-import play.api.Configuration
 import play.api.mvc.Request
-import views.html.email.VerificationEmail
 
-import java.nio.charset.StandardCharsets
 import java.util.UUID
-import javax.inject.Inject
 import scala.concurrent.{Future, ExecutionContext => ExC}
 
-class DefaultEmailService @Inject()(val config: Configuration,
-                                    val verificationStore: VerificationStore) extends EmailService {
-  override val emailSenderAddress: String = config.get[String]("email.from")
-  override val verificationSubjectLine: String = config.get[String]("email.verification-subject")
-  override val emailClient: AmazonSimpleEmailService = AmazonSimpleEmailServiceClientBuilder
-    .standard()
-    .withRegion(Regions.EU_WEST_1)
-    .build()
-}
-
 trait EmailService {
+
   val emailSenderAddress: String
   val verificationSubjectLine: String
 
-  val emailClient: AmazonSimpleEmailService
-
   val verificationStore: VerificationStore
 
-  def sendEmailVerificationMessage(to: String, record: Verification)(implicit req: Request[_]): SendEmailResult = {
-    val queryParam = record.encrypt
-
-    val destination: Destination = new Destination()
-      .withToAddresses(to)
-
-    val subjectContent: Content = new Content()
-      .withCharset(StandardCharsets.UTF_8.name())
-      .withData(verificationSubjectLine)
-
-    val bodyContent: Content = new Content()
-      .withCharset(StandardCharsets.UTF_8.name())
-      .withData(VerificationEmail(queryParam).body)
-
-    val body: Body = new Body()
-      .withHtml(bodyContent)
-
-    val message: Message = new Message()
-      .withBody(body)
-      .withSubject(subjectContent)
-
-    val request: SendEmailRequest = new SendEmailRequest()
-      .withDestination(destination)
-      .withMessage(message)
-      .withSource(emailSenderAddress)
-
-    emailClient.sendEmail(request)
-  }
+  def sendEmailVerificationMessage(to: String, record: Verification)(implicit req: Request[_], ec: ExC): Future[EmailResponse]
 
   def saveVerificationRecord(userId: String, email: String, accType: String)(implicit ec: ExC): Future[Option[Verification]] = {
     val record = Verification(
@@ -93,7 +47,7 @@ trait EmailService {
     )
     verificationStore.createVerificationRecord(record) map {
       case MongoSuccessCreate => Some(record)
-      case MongoFailedCreate  => None
+      case MongoFailedCreate => None
     }
   }