Michael ReinschOverview
Processing incoming email with your Rails application (or any other web application) isn't that complicated, but there are several strategies on how to get the incoming email to your code. We've tried various approaches, and this revision is the best solution we've found so far.
In a nutshell: Our self-hosted MTA, Postfix, receives incoming email for Doorkeeper and relays it to the Rails application using a normal HTTP request.
Show me the code
Within Postfix, we route received email to one of the above aliases, which spawns the email_handler script with the appropriate arguments: The first argument is used to construct the API endpoint. The second argument allows for multiple environments; you can easily add a staging environment for instance.
The email_handler script then uses curl to relay the email to the API endpoint.
The controller receiving the email then invokes the standard receive() function of a mailer, which processes the email as usual.
Discussion of this approach
Previous to this approach, we had Postfix simply spawn a new Rails runner and feed it the incoming email, as documented in the Rails Guides.
We found that to not work very well. A huge issue is the startup time required by Rails, and spawning a Rails process from Postfix can get rather tricky when using rvm and bundler. The resulting shell scripts were also outside of our testing framework and relied heavily on the production system setup, so they could break easily without someone noticing it immediately. For reporting issues during processing, extra code was required to catch any exceptions in the mailer's receive() method and send them to coalmine (our favorite error reporting tool).
Now in this latest revision, we feel that we've addressed all of the above issues, without introducing any extra daemons or other 3rd party components. Things got actually simpler and more robust. The exception reporting for instance is using the same mechanism as for the rest of the code base, and all API endpoints are covered by unit tests. In case of any communication issues or temporary outages, the email_handler script will have Postfix defer the email and try again later; using logentries we can monitor the frequency of that happening. And last but not least, processing is much faster and can be scaled up as usually.