Redmine

Great job, I've been wanting to create some sort of email input for Redmine but haven't found the time. So this is a command line script that is feed each email message as text (STDIN)?

Suggestions¶

If you use Rails ActionMailer, you can save a lot of the manual parsing and even support attachments. http://wiki.rubyonrails.org/rails/pages/HowToReceiveEmailsWithActionMailer

Requiring the entire Rails environment for each email can get really costly, especially under heavy loads. I was planning on having Redmine check an IMAP account regularly and batch process emails from it. It will save the startup time and would also make setup easier (i.e. just setup an IMAP account).

I'd love to see this implemented!

This plugin seems pretty close to doing as described, with ActionMailer and a rake task for running in a cron job.

http://github.com/mully/redmine_ticket_emailer/tree/master

It seems there might be some elements of it that are holdovers from the plugin it was maybe adapted from (Simple CI, maybe)? And it doesn't have the "add Journal" function shown in the script attached.

In short, it might only be "almost" right, but it seems like a darned good start. Has anyone seen it before? Tried it?

Randy Harmon wrote:

In short, it might only be "almost" right, but it seems like a darned good start. Has anyone seen it before? Tried it?

Yea I've taken a look at it and it is great start. This week I'm going to try to merge that plugin with the parser from bugmail.rb and see what I can get.

Eric Davis wrote:

I've just forked mully's plugin and [...]

Sweet. There might be some good stuff in this fork too. I don't agree with all of it (user creation from email? yuck), but some might be good.

http://github.com/rollick/redmine_ticket_emailer/commit/ac2b1050d67527aa12106949821ba639e2d89dd0

Randy Harmon wrote:

I don't agree with all of it (user creation from email? yuck)

This is my fork. User creation was ok for me as I was filtering the incoming emails. Yes, it is still yuck.

How would you propose dealing with users/authors not in the system? I can't ignore them, nor can I set the author to an "anonymous" user as I need the email (and name) of the issue author.

Mark Gallop wrote:

How would you propose dealing with users/authors not in the system? I can't ignore them, nor can I set the author to an "anonymous" user as I need the email (and name) of the issue author.

For my case, it's safe to ignore them because there are only a few people who will be emailing the system.

If you wanted to create a user (say for automated bug reports) then the idea that you have is fine. I got some more time allocated to work on it next week, and will be adding a configuration panel which will allow us to get rid of the yml file. I think it would be great to have some options in there for creating new accounts:

• Allow new users to be added to the project
• Role to add new users at

Eric

Clearly some people would want to capture user-info. Just because I don't want it for my environment doesn't mean it's bad. I do wonder if it's a vector for spammers to exploit, but some careful thought (probably already done by you, Mark) can probably address that.

Another method would be to augment the model with a field to capture this information; assign tickets to the "incoming email" user; and allow people (with sufficient privileges) to promote that person to a User object (fixing up Reported-By in the bargain).

Eric, that could be an additional option in the configuration panel, for example:

When the From: address is not recognized:

• Ignore the incoming email
• Bounce the email back to the sender
• Accept the email, and:
  • Add the user to the User table
    • With user role <select>
  • Assign to 'anonymous' [could <select> username]
    • <checkbox> Capture From: address and allow moderated adding of user accounts

Come to think of it, this last checkbox may as well be full-time behavior, if that moderated-adding feature does get implemented.

Issue #1142 is a request for CC: field for additional email addresses. I know 'watch'ing an issue is essentially the same, but there's currently no way of adding someone else that I know would be interested. If you implemented the CC: feature, it could be used when the bug reporter is not a registered user. This might be a simpler solution than monkeying around with user tables.

And yes, this could be a vector for spam, so mail moderation would be important if the incoming email address is publicly available.

Carl Nygard wrote:

Issue #1142 is a request for CC: field for additional email addresses. I know 'watch'ing an issue is essentially the same, but there's currently no way of adding someone else that I know would be interested. If you implemented the CC: feature, it could be used when the bug reporter is not a registered user. This might be a simpler solution than monkeying around with user tables.

Oh, interesting. "Add to CC:" could be another reasonable option.

And yes, this could be a vector for spam, so mail moderation would be important if the incoming email address is publicly available.

To be more explicit, I was thinking less about the risk of spam harvesters, and more about a spammer actively somehow using it as a vector for "broadcasting their pirate signal" ;)

Randy Harmon wrote:

To be more explicit, I was thinking less about the risk of spam harvesters, and more about a spammer actively somehow using it as a vector for "broadcasting their pirate signal" ;)

As was I. This is similar to the problems the bloggers face with link farms and spam comments. Given an email address to a redmine site, I could spam all the issues with comments trying to convince JP he needs pharmaceutical "help", or whatever. Hence the moderation requirements for all changes made via incoming email.

Or at least an option to moderate, since I don't necessarily want to be bothered with moderation if the email methods are internal only, or restricted to small set of clients or something similar.

I've checked in basic support for creating tickets via email in r1568.

IMHO, fetching emails from an IMAP server and parsing them to create/reply a ticket should be dissociated, as there can be other means to retreive the emails but the parsing logic is the same.

r1568 adresses the second part only (email parsing). For now, a simple rake task lets you read an email from STDIN. Another rake task will be added to fetch emails from an IMAP server and submit them to MailHandler.receive.

Looked at your patch. There's no way to set the status, priority, or category. Or the tracker, but there's a TODO comment about that. It might be easy to refactor the code that grabs the value from the options[] or from the text via regex (especially if the option name matches the tag name). I'll work on a patch if you like...

Nice how you can grab the issue info from a reply. But wouldn't that get broke if someone translates the email notifications differently (i.e. removes the []'s that you're looking for in the regex)?

Carl Nygard wrote:

I'll work on a patch if you like...

Your patch is welcome :-)

Nice how you can grab the issue info from a reply. But wouldn't that get broke if someone translates the email notifications differently (i.e. removes the []'s that you're looking for in the regex)?

[]'s are hard-coded in source:trunk/app/models/mailer.rb@1483#L45, not in translation files.
But if you have a better solution, feel free to submit it.

Jean-Philippe Lang wrote:

IMHO, fetching emails from an IMAP server and parsing them to create/reply a ticket should be dissociated, as there can be other means to retreive the emails but the parsing logic is the same.

I agree, but in the interest of time I kept all the logic in the same class because that's how mully had it in the plugin. Ideally it would be great to see the emailer support IMAP, POP, and standard input. Perhaps by subclassing MailHandler or adding modules for the specific server behavior.

I agree with JP that things need to be separated. Even moreso, I really don't understand why this is implemented as a plugin? How is the plugin supposed to poll for pop/imap input? I don't understand why you aren't writing a script (or rake task, as JP had it) that makes it easy for the admin to integrate with their mail server (sendmail, postfix, etc.)? Emails appearing to be processed seems to dictate something that would handle that even, so what am I not getting?

I'm going to guess an answer to my own question... the plugin is for configuration, the rake task is for the cron. If that's true... then you're putting a restriction on how this can be implemented. For example, I'd like to put it as a mail handler in my sendmail configuration, on a different machine. How does it get the config info from the yml file? Whatever you do, I think it should be flexible to allow:

• sendmail/postfix script handler on separate server
• crontab with imap/postfix

PS: Eric, thanks for the credit on github, but why are you pointing to redmine.shaneandpeter.com instead of this issue on redmine.org? And why does it have a certificate for plesk/SWsoft that mismatches? That just looks fishy.

Carl Nygard wrote:

I agree with JP that things need to be separated. Even moreso, I really don't understand why this is implemented as a plugin?

The only reasons I worked on it as a plugin are:

• There was some existing work by mully on it
• I don't have commit rights to the svn repository

How is the plugin supposed to poll for pop/imap input? I don't understand why you aren't writing a script (or rake task, as JP had it) that makes it easy for the admin to integrate with their mail server (sendmail, postfix, etc.)? Emails appearing to be processed seems to dictate something that would handle that even, so what am I not getting?

The plugin comes with a Rake task that will check the IMAP server and process all new mail. I outlined why a shell script or piping from STDIN in my first comment

I'm going to guess an answer to my own question... the plugin is for configuration, the rake task is for the cron.

No, the plugin would do both the configuration and provide a rake task to run. Ideally, I would want the plugin to support other input methods like POP or STDIN but they haven't been implemented yet.

PS: Eric, thanks for the credit on github, but why are you pointing to redmine.shaneandpeter.com instead of this issue on redmine.org?

I"m sorry, it was a typo and has been corrected. Shane and Peter are the people who were sponsoring the work on it.

And why does it have a certificate for plesk/SWsoft that mismatches? That just looks fishy.

Thats the default certificate for a Plesk server. They haven't had the need to upgrade it yet since it's only used for internal projects right now.

Jean-Philippe Lang wrote:

We could also add a simple action on some controller so that a separate email server could post raw incoming emails to the application via a HTTP request. Unlike the rake task that read emails from STDIN, this would avoid to reload the environment each time an email is received.

That would be great. Sounds more like a REST API than a mailer.

Jean-Philippe Lang wrote:

We could also add a simple action on some controller so that a separate email server could post raw incoming emails to the application via a HTTP request. Unlike the rake task that read emails from STDIN, this would avoid to reload the environment each time an email is received.

I like that idea. Configuration-wise, it's not so cumbersome as setting up cron tasks, and the email changes get processed as soon as the email appears.

Carl, this idea was taken from Request Tracker.
It was committed in r1584. Doc is available in RedmineReceivingEmails.

I'm looking at using this new receive email functionality for supporting messages to forums. I'm most interested in the IMAP option and have been looking at the rake task. After looking at the rake task I'm having a hard time understanding how the checks for email create or update issues. For example, the last line of the IMAP rake task is: Redmine::IMAP.check(imap_options, options) This call appears to do the check on the IMAP server but nothing more. I was thinking there would be a call to MailHandler.receive. Am I missing something?

Joshua Hoover wrote:

Am I missing something?

Yes, you are.
See what Redmine::IMAP.check does: source:/trunk/lib/redmine/imap.rb@1572#L23
Here you'll find the call to MailHandler.receive.

Jean-Philippe Lang wrote:

Joshua Hoover wrote:

Am I missing something?

Yes, you are.
See what Redmine::IMAP.check does: source:/trunk/lib/redmine/imap.rb@1572#L23
Here you'll find the call to MailHandler.receive.

To help out anyone who doesn't know how Rails and ActionMailer work the process is like this:

1. Redmine::IMAP.check connects to IMAP server
2. Each new message is sent to MailHandler.receive
3. MailHandler.receive will parse the email and attachments then
4. MailHandler.receive will use Redmine's MailHandler.dispatch to update the issue

Hello, If redmine would recognize otrs forwards mails it would be beautiful.

Could you add a option config / add to code , so #20081110xxxx is not a redmine ticket ?

The problem is when you forward a ticket from otrs system to redmine.

Thank you

Hello, I've done creating issue via email in Readmine. Now I want to update existing issue via email by using issue number or something else. Do I've to change script in redm-mailhandler.rb file? or any other suggestions regarding this?

Ahammad Fekri wrote:

Hello, I've done creating issue via email in Readmine. Now I want to update existing issue via email by using issue number or something else. Do I've to change script in redm-mailhandler.rb file? or any other suggestions regarding this?

You can update Notes through mail. You just have to add Redmine ID in subject e.g. [#123]. Regarding updating other fields I am still testing this feature.

I am using this feature, really amazing and saves lot of time.
I have read doc from RedmineReceivingEmails..