Project

General

Profile

Actions

Feature #24808

open

OAuth2 support for Redmine API Apps (OAuth2 Provider)

Added by Jan from Planio www.plan.io about 7 years ago. Updated 4 months ago.

Status:
New
Priority:
Normal
Category:
REST API
Start date:
Due date:
% Done:

0%

Estimated time:
Resolution:

Description

I think, we should make Redmine an OAuth2 provider, so that client apps using Redmine's API can ask users to authenticate rather than asking for an API key. Another goal would be to limit access to API client apps, so that not all apps gain access to the full scope of data a user has access to.

I'm working on this currently and I will hopefully be able to submit a patch soon. Here's a first screenshot:


Files

redmine_oauth2_provider.png (330 KB) redmine_oauth2_provider.png Jan from Planio www.plan.io, 2017-01-11 11:54
my_account.png (80.6 KB) my_account.png Jan from Planio www.plan.io, 2017-01-18 17:35
authorized_apps.png (206 KB) authorized_apps.png Jan from Planio www.plan.io, 2017-01-18 17:35
auth_prompt.png (218 KB) auth_prompt.png Jan from Planio www.plan.io, 2017-01-18 17:35
apps.png (261 KB) apps.png Jan from Planio www.plan.io, 2017-01-18 17:35
0001-Use-named-routes-for-search-in-base-layout.patch (2.83 KB) 0001-Use-named-routes-for-search-in-base-layout.patch Jan from Planio www.plan.io, 2017-01-18 17:46
0002-Prevent-hash-type-URLs-from-being-namespaced-in-Menu.patch (2.03 KB) 0002-Prevent-hash-type-URLs-from-being-namespaced-in-Menu.patch Jan from Planio www.plan.io, 2017-01-18 17:46
0004-Redmine-style-UI-for-Doorkeeper-OAuth2-provider.patch (15.7 KB) 0004-Redmine-style-UI-for-Doorkeeper-OAuth2-provider.patch Jan from Planio www.plan.io, 2017-01-18 17:46
0005-Add-optional-scope-parameter-to-Role-allowed_to.patch (4.27 KB) 0005-Add-optional-scope-parameter-to-Role-allowed_to.patch Jan from Planio www.plan.io, 2017-01-18 17:46
0006-Use-Redmine-s-permissions-as-OAuth2-scopes.patch (7.87 KB) 0006-Use-Redmine-s-permissions-as-OAuth2-scopes.patch Jan from Planio www.plan.io, 2017-01-18 17:46
0003-Add-OAuth2-provider-capability-using-doorkeeper-gem.patch (9.81 KB) 0003-Add-OAuth2-provider-capability-using-doorkeeper-gem.patch Jan from Planio www.plan.io, 2017-01-29 16:23
0001-oauth-Use-named-routes-in-base-layout-and-account-si.patch (4.39 KB) 0001-oauth-Use-named-routes-in-base-layout-and-account-si.patch Jens Krämer, 2020-07-21 13:05
0002-oauth-Prevent-hash-type-URLs-from-being-namespaced-i.patch (2.04 KB) 0002-oauth-Prevent-hash-type-URLs-from-being-namespaced-i.patch Jens Krämer, 2020-07-21 13:05
0003-oauth-Add-OAuth2-provider-capability-using-doorkeepe.patch (10.1 KB) 0003-oauth-Add-OAuth2-provider-capability-using-doorkeepe.patch Jens Krämer, 2020-07-21 13:05
0004-oauth-Redmine-style-UI-for-Doorkeeper-OAuth2-provide.patch (15.6 KB) 0004-oauth-Redmine-style-UI-for-Doorkeeper-OAuth2-provide.patch Jens Krämer, 2020-07-21 13:05
0005-oauth-Add-optional-scope-parameter-to-Role-allowed_t.patch (4.23 KB) 0005-oauth-Add-optional-scope-parameter-to-Role-allowed_t.patch Jens Krämer, 2020-07-21 13:05
0006-oauth-Use-Redmine-s-permissions-as-OAuth2-scopes.patch (10.1 KB) 0006-oauth-Use-Redmine-s-permissions-as-OAuth2-scopes.patch Jens Krämer, 2020-07-21 13:05
0007-oauth-adds-system-test-to-test-the-oauth-provider-ca.patch (4.68 KB) 0007-oauth-adds-system-test-to-test-the-oauth-provider-ca.patch Jens Krämer, 2020-07-21 13:05
0002-Prevent-hash-type-URLs-from-being-namespaced-in-Menu.patch (2.03 KB) 0002-Prevent-hash-type-URLs-from-being-namespaced-in-Menu.patch Jens Krämer, 2020-08-27 05:04
0001-Use-named-routes-in-base-layout-and-account-sidebar.patch (4.38 KB) 0001-Use-named-routes-in-base-layout-and-account-sidebar.patch Jens Krämer, 2020-08-27 05:04
0003-Add-optional-scope-parameter-to-Role-allowed_to.patch (4.22 KB) 0003-Add-optional-scope-parameter-to-Role-allowed_to.patch Jens Krämer, 2020-08-27 05:04
0004-Add-OAuth2-provider-capability-using-doorkeeper-gem.patch (42.1 KB) 0004-Add-OAuth2-provider-capability-using-doorkeeper-gem.patch Jens Krämer, 2020-08-27 05:04
0003-Add-optional-scope-parameter-to-Role-allowed_to.patch (4.22 KB) 0003-Add-optional-scope-parameter-to-Role-allowed_to.patch Jens Krämer, 2021-04-13 08:16
0004-Add-OAuth2-provider-capability-using-doorkeeper-gem.patch (42.2 KB) 0004-Add-OAuth2-provider-capability-using-doorkeeper-gem.patch Jens Krämer, 2021-04-13 08:16
Actions #1

Updated by Jan from Planio www.plan.io about 7 years ago

The attached patch series implements full OAuth2 provider support for Redmine.

Background

OAuth2 is a widely adopted protocol for granting access to API client applications. More information can be found here:

Screenshots

Here are a few screenshots that show how it looks like:


Admins are able to create/modify/delete OAuth2 client apps from the user interface.


OAuth2 App credentials are generated and can then be used in API clients.


In order for an App to gain access to Redmine, it must ask the user for permission.


Regular users are able to see which apps currently have access to their data from their My Account area.


Regular users are able to revoke access to individual apps.

Live Demo

We are providing a live demo server and client via these links:

https://server.redmine-oauth.planio.org

Username/Password is admin:oauth2. The content on this server will be reset every 60 minutes. Feel free to create your own OAuth2 applications via the Admin section, but please don't modify/delete the "Sinatra Client App".

https://client.redmine-oauth.planio.org

You can use this app to try out the authentication/authorization flow. Feel free to create your own user accounts on the Redmine server for this.

The /issues API will only work if the view_issues scope is requested. If only the standard scopes are requested, you will see an error here (on purpose).

Feel free to download the client app code from our Planio repository to try out everything locally.

The patch series

  • 0001 changes the base layout to use named routes. The old hash-style routes (e.g. {controller => 'search', :action => 'index'}) would get namespaced when the layout is used in a namespaced controller in a plugin or engine which is the case with the Doorkeeper gem introduced later.
  • 0002 changes the MenuManager in a similar way and prevents the rendered menu links from getting namespaced in the above scenario.
  • 0003 adds the Doorkeeper Gem and integrates it with Redmine in the relevant places. You could apply patches 0001-0003 only to get functioning OAuth2 provider support already. In detail, what happens is:
    • Gemfile - Gem is added in
    • app/controllers/application_controller.rb - Doorkeeper is used as a new optional authentication mechanism that is tried when regular Redmine API auth fails
    • app/views/my/account.html.erb - Link to Doorkeeper's views for managing a user's authorized apps
    • config/initializers/doorkeeper.rb - Configure Doorkeeper so that it ties in with Redmine's user and admin authentication
    • config/routes.rb - Add Doorkeeper specific routes and root_url which is needed by it
    • db/migrate/20170107092155_create_doorkeeper_tables.rb - Migrations to add Doorkeeper tables
    • lib/redmine.rb - Link to Doorkeeper's views for managing available apps within the admin section
    • public/stylesheets/application.css - Icon for Apps
    • test/unit/lib/redmine/i18n_test.rb - Fix locale counting in tests because doorkeeper-i18n introduces languages unknown to Redmine
  • 0004 - integrates Doorkeeper further by overriding all views with Redmine compatible markup and makes use of Redmine's deny_access and require_login methods which become available once the Doorkeeper controllers are set to inherit from Redmine's ApplicationController
  • 0005 - changes Role#allowed_to? so that it can accept an optional scope parameter which can be an array of permission symbols that will be used as a logical AND filter.
  • 0006 - Allows OAuth2 client apps to use Redmine permissions as Scopes in the sense of OAuth2. This way, admins and application developers can limit the abilities of client apps. An app will generally have at most the permissions defined by the App definition or as reqested by the app during the auhtorization step. Of course, the app will never have more permissions than the user is has requested authorization for would have in a normal interactive scenario.

Considerations

Why use Doorkeeper?

Implementing OAuth2 "by hand" seemed like re-inventing the wheel. Building and maintaining such security-critical code is both error-prone and non-trivial. The Doorkeeper Gem is the de-facto standard solution for implementing an OAuth2 provider in Rails. It's tried and tested, well-maintained and used by many high profile apps and services.

Integrating it with Redmine required only minor changes to Redmine's code base itself which should hopefully make maintenance of this functionality quite easy.

Why can only admins create apps?

The decision that only Redmine admins can add new API client applications is debatable, but it felt like the easiest solution for this first version. Allowing regular users to create apps would have required more overridden Doorkeeper controllers, new Redmine permissions (e.g. add_apps, view_apps, destroy_apps, etc.)

In addition to that, enabling/diasbling things like the REST API, JSONP support, etc. is currently also only available to admins, so I thought it would be consistent.

Some translations are missing! Where are the I18n keys?

At the moment, I've included them via the doorkeeper-i18n Gem. Technically, we could pull the locales in to Redmine's code base. However, – in the spirit of open source – I'd advcate in favor of keeping them in the external Gem and working with the Doorkeeper maintainers to improve them if needed. I've fixed two issues with the locales already via pull requests and they were accepted rather quickly.

That's it for now. I am looking forward to your feedback!

Actions #2

Updated by Marius BĂLTEANU about 7 years ago

IMO, I think that the OAuth2.0 provider will be a great addition to Redmine and a feature that we'll use for sure at our future integrations with other apps.

What I like very much is the separation between the users and applications. Now we've some users named like "<application_name>-Generic User" used to authenticate the API calls. Having the possibility to define them as apps and manage their permissions from a different screen is very useful.

Actions #3

Updated by Jan from Planio www.plan.io about 7 years ago

  • Description updated (diff)
Actions #4

Updated by Jan from Planio www.plan.io about 7 years ago

  • File deleted (0003-Add-OAuth2-provider-capability-using-doorkeeper-gem.patch)
Actions #5

Updated by Jan from Planio www.plan.io about 7 years ago

  • File 0003-Add-OAuth2-provider-capability-using-doorkeeper-gem.patch added

Slightly updated version of 0003, using wider columns for scopes

Actions #6

Updated by Jan from Planio www.plan.io about 7 years ago

  • File deleted (0003-Add-OAuth2-provider-capability-using-doorkeeper-gem.patch)
Actions #8

Updated by Akipii Oga about 7 years ago

+1

Actions #9

Updated by Cheyenne Wills over 6 years ago

+1

What is the current "status" of this? Is this kind of planned for a future release?

We have two apps that could benefit by this (I've been looking at the redmine_oauth_provider plugin, but it appears that it doesn't work with the current level of Redmine).

Actions #10

Updated by Stephane Evr about 6 years ago

+1

Actions #11

Updated by Peter Volkov almost 5 years ago

I think that "Needs feedback" is a wrong status here. According to #12827 this status means that this ticket is waiting for author's feedback, and such tickets are invisible for developers. Jan could you updated patchset and Status here?

Actions #12

Updated by Jan from Planio www.plan.io almost 5 years ago

  • Status changed from Needs feedback to New

I'm setting the status to New then, as requested. Ideally, we could get some more feedback from other contributors if the feature is desired and if yes, I'd be happy to rebase the patches on current trunk.

Actions #13

Updated by Bernhard Rohloff almost 5 years ago

I think it can be a nice feature for Redmine and would make it much easier to manage things like bots, dashboards and applications of that kind. IMHO it's definitely worth a rebase.
+1

Actions #14

Updated by James H over 4 years ago

+1

Actions #15

Updated by Keisuke Matsuura over 4 years ago

+1

Actions #16

Updated by Jan S almost 4 years ago

I'm also interested in this.

Actions #17

Updated by J. Pablo Zebraitis almost 4 years ago

+1

Actions #19

Updated by Jan from Planio www.plan.io over 3 years ago

  • Target version set to Candidate for next minor release

Thanks Jens. I would really enjoy seeing this making its way into a future Redmine release and I believe it will help Redmine get more third party apps and integrations!

Actions #20

Updated by Jens Krämer over 3 years ago

Another update to this patch. Notable changes are:

  • updated to Doorkeeper 5.4, which allowed for the following improvements:
    • secrets (tokens, application secret) are now stored as hashes
    • support for PKCE (most relevant for non-confidential clients)
  • introduced an admin scope which allows Administrators to grant admin permissions to client applications
  • fixed a stored CSRF vulnerability that was present in one of the original Doorkeeper templates. It was only exploitable by Administrators but if you're using an older version of this patch, at least update your views according to this doorkeeper commit

We also just released the omniauth-redmine-oauth2 gem (source code at Planio and Github). We also built a small Rails app to demonstrate usage of the gem .

Currently this patch makes two I18n tests fail. This is due to the inclusion of the doorkeeper-i18n gem, which introduces 4 locales that aren't present in Redmine. In general, we would need to decide if we want to include these 3rd party translation at all (they do not cover all of Redmine's locales by a large margin) or if we incorporate them into Redmine. As of now the patch just overrides a few strings to make the wording more Redmine-like.

Due to the Doorkeeper upgrade I was more or less forced to squash the last 4 commits of the previous patch series, so it's down to 4 commits now.

Actions #21

Updated by Marius BĂLTEANU about 3 years ago

  • Target version changed from Candidate for next minor release to Candidate for next major release

Jens, I've started to look to the provided patches and to test this feature. For now, I've committed all 4 patches to the Gitlab instance in order to get the tests results. Besides the I18n test fails, there are some Rubocop warnings that should be fixed.

Am I wrong if I say that patches 0001 and 0002 can be extracted from this issue and delivered as separated ticket? In this way, we will down this feature to 2 patches and it will be easier to maintain/rebase it.

I've assigning this to the next major release version because the changes are too huge for a minor version.

Actions #22

Updated by Jens Krämer almost 3 years ago

thanks for looking into this! I just created #35075 and #35076 with the first two patches of this series. Both should not cause test failures or rubocop warnings. I'll look into these next and update this issue accordingly.

Actions #23

Updated by Jan from Planio www.plan.io almost 3 years ago

Thanks Marius for looking into this. I think it should greatly improve the Redmine API to use state of the art authorization!

Actions #24

Updated by Jens Krämer almost 3 years ago

here are the remaining two patches, updated to the most recent doorkeeper release (5.5.1), rebased on current master, and hopefully with a lot less rubocop warnings.

Actions #25

Updated by James H almost 3 years ago

+100000000000000

Actions #26

Updated by Marius BĂLTEANU almost 3 years ago

  • Assignee set to Marius BĂLTEANU
  • Target version changed from Candidate for next major release to 5.0.0
Actions #27

Updated by Harald Welte about 2 years ago

Excellent proposal, looking forward to seeing this merged for the next major release.

Actions #28

Updated by Marius BĂLTEANU about 2 years ago

  • Target version changed from 5.0.0 to Candidate for next major release

The patches need some changes to work with Rails 6 and zeitwerk autoloader.

Actions #29

Updated by Gael Duret over 1 year ago

Hello All,

We actually use RedMine in our company and we use the IMAP fetching email process to create our tickets. This process will stop working on October first as we are connected to an Office365 server and Microsoft will deactivate the basic authentication.
We are looking for a way to use OAuth authentication.
Our Redmine version:
Environment:
Redmine version 5.0.0.stable
Ruby version 2.6.9-p207 (2021-11-24) [x64-mingw32]
Rails version 6.1.4.7
Environment production
Database adapter Mysql2
Mailer queue ActiveJob::QueueAdapters::AsyncAdapter
Mailer delivery smtp

We do not have any Rugby knowledge in the team, we are looking for some explain / help on this subject ?
Which patch for our version ? How to setup a patch ?

Thanks

Actions #30

Updated by pasquale [:dedalus] 4 months ago

Marius BĂLTEANU Go MAEDA
Could this ticket be included in the activities planned for version 6.0?

Actions

Also available in: Atom PDF