Project

General

Profile

Actions

Patch #16069

open

Allow configuration of API limit instead of hardcoding at 100

Added by Domingo Galdos about 10 years ago. Updated 9 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
REST API
Start date:
Due date:
% Done:

0%

Estimated time:

Description

The below tested patch makes the previously harcoded limit of 100 items in a RESTful API response configurable in the Administration GUI. The default value is still 100. This is based on 2.4.1.stable.

This is quite useful since it is nice and simple to just get the full response in a single page rather than having to traverse pagination, and a limit of a bit more than 100 is still very performant for many sites -- the limit beyond which performance starts to degrade really depends on the particular site's usage, so should be controllable by the local administrator.

Index: app/controllers/application_controller.rb
===================================================================
--- app/controllers/application_controller.rb       (.../trunk/app) (revision 13385)
+++ app/controllers/application_controller.rb       (.../branches/code-8112/app)    (revision 13389)
@@ -519,8 +519,8 @@
     limit = options[:limit].to_i
     if limit < 1
       limit = 25
-    elsif limit > 100
-      limit = 100
+    elsif limit > Setting.api_limit.to_i
+      limit = Setting.api_limit.to_i
     end
     if offset.nil? && options[:page].present?
       offset = (options[:page].to_i - 1) * limit
Index: app/views/settings/_general.html.erb
===================================================================
--- app/views/settings/_general.html.erb    (.../trunk/app) (revision 13385)
+++ app/views/settings/_general.html.erb    (.../branches/code-8112/app)    (revision 13389)
@@ -24,6 +24,8 @@

 <p><%= setting_select :wiki_compression, [['Gzip', 'gzip']], :blank => :label_none %></p>

+<p><%= setting_text_field :api_limit, :size => 6 %></p>
+
 <p><%= setting_text_field :feeds_limit, :size => 6 %></p>

 <p><%= setting_text_field :file_max_size_displayed, :size => 6 %> <%= l(:"number.human.storage_units.units.kb") %></p>
Index: config/settings.yml
===================================================================
--- config/settings.yml        (revision 13387)
+++ config/settings.yml        (revision 13389)
@@ -73,6 +73,9 @@
   default: localhost:3000
 protocol:
   default: http
+api_limit:
+  format: int
+  default: 100
 feeds_limit:
   format: int
   default: 15
Index: config/locales/en.yml
===================================================================
--- config/locales/en.yml      (revision 13387)
+++ config/locales/en.yml      (revision 13389)
@@ -352,6 +352,7 @@
   setting_host_name: Host name and path
   setting_text_formatting: Text formatting
   setting_wiki_compression: Wiki history compression
+  setting_api_limit: Maximum number of items returned in a JSON/XML API response
   setting_feeds_limit: Maximum number of items in Atom feeds
   setting_default_projects_public: New projects are public by default
   setting_autofetch_changesets: Fetch commits automatically


Files

0001-domingo-galdos.patch (1.93 KB) 0001-domingo-galdos.patch vzvu 3k6k, 2020-10-24 09:56
0002-add-test.patch (658 Bytes) 0002-add-test.patch vzvu 3k6k, 2020-10-24 09:56
0004-simplify-label.patch (643 Bytes) 0004-simplify-label.patch vzvu 3k6k, 2020-10-24 09:56
0003-move-to-api-tab.patch (905 Bytes) 0003-move-to-api-tab.patch vzvu 3k6k, 2020-10-24 09:56
0005-quickfix-allow-less-than-25.patch (544 Bytes) 0005-quickfix-allow-less-than-25.patch vzvu 3k6k, 2020-10-24 09:56
Add-validation-to-api-limit.patch (1.41 KB) Add-validation-to-api-limit.patch Mizuki ISHIKAWA, 2020-10-26 08:08

Related issues

Related to Redmine - Feature #38173: Increase limit of entries in json/xml and, preferably, make it configurableClosed

Actions
Has duplicate Redmine - Feature #7539: Rest-API limit of items not 100, but changeable by the administratorClosed2011-02-04

Actions
Has duplicate Redmine - Defect #25555: Make api paging limit configurableClosed

Actions
Has duplicate Redmine - Feature #26553: Overly restricted number of ReST resultsClosed

Actions
Has duplicate Redmine - Feature #33526: Add possibility to configure limit used to retrieve objects for an API responseClosed

Actions
Actions #1

Updated by Domingo Galdos over 9 years ago

Is there anything I can do to help move this patch forward?

Actions #2

Updated by Anton Statutov over 8 years ago

+1

Actions #3

Updated by V Lindell over 8 years ago

+1

Actions #4

Updated by Toshi MARUYAMA over 8 years ago

  • Target version set to 3.2.0
Actions #5

Updated by Go MAEDA over 8 years ago

The user should not be able to set the value of API limit smaller than 100.
Probably some clients suppose that the value is always 100, smaller limit may affect compatibility with such clients.

Actions #6

Updated by Jens Peter Kempkes over 8 years ago

We would also appreciate higher values than 100. Some of our tools need to load all items visible to a user and therefore need to split the requests.

Actions #7

Updated by Sebastian Paluch over 8 years ago

+1

Actions #8

Updated by Jean-Philippe Lang over 8 years ago

  • Target version changed from 3.2.0 to Candidate for next major release
Actions #9

Updated by Jens Peter Kempkes over 8 years ago

Too sad, this patch was shifted.
What is the reason?

We're really waiting for this patch and all seems to be done. Am I wrong?

Actions #10

Updated by Niklaus Jordi about 8 years ago

+1

Actions #11

Updated by Go MAEDA almost 8 years ago

  • Has duplicate Feature #7539: Rest-API limit of items not 100, but changeable by the administrator added
Actions #12

Updated by Dam Dam over 7 years ago

+1

Hi,
This request was posted 2 years ago, so what's up?

Actions #13

Updated by Stephane Evr over 7 years ago

+1

Actions #14

Updated by Go MAEDA almost 7 years ago

  • Has duplicate Defect #25555: Make api paging limit configurable added
Actions #15

Updated by George Brooke almost 7 years ago

Is there anything wrong/non-ideal with this patch which means that it has not been integrated?

Actions #16

Updated by Bohdan Artemchuk almost 7 years ago

+1

Actions #17

Updated by Toshi MARUYAMA over 6 years ago

  • Description updated (diff)
Actions #18

Updated by Toshi MARUYAMA over 6 years ago

  • Has duplicate Feature #26553: Overly restricted number of ReST results added
Actions #19

Updated by Michael Gerz about 6 years ago

What happened to this patch? Nothing?

Actions #20

Updated by Jose Luis Parrilla almost 6 years ago

+1

Actions #21

Updated by Гордеев Алексей over 5 years ago

Silence... :(

Actions #22

Updated by Yuuki NARA about 5 years ago

+1

I think that it should be changeable by the responsibility of the system administrator.

There are many configuration items in the world system that may cause troubles due to setting change contents.
Since there are cases where it is necessary to change these items, it is made public after clearly stating that there is a risk when changing.

It is enough to specify the following points.

If you change the default value, there is a possibility of trouble in the operation of REST using application.
The operation verification responsibility is on Redmine system administrator who changed.

Actions #23

Updated by C F about 5 years ago

+1

Actions #24

Updated by Gianni Cavallotto almost 5 years ago

+1

Actions #25

Updated by int redmine over 4 years ago

+1

Actions #26

Updated by Meiki Neumann over 4 years ago

+1

Actions #27

Updated by r okui over 4 years ago

+1

Actions #28

Updated by Cédric Berger over 4 years ago

+1

Actions #29

Updated by Mischa The Evil almost 4 years ago

  • Has duplicate Feature #33526: Add possibility to configure limit used to retrieve objects for an API response added
Actions #30

Updated by VVD VVD almost 4 years ago

Russian translation of:

setting_api_limit: Maximum number of items returned in a JSON/XML API response

is

setting_api_limit: Максимальное количество элементов, передаваемых в ответе JSON/XML API

Actions #31

Updated by VVD VVD almost 4 years ago

What blocks this change?

Actions #32

Updated by Txinto Vaz over 3 years ago

+1

Actions #33

Updated by VVD VVD over 3 years ago

meineerde from IRC #redmine@freenode ask me to describe my use-case.
Sorry for my poor english, but I try to explain why this feature is important for me.

We are using Redmine for manage projects - tasks, issues, bugtracker. 10-20 active projects with hundreds active issues and 100+ active users - employees and customers. Employees are using "Spent time" in issues to record work time for every issue every day. The employees' salary depends on those records.
We have developed application on Java+Tomcat for create reports with values from "Spent time" fields. It using REST API to get values:

http://redmine.host/time_entries.json?spent_on=><BEGIN_OF_MONTH|END_OF_MONTH&offset=N&limit=10000

For example to get 20000 objects - 200 requests in ~3 minutes even on localhost. But it's faster with limit 10000 - ~20 seconds only.
After end of each month 10+ project managers (I'm one of them) using this application to create reports for each project. The employees salary calculated from this reports. Every mouth we have thousands of "Spent time" entries. It's too long for users wait several minutes for "simple" report, and even more - some browsers timedout connection during this time…

Please, commit this patch!

Actions #34

Updated by vzvu 3k6k over 3 years ago

Looks good. We've wrote a patch based on Domingo Galdos' one (Patch 0001) with Juno NISHIZAKI, sanak and me on Redmine patch meeting 4th.

The changes are as follows:
  • Added a test. (Patch 0002)
  • Moved the configuration value form from the "General" tab to the "API" tab and shorten the label name. (Patch 0003, 0004)
  • In the current implementation, if the limit parameter is less than 1, Redmine sets limit to 25. the limit is less than 25, the conditional statement is modified to be replaced by another one. (Patch 0005)

Go MAEDA wrote:

The user should not be able to set the value of API limit smaller than 100.
Probably some clients suppose that the value is always 100, smaller limit may affect compatibility with such clients.

Sounds reasonable, but currently the setting page doesn't have any validations. Do you have any good idea to apply the constraint?

Actions #35

Updated by Mizuki ISHIKAWA over 3 years ago

vzvu 3k6k wrote:

Go MAEDA wrote:

The user should not be able to set the value of API limit smaller than 100.
Probably some clients suppose that the value is always 100, smaller limit may affect compatibility with such clients.

Sounds reasonable, but currently the setting page doesn't have any validations. Do you have any good idea to apply the constraint?

The attached patch adds validation, though it may not be the best way to write it.

Actions #36

Updated by Marius BĂLTEANU over 3 years ago

  • Target version changed from Candidate for next major release to 4.2.0

The final result of the patch series posted by vzvu 3k6k and Mizuki looks good to me and all the tests pass: https://gitlab.com/redmine-org/redmine/-/pipelines/207877707

I propose to evaluate this for 4.2.0, but we should have Jean-Philippe agree on this because he rejected the patch some time ago. The feature is useful, for sure.

Actions #37

Updated by Roberto Mendes about 3 years ago

I need this. To get more than 100 items.

Actions #38

Updated by Marius BĂLTEANU almost 3 years ago

  • Target version changed from 4.2.0 to 5.0.0
Actions #39

Updated by Marius BĂLTEANU about 2 years ago

  • Target version changed from 5.0.0 to Candidate for next major release
Actions #40

Updated by Go MAEDA about 1 year ago

  • Related to Feature #38173: Increase limit of entries in json/xml and, preferably, make it configurable added
Actions #41

Updated by Maciej Czechowicz 9 months ago

Is it possible to push this request? I have application which sends 13 requests to Redmine and takes internity for that.

Actions

Also available in: Atom PDF