Project

General

Profile

Actions
Copy link

Feature #35073

closed

Escape values in LIKE statements to prevent injection of placeholders (_ or %)

Added by Jens Krämer over 3 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Marius BĂLTEANU
Category:
Database
Target version:
5.0.0
Start date:
Due date:
% Done:

0%

Estimated time:
Resolution:

Description

While not technically a security risk, LIKE queries with lots of placeholders can result in high database load, very slow query execution and therefore are a possible vector for denial of service attacks. Further, escaping the wildcard characters in actual query values now allows to actually search for values containing these characters.

The attached patches, which have been extracted from Planio are:

  • The first patch removes '%' signs from query strings used in the search test case. These did not matter until now since they just resulted in queries like field LIKE '%%value%%', but now would be looking for a value with literal percent signs. Tests pass before and after that change.
  • Patches 2-4 add sanitize_sql_like calls to the various places where we currrently generate sql LIKE statements. Corresponding tests are included.
  • The last patch is just a cosmetic change that replaces a .send with a direct call since the called method is now public.

Files

Download all files
0005-sanitize_sql_for_conditions-is-now-public.patch (1.02 KB) 0005-sanitize_sql_for_conditions-is-now-public.patch Jens Krämer, 2021-04-12 08:44
0004-use-sanitize_sql_like-in-Query-sql_contains.patch (1.75 KB) 0004-use-sanitize_sql_like-in-Query-sql_contains.patch Jens Krämer, 2021-04-12 08:44
0003-use-sanitize_sql_like-in-like-scopes.patch (6.13 KB) 0003-use-sanitize_sql_like-in-like-scopes.patch Jens Krämer, 2021-04-12 08:44
0002-use-sanitize_sql_like-on-search-tokens.patch (2.34 KB) 0002-use-sanitize_sql_like-on-search-tokens.patch Jens Krämer, 2021-04-12 08:44
0001-removes-signs-from-test-strings-in-search-test.patch (1.16 KB) 0001-removes-signs-from-test-strings-in-search-test.patch Jens Krämer, 2021-04-12 08:44
0001-tokenize-query-strings-for-Issue.like-and-Query-sql_.patch (3.93 KB) 0001-tokenize-query-strings-for-Issue.like-and-Query-sql_.patch Jens Krämer, 2021-07-07 08:42
35073-sqlite-fix.patch (2.29 KB) 35073-sqlite-fix.patch Go MAEDA, 2021-10-06 16:53
35073-sqlite-fix.patch (2.32 KB) 35073-sqlite-fix.patch Go MAEDA, 2021-10-07 16:12

Related issues

Related to Redmine - Feature #13347: Filtering by issue subject with wildcardNew

Actions
Related to Redmine - Defect #19786: '%' and '_' are treated as SQL wildcards in issue filterClosed

Actions
Blocks Redmine - Feature #35764: Multiple search terms in the "contains" operator of text filtersClosedMarius BĂLTEANU

Actions
Actions
Copy link
 #1

Updated by Go MAEDA over 3 years ago

  • Related to Feature #13347: Filtering by issue subject with wildcard added
Actions
Copy link
 #2

Updated by Go MAEDA over 3 years ago

  • Related to Defect #19786: '%' and '_' are treated as SQL wildcards in issue filter added
Actions
Copy link
 #3

Updated by Go MAEDA over 3 years ago

I actually find the current behavior useful. For example, when I search for photos using the "File" filter, I use the query string "dsc%.jpg".

If Redmine prohibits the inclusion of "_" or "%" in the query string, then I want some alternative.

Actions
Copy link
 #4

Updated by Jens Krämer over 3 years ago

In general I believe these SQL wild cards are nowhere documented in the Redmine context (please correct me if I'm wrong), and therefore more or less a 'hidden feature' for power users who are familiar with SQL. Given the potential of abuse (and the current inability to search for terms that actually contain these wildcards), I still believe it is the right way to not expose such a low level database feature.

However I get your point and in fact I am right now revisiting this because some Planio power users do miss the SQL wild cards as well :)

The alternative I am experimenting with right now is breaking up the user's query string into tokens like the global search does, and use these to build a query with multiple LIKE clauses that are combined with AND. I'm attaching a preliminary patch that implements this for the Issue.like scope (used by the autocompleter) and the Query#sql_contains method (which should cover all query filters). Do you think that's a viable approach?

Actions
Copy link
 #5

Updated by Go MAEDA over 3 years ago

Jens Krämer wrote:

In general I believe these SQL wild cards are nowhere documented in the Redmine context (please correct me if I'm wrong), and therefore more or less a 'hidden feature' for power users who are familiar with SQL. Given the potential of abuse (and the current inability to search for terms that actually contain these wildcards), I still believe it is the right way to not expose such a low level database feature.

I agree with you. Actually, I opened #13347 that reports about it 6 years ago.

The alternative I am experimenting with right now is breaking up the user's query string into tokens like the global search does, and use these to build a query with multiple LIKE clauses that are combined with AND. I'm attaching a preliminary patch that implements this for the Issue.like scope (used by the autocompleter) and the Query#sql_contains method (which should cover all query filters). Do you think that's a viable approach?

I tried out the patch and found it is really useful. I think it is a great improvement to be able to do OR searches with multiple keywords, which is currently not possible with the current filters. Although it is a breaking change, it is worth more than that.

Actions
Copy link
 #6

Updated by Go MAEDA over 3 years ago

  • Target version set to Candidate for next major release
Actions
Copy link
 #7

Updated by Go MAEDA over 3 years ago

  • Blocks Feature #35764: Multiple search terms in the "contains" operator of text filters added
Actions
Copy link
 #8

Updated by Marius BĂLTEANU over 3 years ago

  • Target version changed from Candidate for next major release to 5.0.0
Actions
Copy link
 #9

Updated by Go MAEDA over 3 years ago

Jens Krämer wrote:

The alternative I am experimenting with right now is breaking up the user's query string into tokens like the global search does, and use these to build a query with multiple LIKE clauses that are combined with AND. I'm attaching a preliminary patch that implements this for the Issue.like scope (used by the autocompleter) and the Query#sql_contains method (which should cover all query filters). Do you think that's a viable approach?

I have extracted attachment:0001-tokenize-query-strings-for-Issue.like-and-Query-sql_.patch to a separete issue #35764, because it adds a new feature but others fix undesirable behavior as described in the subject.

Actions
Copy link
 #10

Updated by Marius BĂLTEANU about 3 years ago

  • Status changed from New to Closed
  • Assignee set to Marius BĂLTEANU

All five patches committed, thanks.

Actions
Copy link
 #11

Updated by Go MAEDA about 3 years ago

  • Status changed from Closed to Reopened

After applying the patch, some tests fail when the backend database is SQLite. I will post a fix soon.

Actions
Copy link
 #12

Updated by Go MAEDA about 3 years ago

Go MAEDA wrote:

After applying the patch, some tests fail when the backend database is SQLite. I will post a fix soon.

I am attaching a patch that fixes the issue. There is no default escape character in SQLite, so it must be specified explicitly using an ESCAPE clause.

Actions
Copy link
 #13

Updated by Marius BĂLTEANU about 3 years ago

Go MAEDA wrote:

Go MAEDA wrote:

After applying the patch, some tests fail when the backend database is SQLite. I will post a fix soon.

I am attaching a patch that fixes the issue. There is no default escape character in SQLite, so it must be specified explicitly using an ESCAPE clause.

I don't have a local environment with SQLite to test it. Is it ok to commit the patch directly?

Actions
Copy link
 #14

Updated by Go MAEDA about 3 years ago

Updated the fix for SQLite. The previous patch fails when the database is MS SQL Server.

Marius BALTEANU wrote:

I don't have a local environment with SQLite to test it. Is it ok to commit the patch directly?

Yes, it can be committed as is. The new patch was tested with all supported databases (PostgreSQL, MySQL, SQLite, and MS SQL Server).

Actions
Copy link
 #15

Updated by Marius BĂLTEANU about 3 years ago

  • Subject changed from escape values in LIKE statements to prevent injection of placeholders (_ or %) to Escape values in LIKE statements to prevent injection of placeholders (_ or %)
  • Status changed from Reopened to Closed

All tests passed now.

Actions
Copy link
 #16

Updated by Go MAEDA over 2 years ago

  • Tracker changed from Patch to Feature
Actions
Copy link

Also available in: Atom PDF