Defect #41434: CSV import fails with error for quoted fields containing newlines and CRLF line endings - Redmine

Actions

Copy link

Defect #41434

open

CSV import fails with error for quoted fields containing newlines and CRLF line endings

Added by Go MAEDA 3 months ago. Updated about 1 month ago.

Status:

New

Priority:

Normal

Assignee:

Category:

Importers

Target version:

6.1.0

Start date:

Due date:

% Done:

Estimated time:

Resolution:

Affected version:

Description

When you attempt to import the attached CSV file (test.csv), the following error message is displayed:

The file is not a CSV file or does not match the settings below (Unquoted fields do not allow new line <"\r\n"> in line 1.)

This error occurs when the CSV file contains values with newline characters ("\n") inside double-quoted fields, and the lines end with DOS-style line endings ("\r\n"). If the file uses UNIX-style line endings (\n), the error does not occur.

This type of CSV file can be generated by Microsoft Excel when cells contain line breaks.

Files

Download all files

test.csv (62 Bytes) test.csv		Go MAEDA, 2024-10-04 23:51
clipboard-202410050652-xaakj.png (131 KB) clipboard-202410050652-xaakj.png		Go MAEDA, 2024-10-04 23:52
spreadsheet.png (16.7 KB) spreadsheet.png		Go MAEDA, 2024-10-04 23:54
41434-csv-crlf.patch (1.17 KB) 41434-csv-crlf.patch		Go MAEDA, 2024-11-04 06:07

Actions

Copy link

Updated by Go MAEDA 3 months ago

I found that the error occurs only when quoted values that contain "\n" exist in the first line.

redmine-app(prod)> CSV.parse(%Q|FieldName1,"Field\nName2"\r\nValue1,"Value2"|)
/Users/maeda/redmines/gems/ruby/3.3.0/gems/irb-1.14.1/lib/irb.rb:1260:in `full_message': Any value after quoted field isn't allowed in line 1. (CSV::MalformedCSVError)

No problem when it exists in the second line or later.

redmine-app(prod)> CSV.parse(%Q|FieldName1,"FieldName2"\r\nValue1,"Value\n2"|)
=> [["FieldName1", "FieldName2"], ["Value1", "Value\n2"]]

Setting the row_sep option also avoids the error.

redmine-app(prod)> CSV.parse(%Q|FieldName1,"Field\nName2"\r\nValue1,"Value2"|,  row_sep: "\r\n")
=> [["FieldName1", "Field\nName2"], ["Value1", "Value2"]]

Actions

Copy link