https://www.redmine.org/https://www.redmine.org/favicon.ico?16793021292016-06-01T07:22:03ZRedmineRedmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=712102016-06-01T07:22:03ZSebastian Paluch
<ul></ul><p>+1</p> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=712652016-06-05T01:13:04ZGo MAEDA
<ul></ul><p>Very interesting feature.<br />Could you add tests? Tests are required to be merged into the core.</p> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=712892016-06-06T10:28:23ZHaihan Ji
<ul><li><strong>File</strong> <a href="/attachments/16089">20160606_auto_select_fields_test.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/16089/20160606_auto_select_fields_test.patch">20160606_auto_select_fields_test.patch</a> added</li></ul><p>Append Unit Test Code.</p> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=714712016-06-13T01:10:22ZGo MAEDA
<ul><li><strong>Target version</strong> set to <i>Candidate for next major release</i></li></ul> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=753882016-12-26T12:54:11ZGo MAEDA
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Needs feedback</i></li></ul><p>I tried this patch but got the following error while processing ImportsController#mapping. Could you test the patch on the current trunk?</p>
<pre>
NameError (undefined local variable or method `issue' for #<ImportsController:0x007fc8be2479b0>
Did you mean? issue_url):
app/controllers/imports_controller.rb:72:in `mapping'
lib/redmine/sudo_mode.rb:63:in `sudo_mode'
</pre> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=950012019-11-26T06:00:12ZYuichi HARADA
<ul><li><strong>File</strong> <a href="/attachments/24362">22913_auto_mapping.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/24362/22913_auto_mapping.patch">22913_auto_mapping.patch</a> added</li></ul><p>I fixed <a class="attachment" href="https://www.redmine.org/attachments/16038">auto_select_fields.patch</a> and <a class="attachment" href="https://www.redmine.org/attachments/16089">20160606_auto_select_fields_test.patch</a> to work with the current trunk(<a class="changeset" title="rubocop: remove test/functional/issues_controller_test.rb from Layout/MultilineHashBraceLayout fi..." href="https://www.redmine.org/projects/redmine/repository/svn/revisions/19312">r19312</a>).<br />I attach a fixed patch.</p> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=950082019-11-26T15:37:10ZJoshua Jobin
<ul></ul><p>Yuichi HARADA wrote:</p>
<blockquote>
<p>I fixed <a class="attachment" href="https://www.redmine.org/attachments/16038">auto_select_fields.patch</a> and <a class="attachment" href="https://www.redmine.org/attachments/16089">20160606_auto_select_fields_test.patch</a> to work with the current trunk(<a class="changeset" title="rubocop: remove test/functional/issues_controller_test.rb from Layout/MultilineHashBraceLayout fi..." href="https://www.redmine.org/projects/redmine/repository/svn/revisions/19312">r19312</a>).<br />I attach a fixed patch.</p>
</blockquote>
<p>This is great. Thank you!<br />Do you find that if custom fields are absent from the default tracker, that they will not match when you load a tracker with more custom fields? I implemented this exactly and found that to be the case.</p> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=953982019-12-20T08:51:09ZGo MAEDA
<ul></ul><p>Joshua Jobin wrote:</p>
<blockquote>
<p>Do you find that if custom fields are absent from the default tracker, that they will not match when you load a tracker with more custom fields? I implemented this exactly and found that to be the case.</p>
</blockquote>
<p>Could you show the steps to reproduce the problem?</p>
<p>I didn't see the problem you pointed out with Yuichi HARADA's patch applied. What I did is disable a custom field only in the first tracker.</p> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=954712020-01-04T01:43:20ZGo MAEDA
<ul><li><strong>Status</strong> changed from <i>Needs feedback</i> to <i>New</i></li><li><strong>Target version</strong> changed from <i>Candidate for next major release</i> to <i>4.2.0</i></li></ul><p>Setting the target version to 4.2.0.</p> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=954862020-01-04T09:45:07ZMarius BĂLTEANU
<ul><li><strong>Assignee</strong> set to <i>Marius BĂLTEANU</i></li></ul><p>Go MAEDA wrote:</p>
<blockquote>
<p>Setting the target version to 4.2.0.</p>
</blockquote>
<p>The patch cannot be committed as it is, please let me review it.</p> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=958852020-01-26T23:15:22ZMarius BĂLTEANU
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-5 priority-4 priority-default closed" href="/issues/28234">Feature #28234</a>: Add CSV Import for Time Entries</i> added</li></ul> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=958892020-01-26T23:46:29ZMarius BĂLTEANU
<ul><li><strong>File</strong> <a href="/attachments/24683">demo_auto_map_fields.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/24683/demo_auto_map_fields.patch">demo_auto_map_fields.patch</a> added</li><li><strong>Assignee</strong> deleted (<del><i>Marius BĂLTEANU</i></del>)</li></ul><p>Sorry for my late review on this.</p>
<p>My main concern with the proposed patch are the hardcoded fields from <code>mapping</code> method (<code>ImportsController</code>). Because of them, the "auto mapping" feature it won't be easily extended by plugins or other models (it is against the generalisation added in <a class="changeset" title="Generalize issues imports (#28234). Extend import controller to support arbitrary imports based ..." href="https://www.redmine.org/projects/redmine/repository/svn/revisions/18145">r18145</a>).</p>
<p>I'm attaching a patch with another solution, where each subclass of the import have a class constant (<code>AUTO_MAPPABLE_FIELDS</code> in my patch) or a method that returns an array with the fields that can be auto-mapped. In <code>ImportsController</code>, the method <code>auto_map_fields</code> uses this constant and try to auto map the fields based on some conditions. This auto mapping uses only the "internal" name of the fields, not labels or translations.</p>
<p>To support labels and translations as well, I see two options:<br />1. We modify the <code>AUTO_MAPPABLE_FIELDS</code> to return a hash with each internal field and his label. For example:<br /><pre>
AUTO_MAPPABLE_FIELDS = [
'tracker' => 'label_tracker',
'status' => 'label_status',
...
]
</pre><br />and then inside the <code>auto_map_fields</code> method we use it.</p>
<p>2. We use Javascript where we have access to all the required information, we just need to iterate throw each select element. This solution can handle both cases.</p> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=962332020-02-13T01:35:24ZYuichi HARADA
<ul><li><strong>File</strong> <a href="/attachments/24816">22913_auto_mapping_v2.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/24816/22913_auto_mapping_v2.patch">22913_auto_mapping_v2.patch</a> added</li></ul><p>Marius BALTEANU wrote:</p>
<blockquote>
<p>My main concern with the proposed patch are the hardcoded fields from <code>mapping</code> method (<code>ImportsController</code>). Because of them, the "auto mapping" feature it won't be easily extended by plugins or other models (it is against the generalisation added in <a class="changeset" title="Generalize issues imports (#28234). Extend import controller to support arbitrary imports based ..." href="https://www.redmine.org/projects/redmine/repository/svn/revisions/18145">r18145</a>).</p>
<p>I'm attaching a patch with another solution, where each subclass of the import have a class constant (<code>AUTO_MAPPABLE_FIELDS</code> in my patch) or a method that returns an array with the fields that can be auto-mapped. In <code>ImportsController</code>, the method <code>auto_map_fields</code> uses this constant and try to auto map the fields based on some conditions. This auto mapping uses only the "internal" name of the fields, not labels or translations.</p>
<p>To support labels and translations as well, I see two options:<br />1. We modify the <code>AUTO_MAPPABLE_FIELDS</code> to return a hash with each internal field and his label. For example:<br />[...]<br />and then inside the <code>auto_map_fields</code> method we use it.</p>
<p>2. We use Javascript where we have access to all the required information, we just need to iterate throw each select element. This solution can handle both cases.</p>
</blockquote>
<p>Marius, thank you for pointing it out. It was very helpful because there were some specifications that I knew for the first time.<br />I fixed <a class="attachment" href="https://www.redmine.org/attachments/24362">22913_auto_mapping.patch</a> based on the provided sample code.</p> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=962812020-02-16T10:41:44ZMarius BĂLTEANU
<ul><li><strong>File</strong> <i>0002-Auto-select-fields-mapping-in-import-based-on-the-in.patch</i> added</li><li><strong>File</strong> <a href="/attachments/24832">0001-Use-user-as-internal-field-instead-of-user_id-becaus.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/24832/0001-Use-user-as-internal-field-instead-of-user_id-becaus.patch">0001-Use-user-as-internal-field-instead-of-user_id-becaus.patch</a> added</li></ul><p>Thanks Yuichi for updating the patch.</p>
<p>I tried it and I've found some issues:<br />- the auto mapping is case sensitive. For example: <code>tracker</code> is not mapped to <code>Tracker</code>.<br />- auto mapping by internal fields name (tracker, estimared_hours, cf_6, spent_on) is not working. I find it useful to allow this as well for advanced users.<br />- the auto mapping works only when <code>import.mapping</code> is empty. I don't think that this condition is necessary because you already check inside the <code>auto_map_fields</code> method if the mapping for that field exists or not.</p>
<p>I decided to rework a little bit your patch in order to fix those issues, other edge cases and to reuse the existing import time entries CSV file, please let me know what do you thing. The first patch changes the internal field name for <code>User</code> from <code>user_id</code> to <code>user</code> because the import accepts also logins.</p>
<p>Test results here: <a class="external" href="https://gitlab.com/redmine-org/redmine/pipelines/118278329">https://gitlab.com/redmine-org/redmine/pipelines/118278329</a></p> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=962932020-02-18T01:49:33ZYuichi HARADA
<ul></ul><p>Marius BALTEANU wrote:</p>
<blockquote>
<p>Thanks Yuichi for updating the patch.</p>
<p>I tried it and I've found some issues:<br />- the auto mapping is case sensitive. For example: <code>tracker</code> is not mapped to <code>Tracker</code>.<br />- auto mapping by internal fields name (tracker, estimared_hours, cf_6, spent_on) is not working. I find it useful to allow this as well for advanced users.<br />- the auto mapping works only when <code>import.mapping</code> is empty. I don't think that this condition is necessary because you already check inside the <code>auto_map_fields</code> method if the mapping for that field exists or not.</p>
<p>I decided to rework a little bit your patch in order to fix those issues, other edge cases and to reuse the existing import time entries CSV file, please let me know what do you thing. The first patch changes the internal field name for <code>User</code> from <code>user_id</code> to <code>user</code> because the import accepts also logins.</p>
<p>Test results here: <a class="external" href="https://gitlab.com/redmine-org/redmine/pipelines/118278329">https://gitlab.com/redmine-org/redmine/pipelines/118278329</a></p>
</blockquote>
<p>Thanks, Marius for improving the patches. I confirmed that it works properly.<br />In particular, it was very good that the <code>test/functional/imports_controller_test.rb</code> was written in detail and was easy to understand. However, I think it would be better if the following were modified.</p>
<pre><code class="diff syntaxhl"><span class="gh">diff --git a/app/controllers/imports_controller.rb b/app/controllers/imports_controller.rb
index 664d40019..96c493a31 100644
</span><span class="gd">--- a/app/controllers/imports_controller.rb
</span><span class="gi">+++ b/app/controllers/imports_controller.rb
</span><span class="p">@@ -186,7 +186,7 @@</span> class ImportsController < ApplicationController
next if mappings.include?(field_nm)
index = headers.index(field_nm) || headers.index(field.name.downcase)
if index
<span class="gd">- mappings[field_nm] = headers.index(field_nm) || headers.index(field.name.downcase)
</span><span class="gi">+ mappings[field_nm] = index
</span> end
end
mappings
</code></pre> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=963132020-02-18T17:02:12ZMarius BĂLTEANU
<ul><li><strong>File</strong> <a href="/attachments/24851">0002-Auto-select-fields-mapping-in-import-based-on-the-in.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/24851/0002-Auto-select-fields-mapping-in-import-based-on-the-in.patch">0002-Auto-select-fields-mapping-in-import-based-on-the-in.patch</a> added</li></ul><p>Yuichi HARADA wrote:</p>
<blockquote>
<p>Marius BALTEANU wrote:<br />[...]</p>
<p>Thanks, Marius for improving the patches. I confirmed that it works properly.</p>
</blockquote>
<p>You're welcome.</p>
<blockquote>
<p>In particular, it was very good that the <code>test/functional/imports_controller_test.rb</code> was written in detail and was easy to understand. However, I think it would be better if the following were modified.</p>
</blockquote>
<p>Agree, I've updated the second patch. Thanks for pointing this out.</p> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=963142020-02-18T17:02:19ZMarius BĂLTEANU
<ul><li><strong>File</strong> deleted (<del><i>0002-Auto-select-fields-mapping-in-import-based-on-the-in.patch</i></del>)</li></ul> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=963202020-02-19T05:10:39ZGo MAEDA
<ul></ul><p>The patch series works fine. Thank you.<br />I will commit them in a few days.</p> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=963232020-02-19T05:18:31ZMarius BĂLTEANU
<ul><li><strong>Assignee</strong> set to <i>Go MAEDA</i></li></ul> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=963472020-02-19T14:54:42ZGo MAEDA
<ul><li><strong>Subject</strong> changed from <i>Auto-select fields mapping in Importing</i> to <i>Auto-select fields mapping in Importing a CSV file</i></li><li><strong>Status</strong> changed from <i>New</i> to <i>Closed</i></li></ul><p>Committed the patches.</p>
<p>Thank you all for working on this nice improvement.</p> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=963502020-02-19T23:27:46ZGo MAEDA
<ul><li><strong>Subject</strong> changed from <i>Auto-select fields mapping in Importing a CSV file</i> to <i>Auto-select fields mapping in Importing</i></li></ul> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=998552020-11-27T05:58:30ZGo MAEDA
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-5 priority-4 priority-default closed" href="/issues/34326">Defect #34326</a>: CSV import raises an exception if CSV header has empty columns</i> added</li></ul> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=1015762021-03-28T00:10:00ZGo MAEDA
<ul><li><strong>Tracker</strong> changed from <i>Patch</i> to <i>Feature</i></li><li><strong>Resolution</strong> set to <i>Fixed</i></li></ul> Redmine - Feature #22913: Auto-select fields mapping in Importinghttps://www.redmine.org/issues/22913?journal_id=1021422021-04-20T19:08:18ZMarius BĂLTEANU
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-5 priority-4 priority-default closed" href="/issues/35131">Defect #35131</a>: Issue import - allow auto mapping for Unique ID and relation type fields</i> added</li></ul>