0 Replies Latest reply: May 4, 2017 2:25 PM by Vladimir Komarov RSS

    Qlik CSV load fails to detect duplicate fields' names

    Vladimir Komarov

      Hello all,

       

      Here is the scenario:

       

      I do have a CSV file that has several identical fields' names:

      Screen Shot 2017-05-04 at 12.26.28 PM.png

      Loading this file in the original CSV format creates a problem since QS does not detect the duplicate fields' names until the actual script's loading.

      The script it generated as:

      Screen Shot 2017-05-04 at 12.48.59 PM.png

      Which requires fields renaming before the load:

      Screen Shot 2017-05-04 at 12.55.11 PM.png

       

      The problem is that Qlik loads the data for all these AAAAA fields from the last (??!!) field listed in the table:

      Here is the result:

      Screen Shot 2017-05-04 at 12.53.44 PM.png

       

       

      Using "Data Manager" and "Add Data" mode produces even more radical solution. Qlik just ignores duplicate fields:

      Screen Shot 2017-05-04 at 1.11.31 PM.png

       

       

       

      The same file converted to XLS format works fine. Qlik recognizes duplicates and generates the script as:

      Screen Shot 2017-05-04 at 1.03.29 PM.png

      Which produces the proper result:

      Screen Shot 2017-05-04 at 1.05.45 PM.png

      Any reason Qlik has not applied the same logic to CSV load?

       

       

      I know that I can ignore file's Header and load data as @1, @2, ... @n fields... Which requires a lot of fields' renaming (I am loading US Census data with 500+ columns) and is not very stable because of the possible data structure changes....

       

      I also know that this is a legacy issue and was discussed here few times (Same Field Names) and (Re: Reload Excel file with duplicate field name). But I am just wondering if any other solution would be suggested....

       

      Appreciate a feedback and suggestions...

       

      Regards,

      Vlad