4 Replies Latest reply: Jun 21, 2017 7:15 PM by Saima SM RSS

    Data duplication problem

    Saima SM

      Hi All,

       

      I have 9 tables loaded from SQL database. They are all master detailed highly normalized tables. They are dimensions with no measures. I need them to use them altogether as one model. When I load them in qlik sense they sows me right count in each table in data model viewers. But due to they share IDs as foreign keys whenever I do count of any ID from one table it give me high numbers. Is there any solution having these interlinked tables but without getting duplicated ID counts. Thanks All

        • Re: Data duplication problem
          Mary Jo Taft

          Saima,

           

          Can you put more information in your post about the tables?  Do the tables contain fields that have the same name (are you getting synthetic keys in your Qlik data model)?  It is hard to help with so little information provided.

            • Re: Data duplication problem
              Saima SM

              Hi, I do not have syn Key but my tables came form SQL database and highly normalized. when 2 tables are connected with primary /foreign key there comes duplication. but I need them to use together to get cross table information and having no measure to create star schema. thanks

            • Re: Data duplication problem
              Justin Dallas

              Difficult problem to diagnose without an example, but I have my own example.  I was working with a Great Plains/MS Dynamics database, specifically the General Ledge stuff.  Because all the Account History, Account Summary, Account Transaction tables all had VERY similar fields (Acct #1,Acct #2,Acct #3,Acct #4 - GL) there were synthetic keys everywhere.


              My solution was to mark the rows and then Concatenate them all into one fat Fact Table. i.e

              
              
              Concatenate(AccountsNStuff)
              
              Load * ,
                  1 as IsAccountSummary
              ;
              LOAD * Resident GLsAndStuff
              ;
              
              


              • Re: Data duplication problem
                David Forest

                Agree with previous responses, it may be as simple as adding DISTINCT to the Count.

                Understanding and resolving synthetic keys is important. Qlik performs best when the data model is set up as a star schema.