topic Migration of 2000 Ms-SQL-Server Tables to Cassandra in Talend Studio

Migration of 2000 Ms-SQL-Server Tables to Cassandra

TomG1 — Wed, 17 Feb 2016 13:04:33 GMT

Hi,
We have a requirement to migrate around 2000 tables from SQL server to Cassandra(Big Data) Database.
The jobs are simple in which SQL server table will be mapped to a Cassandra table.
There is no complex logic. Its just copying data from one table(in SQL SERVER) to another table (in Cassandra)
SQL_SERVER_TABLE_1------------>Cassandra_Table_1 and so on..
So for 2000 tables in SQL Server we have to create 2000 jobs to copy data into Cassandra.
can you please suggest some automation so that we can save time of creating 2000 jobs manually?
P.S: Note that in Cassandra a table is called a 'column family'. For the sake of clarity I have mentioned it as table.
Thanks
Tom

Re: Migration of 2000 Ms-SQL-Server Tables to Cassandra

Anonymous — Thu, 18 Feb 2016 08:16:17 GMT

Hi,
Are your 2000 tables same schema structure?
So far, tCassandraOutput don't support for "Dynamic Schema" feature.
Best regards
Sabrina

Re: Migration of 2000 Ms-SQL-Server Tables to Cassandra

Anonymous — Tue, 23 Feb 2016 11:06:15 GMT

Hi xdshi,
We have a query regarding the above scenario which was raised by Tom.
Lets suppose we have Source with Schema1 and Target with Schema2. And target table structures are same as Source table structures.
But each tables are different from each other structurally. Our requirement is to load all the 2000 target tables with single or less number of jobs.
Please advise this can be done or not.
Thanks

Re: Migration of 2000 Ms-SQL-Server Tables to Cassandra

Anonymous — Wed, 24 Feb 2016 08:20:52 GMT

Hi,
So far, talend big data components don't support for dynamic schema.
Best regards
Sabrina