SpringXD : Pre-Packaged Batch Jobs Import CSV Files to GemFireXD

Spring XD comes with several batch import and export modules. You can run them out of the box and show using the "Import CSV Files to JDBC (filejdbc)" as shown below.

This example show's inserting data into GemFireXD, you can use any RDBMS which supports a JDBC driver.

Note: XD_BASE = /Users/papicella/vmware/software/spring/spring-xd-1.0.0.M4

1. Configure connection to RDBMS in "$XD_BASE/xd/config/batch-jdbc-import.properties".

# Setting for the JDBC batch import job module
driverClass=com.vmware.sqlfire.jdbc.ClientDriver
url=jdbc:sqlfire://10.32.240.113:1527
username=APP
password=APP

2. Add jdbc jar file to $XD_BASE/xd/lib directory, in this case GemFireXD which uses "sqlfireclient.jar"

3. Start SpringXD single node using "./$XD_BASE/xd/bin/xd-singlenode"

4. Create a file called people.csv with contents as follows

[Fri Jan 10 14:35:00 papicella@:~/vmware/software/spring/spring-xd-1.0.0.M4/files ] $ cat people.csv
1,pas
2,lucia
3,lucas
4,siena

5. Log into SpringXD shell as shown below using "$XD_BASE/shell/bin/xd-shell"

6. Create table in GemFireXD as shown below.

  
sqlf> create table people (id int, name varchar(20));
0 rows inserted/updated/deleted 

7. Create a JOB as shown below

xd:>job create myjob --definition "filejdbc --resources=file:/Users/papicella/vmware/software/spring/spring-xd-1.0.0.M4/files/*.csv --names=id,name --tableName=people"

Successfully created and deployed job 'myjob'

8. Start the JOB

xd:>job launch myjob
Successfully launched the job 'myjob'

9. Verify that the CSV data has been inserted into the table PEOPLE in GemFireXD

  
sqlf> select * from people;
ID         |NAME                
--------------------------------
2          |lucia               
1          |pas                 
4          |siena               
3          |lucas               

4 rows selected 

More Information

For more information on SpringXD or GemFireXD see the links below.

http://projects.spring.io/spring-xd/ - SpringXD

http://gopivotal.com/products/pivotal-hd - GemFireXD Beta