Pivotal Cloud Foundry (PCF) binding a User Defined service for Oracle 11g r2

In this example below I show how you would create a user defined service for Oracle 11g R2 which is running outside of the PCF instance and how your applications would bind to that service. The example below is from the spring-music sample application which is a spring based web application.

https://github.com/cloudfoundry-samples/spring-music

1. First lets create our user defined service for Oracle , here we just use a simple URI which the Spring Cloud library knows how to use.

c:\pivotal\spring-music-oracle>cf create-service user-provided
Name?> oracle11gr2

What credential parameters should applications use to connect to this service instance?
(e.g. hostname, port, password)> uri

uri> oracle://scott:tiger@192.168.110.209:1521/XE

Creating service oracle11gr2... OK

c:\pivotal\spring-music-oracle>


The spring cloud project is here.

https://github.com/spring-projects/spring-cloud

2. Now with a previously created WAR file lets push the spring music app to PCF and bind to our oracle service as shown below.

c:\pivotal\spring-music-oracle>cf push --path=spring-music-oracle.war
Name> spring-music-oracle

Instances> 1

1: 128M
2: 256M
3: 512M
4: 1G
Memory Limit> 3

Creating spring-music-oracle... OK

1: spring-music-oracle
2: none
Subdomain> spring-music-oracle

1: system.com
2: none
Domain> system.com

Binding spring-music-oracle.system.com to spring-music-oracle... OK

Create services for application?> n

Bind other services to application?> y

1: mysql-dev
2: rmq
3: user-provided-705bf
4: oracle11gr2
Which service?> 4

Binding oracle11gr2 to spring-music-oracle... OK
Bind another service?> n

Save configuration?> n

Uploading spring-music-oracle... OK
Preparing to start spring-music-oracle... OK
-----> Downloaded app package (22M)
-----> Downloading OpenJDK 1.7.0_51 from http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.7.0_
51.tar.gz (1m 52s)
       Expanding JRE to .java (2.4s)
-----> Downloading Spring Auto-reconfiguration 0.8.4 from http://download.pivotal.io.s3.amazonaws.com/auto-reconfigurati
on/auto-reconfiguration-0.8.4.jar (4.3s)
       Modifying /WEB-INF/web.xml for Auto Reconfiguration
-----> Downloading Tomcat 7.0.50 from http://download.pivotal.io.s3.amazonaws.com/tomcat/tomcat-7.0.50.tar.gz (27.9s)
       Expanding Tomcat to .tomcat (0.3s)
-----> Downloading Buildpack Tomcat Support 1.1.1 from http://download.pivotal.io.s3.amazonaws.com/tomcat-buildpack-supp
ort/tomcat-buildpack-support-1.1.1.jar (0.2s)
-----> Uploading droplet (63M)
Checking status of app 'spring-music-oracle'...
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  1 of 1 instances running (1 running)
Push successful! App 'spring-music-oracle' available at spring-music-oracle.system.com

c:\pivotal\spring-music-oracle>


3. Here are some screenshots showing it deployed , invoking the application, and data within the Oracle 11g r2 schema.

The code to make this happen for Oracle is as follows, we do this to make use of the spring cloud code to bind to the user defined service "oracle11gr2" we created above. The spring cloud Library enables cloud applications to connect to services. The classes below show how we enable our oracle service for the spring music application.

OracleServiceInfo.java

  
package org.cloudfoundry.samples.music.cloud;

import org.springframework.cloud.service.ServiceInfo.ServiceLabel;
import org.springframework.cloud.service.common.RelationalServiceInfo;

@ServiceLabel("oracle")
public class OracleServiceInfo extends RelationalServiceInfo {

    public OracleServiceInfo(String id, String url) 
    {       
     super(id, url, "oracle");
    }

 @Override
 public String getJdbcUrl() 
 {
  return String.format("jdbc:%s:thin:%s/%s@%s:%d/%s",
                              "oracle",
                              super.getUserName(), super.getPassword(),
                              super.getHost(), super.getPort(), super.getPath());
 }
      
} 

OracleDataSourceCreator.java

  
package org.cloudfoundry.samples.music.cloud;

import org.cloudfoundry.samples.music.cloud.OracleServiceInfo;
import org.springframework.cloud.service.relational.DataSourceCreator;

public class OracleDataSourceCreator extends DataSourceCreator<OracleServiceInfo> {
    private static final String[] DRIVERS = new String[]{"oracle.jdbc.OracleDriver"};

    private static final String VALIDATION_QUERY = "SELECT 'Y' from dual";

    public OracleDataSourceCreator() {
     super("spring-cloud.oracle.driver", DRIVERS, VALIDATION_QUERY);
    }
} 

UserProvidedOracleServiceInfoCreator.java

  
package org.cloudfoundry.samples.music.cloud;

import org.springframework.cloud.cloudfoundry.CloudFoundryServiceInfoCreator;

import java.util.Map;

public class UserProvidedOracleServiceInfoCreator extends CloudFoundryServiceInfoCreator<OracleServiceInfo> {
    public UserProvidedOracleServiceInfoCreator() {
        super("user-provided");
    }

    @Override
    @SuppressWarnings("unchecked")
    public boolean accept(Map<String, Object> serviceData) {
        String label = (String) serviceData.get("label");

        if (label.equals(getTag())) {
            Map<String, Object> credentials = (Map<String, Object>) serviceData.get("credentials");
            String uri = (String) credentials.get("uri");

            if (uri != null && uri.startsWith("oracle:")) {
                return true;
            }
        }

        return false;
    }

    @Override
    @SuppressWarnings("unchecked")
    public OracleServiceInfo createServiceInfo(Map<String, Object> serviceData) {
        String id = (String) serviceData.get("name");
        Map<String, Object> credentials = (Map<String, Object>) serviceData.get("credentials");
        String uri = credentials.get("uri").toString();
        return new OracleServiceInfo(id, uri);
    }
}  

OracleRepositoryConfig.java

  
package org.cloudfoundry.samples.music.config.data;

import org.hibernate.dialect.MySQL5Dialect;
import org.hibernate.dialect.Oracle10gDialect;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;

@Configuration
@Profile("oracle")
@EnableJpaRepositories("org.cloudfoundry.samples.music.repositories.jpa")
public class OracleRepositoryConfig extends AbstractJpaRepositoryConfig {

    protected String getHibernateDialect() {
        return Oracle10gDialect.class.getName();
    }

}

RelationalCloudDataSourceConfig.java

  
package org.cloudfoundry.samples.music.config.data;

import org.springframework.cloud.config.java.AbstractCloudConfig;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;

import javax.sql.DataSource;

@Configuration
@Profile({"mysql-cloud", "postgres-cloud", "oracle-cloud"})
public class RelationalCloudDataSourceConfig extends AbstractCloudConfig {

    @Bean
    public DataSource dataSource() {
        return connectionFactory().dataSource();
    }

} 

src/main/resources/META-INF/servicesorg.springframework.cloud.cloudfoundry.CloudFoundryServiceInfoCreator

org.cloudfoundry.samples.music.cloud.UserProvidedMysqlServiceInfoCreator
org.cloudfoundry.samples.music.cloud.UserProvidedOracleServiceInfoCreator

src/main/resources/META-INF/services/org.springframework.cloud.service.ServiceConnectorCreator

org.springframework.cloud.service.relational.MysqlDataSourceCreator
org.cloudfoundry.samples.music.cloud.OracleDataSourceCreator
org.springframework.cloud.service.relational.PostgresqlDataSourceCreator
org.springframework.cloud.service.keyval.RedisConnectionFactoryCreator
org.springframework.cloud.service.document.MongoDbFactoryCreator
org.springframework.cloud.service.messaging.RabbitConnectionFactoryCreator
org.springframework.cloud.service.smtp.MailSenderCreator

Finally you need to ensure that ojdbc6.jar is shipped with the WAR file within "WEB-INF/lib" which is the Oracle JDBC driver to enable the application to connect to oracle.

More Information on Pivotal CF

http://www.gopivotal.com/products/cloud-foundry

Spring MVC with Thymeleaf HTML template pages

Thymeleaf offers a set of Spring integrations that allow you to use it as a full-featured substitute for JSP in Spring MVC applications.

So here is a simple example just so we can compare how things differ in HTML versus JSP, does remind me of JSF / Viewlets to be honest.

1. Let start with what our simple project looks like in IntelliJ

2. The web.xml is defined as follows

  
<web-app version="2.4"
 xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee 
 http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">

 <display-name>Spring MVC Application</display-name>

    <servlet>
  <servlet-name>mvc-dispatcher</servlet-name>
  <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
 </servlet>

    <servlet-mapping>
        <servlet-name>mvc-dispatcher</servlet-name>
        <url-pattern>/</url-pattern>
    </servlet-mapping>

</web-app> 

3. The spring context XML is defined as follows

WEB-INF/mvc-dispatcher-servlet.xml

  
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">

    <context:component-scan base-package="pivotal.au.pas.springapp.mvc"/>

    <bean id="templateResolver"
          class="org.thymeleaf.templateresolver.ServletContextTemplateResolver">
        <property name="prefix" value="/WEB-INF/resources/templates/" />
        <property name="suffix" value=".html" />
        <property name="templateMode" value="HTML5" />
    </bean>

    <bean id="templateEngine"
          class="org.thymeleaf.spring3.SpringTemplateEngine">
        <property name="templateResolver" ref="templateResolver" />
    </bean>

    <bean class="org.thymeleaf.spring3.view.ThymeleafViewResolver">
        <property name="templateEngine" ref="templateEngine" />
    </bean>

</beans>  

4. Now the PeopleController.java class looks just like it would when rendering a JSP page view and so it should all we have done here is change what the view page will be which in this case is HTML itself. I have created some mock up data , rather then connecting to a RDBMS here.

  
package pivotal.au.pas.springapp.mvc;

import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.RequestMapping;
import pivotal.au.pas.springapp.beans.Person;

import java.util.ArrayList;
import java.util.List;

@Controller
public class PeopleController
{
    private static List<Person> people = null;

    static
    {
        people = new ArrayList<Person>();
        people.add(new Person(1, "pas"));
        people.add(new Person(2, "lucia"));
        people.add(new Person(3, "lucas"));
        people.add(new Person(4, "siena"));
    }

    @RequestMapping("/showallpeople")
    public String showallpeople(Model model)
    {
        model.addAttribute("people", people);
        return "people";
    }
}    

5. The view page is just a HTML page as follows

WEB-INF/resources/templates/people.html

  
<!DOCTYPE html>
<html xmlns:th="http://www.thymeleaf.org">
<head>
    <title>Spring MVC with Thymeleaf - All people</title>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body>
<h2> Spring MVC with Thymeleaf - All people </h2>

<table border="1">
    <thead>
    <tr>
        <th>Id#</th>
        <th>Name</th>
    </tr>
    </thead>
    <tbody>
    <tr th:each="var : ${people}">
        <td th:text="${var.id}"></td>
        <td th:text="${var.name}"></td>
    </tr>
    </tbody>
</table>

<br />
<hr />
<br />
<address>
    <a href="mailto:papicella@gopivotal.com">Pas Apicella</a>
</address>
</body>
</html>

6. So here is a simple JSP page as the entry page, could of been a HTML page.

index.jsp

  
<%@ page contentType="text/html;charset=UTF-8" language="java" %>
<html>
<head>
    <title>Spring MVC with Thymeleaf</title>
</head>
<body>
<h2>Spring MVC with Thymeleaf</h2>
<ul>
    <li><a href="/greeting">Greeting</a></li>
    <li><a href="/showallpeople">Show All People</a></li>
</ul>
<hr />
<br />
<address>
    <a href="mailto:papicella@gopivotal.com">Pas Apicella</a>
</address>
</body>
</html>   

7. Run index.jsp

8. Finally when run and I click on "Show All People" link we get display as follows

More Information

http://spring.io/guides/gs/serving-web-content/

http://www.thymeleaf.org/index.html

SpringXD : Pre-Packaged Batch Jobs Import CSV Files to GemFireXD

Spring XD comes with several batch import and export modules. You can run them out of the box and show using the "Import CSV Files to JDBC (filejdbc)" as shown below.

This example show's inserting data into GemFireXD, you can use any RDBMS which supports a JDBC driver.

Note: XD_BASE = /Users/papicella/vmware/software/spring/spring-xd-1.0.0.M4

1. Configure connection to RDBMS in "$XD_BASE/xd/config/batch-jdbc-import.properties".

# Setting for the JDBC batch import job module
driverClass=com.vmware.sqlfire.jdbc.ClientDriver
url=jdbc:sqlfire://10.32.240.113:1527
username=APP
password=APP

2. Add jdbc jar file to $XD_BASE/xd/lib directory, in this case GemFireXD which uses "sqlfireclient.jar"

3. Start SpringXD single node using "./$XD_BASE/xd/bin/xd-singlenode"

4. Create a file called people.csv with contents as follows

[Fri Jan 10 14:35:00 papicella@:~/vmware/software/spring/spring-xd-1.0.0.M4/files ] $ cat people.csv
1,pas
2,lucia
3,lucas
4,siena

5. Log into SpringXD shell as shown below using "$XD_BASE/shell/bin/xd-shell"

6. Create table in GemFireXD as shown below.

  
sqlf> create table people (id int, name varchar(20));
0 rows inserted/updated/deleted 

7. Create a JOB as shown below

xd:>job create myjob --definition "filejdbc --resources=file:/Users/papicella/vmware/software/spring/spring-xd-1.0.0.M4/files/*.csv --names=id,name --tableName=people"

Successfully created and deployed job 'myjob'

8. Start the JOB

xd:>job launch myjob
Successfully launched the job 'myjob'

9. Verify that the CSV data has been inserted into the table PEOPLE in GemFireXD

  
sqlf> select * from people;
ID         |NAME                
--------------------------------
2          |lucia               
1          |pas                 
4          |siena               
3          |lucas               

4 rows selected 

More Information

For more information on SpringXD or GemFireXD see the links below.

http://projects.spring.io/spring-xd/ - SpringXD

http://gopivotal.com/products/pivotal-hd - GemFireXD Beta

SpringXD FILE/TAIL input ingestion / JDBC Sink output to GemFireXD

The following examples shows what we can do out of the box with SpringXD TAIL input source and JDBC sink as the output source. This can all be done without writing and code.

Assumes you have SpringXD installed and GemFireXD installed. See the getting stared guide which explains how to setup SpringXD. As for a database output for JDBC you can use any RDBMS here so you could use MYSQL , ORACLE etc if you don't have GemFireXD installed.

Note: XD_BASE = /Users/papicella/vmware/software/spring/spring-xd-1.0.0.M4

1. Configure connection to RDBMS in "$XD_BASE/xd/config/jdbc.properties".

driverClass=com.vmware.sqlfire.jdbc.ClientDriver
url=jdbc:sqlfire://10.32.240.113:1527
username=APP
password=APP

2. Add jdbc jar file to $XD_BASE/xd/lib directory, in this case GemFireXD which uses "sqlfireclient.jar"

3. Start SpringXD single node using "./$XD_BASE/xd/bin/xd-singlenode"

./$XD_BASE/xd/bin/xd-singlenode
.....

11:29:00,417  INFO main container.XDContainer:175 - started container: 0
11:29:00,424  INFO main launcher.LocalContainerLauncher:95 - 
 _____                           __   _______
/  ___|          (-)             \ \ / /  _  \
\ `--. _ __  _ __ _ _ __   __ _   \ V /| | | |
 `--. \ '_ \| '__| | '_ \ / _` |  / ^ \| | | |
/\__/ / |_) | |  | | | | | (_| | / / \ \ |/ /
\____/| .__/|_|  |_|_| |_|\__, | \/   \/___/
      | |                  __/ |
      |_|                 |___/
1.0.0.M4                         eXtreme Data

Using local mode JMX is disabled for XD components
XD Configuration:
XD_HOME=/Users/papicella/vmware/software/spring/spring-xd-1.0.0.M4/xd
XD_TRANSPORT=local
XD_STORE=memory
XD_ANALYTICS=memory
XD_HADOOP_DISTRO=hadoop12

started container : 85631@localhost
Documentation: https://github.com/SpringSource/spring-xd/wiki

4. Create a empty file that the stream will read from using TAIL as the input

> touch filetest

eg:

[Fri Jan 10 11:29:06 papicella@:/tmp/xd/input ] $ ls -la /tmp/xd/input/filetest 
-rw-r--r--  1 papicella  wheel  0 10 Jan 11:29 /tmp/xd/input/filetest

5. Log into SpringXD shell as shown below using "$XD_BASE/shell/bin/xd-shell"

[Fri Jan 10 10:46:39 papicella@:~/vmware/software/spring/spring-xd-1.0.0.M4/shell/bin ] $ ./xd-shell

 _____                           __   _______

/  ___|          (-)             \ \ / /  _  \

\ `--. _ __  _ __ _ _ __   __ _   \ V /| | | |

 `--. \ '_ \| '__| | '_ \ / _` |  / ^ \| | | |

/\__/ / |_) | |  | | | | | (_| | / / \ \ |/ /

\____/| .__/|_|  |_|_| |_|\__, | \/   \/___/

      | |                  __/ |

      |_|                 |___/

eXtreme Data

1.0.0.M4 | Admin Server Target: http://localhost:9393

Welcome to the Spring XD shell. For assistance hit TAB or type "help".

6. Create a STREAM as shown below.

xd:> stream create --name filetest --definition "tail --name=/tmp/xd/input/filetest |  jdbc --columns=id,name"

Created new stream 'filetest'

7. Create table in database as follows.

  
sqlf> create table filetest (id int, name varchar(20));

8. Create file as follows called "testfile" we will copy this to stream file soon.

[Fri Jan 10 11:38:44 papicella@:~/vmware/software/spring/spring-xd-1.0.0.M4 ] $ cat filetest

{"id":"1","name":"pas"}

{"id":"2","name":"lucia"}

{"id":"3","name":"lucas"}

{"id":"4","name":"siena"}

7. Copy file "filetest" above to /tmp/xd/input/filetest

9. Check base table in RDBMS for data inserts.

  
sqlf> select * from filetest;
ID         |NAME                
--------------------------------
4          |siena               
1          |pas                 
3          |lucas               
2          |lucia               

4 rows selected

More Information

For more information on SpringXD or GemFireXD see the links below.

http://projects.spring.io/spring-xd/ - SpringXD

http://gopivotal.com/products/pivotal-hd - GemFireXD Beta