After installing PCF 2.0 here is how you can verify your installation using the new bosh2 CLI. In this example I use "bosh2" BUT with PCF 2.0 you can actually use "bosh". bosh2 v2 existed for a while in PCF 1.12 and some previous versions while we left bosh v1
1. SSH into your ops manager VM as shown below, in this example we using GCP
https://docs.pivotal.io/pivotalcf/2-0/customizing/trouble-advanced.html#ssh
2. Create an alias for your ENV as shown below
Note: You will need the bosh director IP address which you can obtain using
https://docs.pivotal.io/pivotalcf/2-0/customizing/trouble-advanced.html#gather
ubuntu@opsman-pcf:~$ bosh2 alias-env gcp -e y.y.y.y --ca-cert /var/tempest/workspaces/default/root_ca_certificate
Using environment 'y.y.y.y' as anonymous user
Name p-bosh
UUID 3c886290-144f-4ec7-86dd-b7586b98dc3b
Version 264.4.0 (00000000)
CPI google_cpi
Features compiled_package_cache: disabled
config_server: enabled
dns: disabled
snapshots: disabled
User (not logged in)
Succeeded
3. Log in to the BOSH Director with UAA
Note: You will need the username / password for the bosh director which you can obtain as follows
https://docs.pivotal.io/pivotalcf/2-0/customizing/trouble-advanced.html#gather
ubuntu@opsman-pcf:~$ bosh2 -e gcp log-in
Email (): director
Password ():
Successfully authenticated with UAA
Succeeded
4. View all the VM's managed by BOSH as follows
ubuntu@opsman-pcf:~/scripts$ bosh2 -e gcp vms --column=Instance --column="Process State" --column=AZ --column="VM Type"
Using environment 'y.y.y.y' as user 'director' (bosh.*.read, openid, bosh.*.admin, bosh.read, bosh.admin)
Task 65. Done
Deployment 'cf-adee3657c74c7b9a8e35'
Instance Process State AZ VM Type
backup-prepare/996340c7-4114-472e-b660-a5e353493fa4 running australia-southeast1-a micro
blobstore/cdd6fc8d-25c9-4cfb-9908-89eb0164fb80 running australia-southeast1-a medium.mem
compute/2dfcc046-c16a-4a36-9170-ef70d1881818 running australia-southeast1-a xlarge.disk
control/2f3d0bc6-9a2d-4c08-9ccc-a88bad6382a3 running australia-southeast1-a xlarge
database/da60f0e7-b8e3-4f8d-945d-306b267ac161 running australia-southeast1-a large.disk
mysql_monitor/a88331c4-1659-4fe4-b8e9-89ce4bf092fd running australia-southeast1-a micro
router/276e308e-a476-4c8d-9555-21623dada492 running australia-southeast1-a micro
7 vms
Succeeded
** Few other examples **
- View all the deployments, in this example we just have the PAS small footprint tile installed so it only exists and no other bosh managed tiles xist
ubuntu@opsman-pcf:~/scripts$ bosh2 -e gcp deployments --column=name
Using environment 'y.y.y.y' as user 'director' (bosh.*.read, openid, bosh.*.admin, bosh.read, bosh.admin)
Name
cf-adee3657c74c7b9a8e35
1 deployments
Succeeded
- Run cloud check to check for issues
ubuntu@opsman-pcf:~/scripts$ bosh2 -e gcp -d cf-adee3657c74c7b9a8e35 cloud-check
Using environment 'y.y.y.y' as user 'director' (bosh.*.read, openid, bosh.*.admin, bosh.read, bosh.admin)
Using deployment 'cf-adee3657c74c7b9a8e35'
Task 66
Task 66 | 04:20:52 | Scanning 7 VMs: Checking VM states (00:00:06)
Task 66 | 04:20:58 | Scanning 7 VMs: 7 OK, 0 unresponsive, 0 missing, 0 unbound (00:00:00)
Task 66 | 04:20:58 | Scanning 3 persistent disks: Looking for inactive disks (00:00:01)
Task 66 | 04:20:59 | Scanning 3 persistent disks: 3 OK, 0 missing, 0 inactive, 0 mount-info mismatch (00:00:00)
Task 66 Started Fri Dec 29 04:20:52 UTC 2017
Task 66 Finished Fri Dec 29 04:20:59 UTC 2017
Task 66 Duration 00:00:07
Task 66 done
# Type Description
0 problems
Succeeded
More Information
https://docs.pivotal.io/pivotalcf/2-0/customizing/trouble-advanced.html
Search This Blog
Friday, 29 December 2017
Tuesday, 19 December 2017
Terminating a specific application instance using it's index number in Pivotal Cloud Foundry
I was recently asked how to terminate a specific application instance rather then terminate all instances using "cf delete".
We can easily using the CF REST API or even easier the CF CLI "cf curl" command which makes it straight forward to make REST based calls into cloud foundry as shown below.
CF REST API Docs
https://apidocs.cloudfoundry.org/280/
Below assumes you already logged into PCF using the CF CLI
1. First find an application that has multiple instances
pasapicella@pas-macbook:~$ cf app pas-cf-manifest
Showing health and status for app pas-cf-manifest in org apples-pivotal-org / space development as papicella@pivotal.io...
name: pas-cf-manifest
requested state: started
instances: 2/2
usage: 756M x 2 instances
routes: pas-cf-manifest.cfapps.io
last uploaded: Sun 19 Nov 21:26:26 AEDT 2017
stack: cflinuxfs2
buildpack: client-certificate-mapper=1.2.0_RELEASE container-security-provider=1.8.0_RELEASE java-buildpack=v4.5-offline-https://github.com/cloudfoundry/java-buildpack.git#ffeefb9 java-main
java-opts jvmkill-agent=1.10.0_RELEASE open-jdk-like-jre=1.8.0_1...
state since cpu memory disk details
#0 running 2017-12-16T00:11:27Z 0.0% 241.5M of 756M 139.9M of 1G
#1 running 2017-12-17T10:39:09Z 0.3% 221.3M of 756M 139.9M of 1G
2. Use a "cf curl" curl which uses the application GUID to determine which application to check all application instances and their current state
pasapicella@pas-macbook:~$ cf curl /v2/apps/`cf app pas-cf-manifest --guid`/instances
{
"0": {
"state": "RUNNING",
"uptime": 293653,
"since": 1513383087
},
"1": {
"state": "RUNNING",
"uptime": 169591,
"since": 1513507149
}
}
3. Now let's delete instance with index "1". Don't forget that PCF will determine the current desired state of the application is not the current state and will re-start the application instance very quickly
pasapicella@pas-macbook:~$ cf curl /v2/apps/`cf app pas-cf-manifest --guid`/instances/1 -X DELETE
We can easily using the CF REST API or even easier the CF CLI "cf curl" command which makes it straight forward to make REST based calls into cloud foundry as shown below.
CF REST API Docs
https://apidocs.cloudfoundry.org/280/
Below assumes you already logged into PCF using the CF CLI
1. First find an application that has multiple instances
pasapicella@pas-macbook:~$ cf app pas-cf-manifest
Showing health and status for app pas-cf-manifest in org apples-pivotal-org / space development as papicella@pivotal.io...
name: pas-cf-manifest
requested state: started
instances: 2/2
usage: 756M x 2 instances
routes: pas-cf-manifest.cfapps.io
last uploaded: Sun 19 Nov 21:26:26 AEDT 2017
stack: cflinuxfs2
buildpack: client-certificate-mapper=1.2.0_RELEASE container-security-provider=1.8.0_RELEASE java-buildpack=v4.5-offline-https://github.com/cloudfoundry/java-buildpack.git#ffeefb9 java-main
java-opts jvmkill-agent=1.10.0_RELEASE open-jdk-like-jre=1.8.0_1...
state since cpu memory disk details
#0 running 2017-12-16T00:11:27Z 0.0% 241.5M of 756M 139.9M of 1G
#1 running 2017-12-17T10:39:09Z 0.3% 221.3M of 756M 139.9M of 1G
2. Use a "cf curl" curl which uses the application GUID to determine which application to check all application instances and their current state
pasapicella@pas-macbook:~$ cf curl /v2/apps/`cf app pas-cf-manifest --guid`/instances
{
"0": {
"state": "RUNNING",
"uptime": 293653,
"since": 1513383087
},
"1": {
"state": "RUNNING",
"uptime": 169591,
"since": 1513507149
}
}
3. Now let's delete instance with index "1". Don't forget that PCF will determine the current desired state of the application is not the current state and will re-start the application instance very quickly
pasapicella@pas-macbook:~$ cf curl /v2/apps/`cf app pas-cf-manifest --guid`/instances/1 -X DELETE
Note: You won't get any output BUT you can verify it has done what you asked for by running the command at step #2 again
pasapicella@pas-macbook:~$ cf curl /v2/apps/`cf app pas-cf-manifest --guid`/instances
{
"0": {
"state": "RUNNING",
"uptime": 293852,
"since": 1513383087
},
"1": {
"state": "DOWN",
"uptime": 0
}
}
If you run it again say 30 seconds later you should see your application instance re-started as shown below
pasapicella@pas-macbook:~$ cf curl /v2/apps/`cf app pas-cf-manifest --guid`/instances
{
"0": {
"state": "RUNNING",
"uptime": 293870,
"since": 1513383087
},
"1": {
"state": "STARTING",
"uptime": 11,
"since": 1513676947
}
}
pasapicella@pas-macbook:~$ cf curl /v2/apps/`cf app pas-cf-manifest --guid`/instances
{
"0": {
"state": "RUNNING",
"uptime": 293924,
"since": 1513383087
},
"1": {
"state": "RUNNING",
"uptime": 45,
"since": 1513676965
}
}
More Information
pasapicella@pas-macbook:~$ cf curl --help
NAME:
curl - Executes a request to the targeted API endpoint
USAGE:
cf curl PATH [-iv] [-X METHOD] [-H HEADER] [-d DATA] [--output FILE]
By default 'cf curl' will perform a GET to the specified PATH. If data
is provided via -d, a POST will be performed instead, and the Content-Type
will be set to application/json. You may override headers with -H and the
request method with -X.
For API documentation, please visit http://apidocs.cloudfoundry.org.
EXAMPLES:
cf curl "/v2/apps" -X GET -H "Content-Type: application/x-www-form-urlencoded" -d 'q=name:myapp'
cf curl "/v2/apps" -d @/path/to/file
OPTIONS:
-H Custom headers to include in the request, flag can be specified multiple times
-X HTTP method (GET,POST,PUT,DELETE,etc)
-d HTTP data to include in the request body, or '@' followed by a file name to read the data from
-i Include response headers in the output
--output Write curl body to FILE instead of stdout
Thursday, 23 November 2017
Taking Pivotal Cloud Foundry Small Footprint for a test drive
Pivotal Cloud Foundry (PCF) now has a small footprint edition. It features a deployment configuration with as few as 6 VMs. Review the documentation for download and installation instructions as follows
http://docs.pivotal.io/pivotalcf/1-12/customizing/small-footprint.html
There was also a Pivotal blog post on this as follows:
https://content.pivotal.io/blog/big-things-come-in-small-packages-getting-started-with-pivotal-cloud-foundry-small-footprint
As you can see from this image it's considerably smaller control plane that's obvious.
It is important to understand what the limitations of such an install are as per the docs link below.
http://docs.pivotal.io/pivotalcf/1-12/customizing/small-footprint.html#limits
Installing the small footprint looks identical from the Operations Manager UI in fact it's still labelled ERT and from the home page of Operations Manager UI your wouldn't even know you had the small footprint
If you dig a bit further and click on the "ERT tile" and then select "Resource Config" left hand link you will then clearly know it's the Small Footprint PCF install.
I choose to use internal MySQL database and if I didn't then it could be scaled back even more then the default 7 VM's I ended up with.
Lastly I was very curious to find out what jobs are placed on which service VM's. Here is what it looked like for me when I logged into bosh director and run some bosh CLI commands
Now, with Small Footprint, you have yet another way to bring PCF to your organization!
http://docs.pivotal.io/pivotalcf/1-12/customizing/small-footprint.html
There was also a Pivotal blog post on this as follows:
https://content.pivotal.io/blog/big-things-come-in-small-packages-getting-started-with-pivotal-cloud-foundry-small-footprint
As you can see from this image it's considerably smaller control plane that's obvious.
It is important to understand what the limitations of such an install are as per the docs link below.
http://docs.pivotal.io/pivotalcf/1-12/customizing/small-footprint.html#limits
Installing the small footprint looks identical from the Operations Manager UI in fact it's still labelled ERT and from the home page of Operations Manager UI your wouldn't even know you had the small footprint
If you dig a bit further and click on the "ERT tile" and then select "Resource Config" left hand link you will then clearly know it's the Small Footprint PCF install.
I choose to use internal MySQL database and if I didn't then it could be scaled back even more then the default 7 VM's I ended up with.
Lastly I was very curious to find out what jobs are placed on which service VM's. Here is what it looked like for me when I logged into bosh director and run some bosh CLI commands
ubuntu@ip-10-0-0-241:~$ bosh2 -e aws vms --column=Instance --column="Process State" --column=AZ --column="VM Type" Using environment '10.0.16.5' as user 'director' (bosh.*.read, openid, bosh.*.admin, bosh.read, bosh.admin) Task 73. Done Deployment 'cf-a96683b17697c86b8c90' Instance Process State AZ VM Type backup-prepare/16356c40-1f20-42f0-8f2e-de45549be797 running ap-southeast-2a t2.micro blobstore/b6e22107-018b-425d-8fe4-ab47eeaf2c75 running ap-southeast-2a m4.large compute/5439f18f-c842-40a2-b6f3-faf6b6848716 running ap-southeast-2a r4.xlarge control/68979d93-d12b-4d87-b110-d3d41a48b261 running ap-southeast-2a r4.xlarge database/a3efedaa-4df6-48f5-9f20-61cf3d9f3c1b running ap-southeast-2a r4.large mysql_monitor/d40ef638-d2d0-488e-b937-99a7f5b5b334 running ap-southeast-2a t2.micro router/f9573547-c5a1-43d4-be02-38a1d9e9c73e running ap-southeast-2a t2.micro 7 vms Succeeded
ubuntu@ip-10-0-0-241:~$ bosh2 -e aws instances --ps --column=Instance --column=Process Using environment '10.0.16.5' as user 'director' (bosh.*.read, openid, bosh.*.admin, bosh.read, bosh.admin) Task 68. Done Deployment 'cf-a96683b17697c86b8c90' Instance Process autoscaling-register-broker/9feaef45-994e-472c-8ca3-f0c39467dd6b - autoscaling/184df31d-a64c-49e0-8b6b-27eafdb31ca0 - backup-prepare/16356c40-1f20-42f0-8f2e-de45549be797 - ~ service-backup blobstore/b6e22107-018b-425d-8fe4-ab47eeaf2c75 - ~ blobstore_nginx ~ blobstore_url_signer ~ consul_agent ~ metron_agent ~ route_registrar bootstrap/0dc22a1f-a1ee-4a03-85c6-fed08f37c44a - compute/5439f18f-c842-40a2-b6f3-faf6b6848716 - ~ consul_agent ~ garden ~ iptables-logger ~ metron_agent ~ netmon ~ nfsv3driver ~ rep ~ route_emitter ~ silk-daemon ~ vxlan-policy-agent control/68979d93-d12b-4d87-b110-d3d41a48b261 - ~ adapter ~ auctioneer ~ bbs ~ cc_uploader ~ cloud_controller_clock ~ cloud_controller_ng ~ cloud_controller_worker_1 ~ cloud_controller_worker_local_1 ~ cloud_controller_worker_local_2 ~ consul_agent ~ doppler ~ file_server ~ locket ~ loggregator_trafficcontroller ~ metron_agent ~ nginx_cc ~ policy-server ~ reverse_log_proxy ~ route_registrar ~ routing-api ~ scheduler ~ silk-controller ~ ssh_proxy ~ statsd_injector ~ syslog_drain_binder ~ tps_watcher ~ uaa database/a3efedaa-4df6-48f5-9f20-61cf3d9f3c1b - ~ cluster_health_logger ~ consul_agent ~ galera-healthcheck ~ gra-log-purger-executable ~ mariadb_ctrl ~ metron_agent ~ mysql-diag-agent ~ mysql-metrics ~ nats ~ route_registrar ~ streaming-mysql-backup-tool ~ switchboard mysql-rejoin-unsafe/01a0aec3-b103-4c09-bc69-cabc61c513cc - mysql_monitor/d40ef638-d2d0-488e-b937-99a7f5b5b334 - ~ replication-canary nfsbrokerpush/829f0292-59ab-4824-8b0b-c4af4bddbce0 - notifications-ui/50972440-36cd-499d-ad7c-eef4df7e604b - notifications/968d625e-af63-4e2f-a59c-f6b789ef1cff - push-apps-manager/3d9760d7-6f09-453c-a052-32604b6a3235 - push-pivotal-account/5a8782ad-82eb-4469-8242-f0873bc4a587 - push-usage-service/9b781db1-171b-4abe-92f4-7445fd3d487f - router/f9573547-c5a1-43d4-be02-38a1d9e9c73e - ~ consul_agent ~ gorouter ~ metron_agent smoke-tests/a8e2ff97-bae1-4594-90fc-ec8c430fd620 - 77 instances Succeeded
Now, with Small Footprint, you have yet another way to bring PCF to your organization!
Sunday, 19 November 2017
Using Spring Boot Actuator endpoint for Spring Boot application health check type on PCF
An application health check is a monitoring process that continually checks the status of a running Cloud Foundry application. When deploying an app, a developer can configure the health check
type (port, process, or HTTP), a timeout for starting the application,
and an endpoint (for HTTP only) for the application health check.
To use the HTTP option your manifest.yml would look like this
---
applications:
- name: pas-cf-manifest
memory: 756M
instances: 1
hostname: pas-cf-manifest
path: ./target/demo-0.0.1-SNAPSHOT.jar
health-check-type: http
health-check-http-endpoint: /health
stack: cflinuxfs2
timeout: 80
env:
JAVA_OPTS: -Djava.security.egd=file:///dev/urandom
NAME: Apples
Using a HTTP endpoint such as "/health" is possible once you add the Spring Boot Actuator maven dependency as follows
More Information
https://docs.run.pivotal.io/devguide/deploy-apps/healthchecks.html
To use the HTTP option your manifest.yml would look like this
---
applications:
- name: pas-cf-manifest
memory: 756M
instances: 1
hostname: pas-cf-manifest
path: ./target/demo-0.0.1-SNAPSHOT.jar
health-check-type: http
health-check-http-endpoint: /health
stack: cflinuxfs2
timeout: 80
env:
JAVA_OPTS: -Djava.security.egd=file:///dev/urandom
NAME: Apples
Using a HTTP endpoint such as "/health" is possible once you add the Spring Boot Actuator maven dependency as follows
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency>
More Information
https://docs.run.pivotal.io/devguide/deploy-apps/healthchecks.html
Saturday, 28 October 2017
Testing network connectivity from Cloud Foundry Application Instances
This app below simply tests whether a host:port is accessible from a CF app instance. For example can my application
instance access my Oracle Database Instance running outside of PCF given application instances need network
access to the database database for example.
You can use bosh2 ssh to get to the Diego Cells if you have access to the environment or even "cf ssh" if that has been enabled.
GitHub URL:
https://github.com/papicella/cloudfoundry-socket-test
Success
You can use bosh2 ssh to get to the Diego Cells if you have access to the environment or even "cf ssh" if that has been enabled.
GitHub URL:
https://github.com/papicella/cloudfoundry-socket-test
Success
pasapicella@pas-macbook:~$ http http://pas-cf-sockettest.cfapps.io/www.google.com.au/80
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 81
Content-Type: application/json;charset=UTF-8
Date: Wed, 25 Oct 2017 08:38:33 GMT
X-Vcap-Request-Id: 8fd05c77-f680-4966-558b-c45e71825fa0
{
"errorMessage": "N/A",
"hostname": "www.google.com.au",
"port": "80",
"res": "SUCCESS"
}
Failurepasapicella@pas-macbook:~$ http http://pas-cf-sockettest.cfapps.io/10.0.0.10/8080
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 110
Content-Type: application/json;charset=UTF-8
Date: Wed, 25 Oct 2017 11:52:18 GMT
X-Vcap-Request-Id: 91ad2359-7d95-49c6-4538-548e480b7820
{
"errorMessage": "Connection refused (Connection refused)",
"hostname": "10.0.0.10",
"port": "8080",
"res": "FAILED"
}
Sunday, 22 October 2017
Just installed Pivotal Cloud Foundry, what's next should I login to Apps Manager?
I get this question often from customers. Pivotal Cloud Foundry has just been installed and the API endpoint to target the instance is working fine. In short we want to do the following before we get developers onto the platform to ensure we no longer using the UAA server admin login details from the CLI or Apps Manager UI.
--> Create a new ADMIN user which will be used to configure Apps Manager ORGS and spaces for the developers
1. Login to Ops Manager VM using SSH for example
2. Target the UAA server as shown below
Eg: $ uaac target uaa.YOUR-DOMAIN
ubuntu@opsmanager-pcf:~$ uaac target uaa.system.YYYY --skip-ssl-validation
Unknown key: Max-Age = 2592000
Target: https://uaa.system.YYYY
3. Authenticate and obtain an access token for the admin client from the UAA server
Note: Record the uaa:admin:client_secret from your deployment manifest
ubuntu@opsmanager-pcf:~$ uaac token client get admin -s PASSWD
Successfully fetched token via client credentials grant.
Target: https://uaa.system.YYYY
Context: admin, from client admin
4. Use the uaac contexts command to display the users and applications authorized by the UAA server, and the permissions granted to each user and application. Ensure in the "scope" field that "scim.write" exists
ubuntu@opsmanager-pcf:~$ uaac contexts
[0]*[https://uaa.system.YYYY]
skip_ssl_validation: true
[0]*[admin]
client_id: admin
access_token: .....
token_type: bearer
expires_in: 43199
scope: clients.read password.write clients.secret clients.write uaa.admin scim.write scim.read
jti: b1bf094a5c4640dbac4abc5f3bf15b08
5. Run the following command to create an admin user
ubuntu@opsmanager-pcf:~$ uaac user add apples -p PASSWD --emails papicella@pivotal.io
user account successfully added
6. Run uaac member add GROUP NEW-ADMIN-USERNAME to add the new admin to the groups cloud_controller.admin, uaa.admin, scim.read, and scim.write
ubuntu@opsmanager-pcf:~$ uaac member add cloud_controller.admin apples
success
ubuntu@opsmanager-pcf:~$ uaac member add uaa.admin apples
success
ubuntu@opsmanager-pcf:~$ uaac member add scim.read apples
success
ubuntu@opsmanager-pcf:~$ uaac member add scim.write apples
success
--> Create an ORG
1. Login using the new admin user "apples"
pasapicella@pas-macbook:~$ cf login -u apples -p PASSWD -o system -s system
API endpoint: https://api.system.YYYY
Authenticating...
OK
Targeted org system
Targeted space system
API endpoint: https://api.system.YYYY (API version: 2.94.0)
User: papicella@pivotal.io
Org: system
Space: system
2. Create an ORG as follows
pasapicella@pas-macbook:~$ cf create-org myfirst-org
Creating org myfirst-org as apples...
OK
Assigning role OrgManager to user apples in org myfirst-org ...
OK
TIP: Use 'cf target -o "myfirst-org"' to target new org
--> Create at least one Quota maybe more to control memory limit and application instances within an ORG
1. Here we create what I call a medium-quota which allows 20G of memory, 2 service instances, each application instance can be no more then 1G of memory and only 20 Application Instances can be created using this quota.
pasapicella@pas-macbook:~$ cf create-quota medium-quota -m 20G -i 1G -a 20 -s 2 -r 1000 --allow-paid-service-plans
Creating quota medium-quota as apples...
OK
pasapicella@pas-macbook:~$ cf quota medium-quota
Getting quota medium-quota info as apples...
OK
Total Memory 20G
Instance Memory 1G
Routes 1000
Services 2
Paid service plans allowed
App instance limit 20
Reserved Route Ports 0
--> Assign the quota to your ORG
1. Assign the newly created quota to the ORG we created above
pasapicella@pas-macbook:~$ cf set-quota myfirst-org medium-quota
Setting quota medium-quota to org myfirst-org as apples...
OK
pasapicella@pas-macbook:~$ cf org myfirst-org
Getting info for org myfirst-org as apples...
name: myfirst-org
domains: apps.pas-apples.online
quota: medium-quota
spaces:
isolation segments:
Finally we can add a space to the ORG and assign privileges to a user called "pas" as shown below
- Set OrgManager role to the user "pas"
pasapicella@pas-macbook:~$ cf set-org-role pas myfirst-org OrgManager
Assigning role OrgManager to user pas in org myfirst-org as apples...
OK
- Logout as "apples" admin user as "pas" can now do his own admin for the ORG " myfirst-org"
pasapicella@pas-macbook:~$ cf logout
Logging out...
OK
- Login as pas and target the ORG
pasapicella@pas-macbook:~$ cf login -u pas -p PASSWD -o myfirst-org
API endpoint: https://api.system.YYYY
Authenticating...
OK
Targeted org myfirst-org
API endpoint: https://api.system.YYYY (API version: 2.94.0)
User: pas
Org: myfirst-org
Space: No space targeted, use 'cf target -s SPACE'
- Create a space which will set space roles for the user "pas"
pasapicella@pas-macbook:~$ cf create-space dev
Creating space dev in org myfirst-org as pas...
OK
Assigning role RoleSpaceManager to user pas in org myfirst-org / space dev as pas...
OK
Assigning role RoleSpaceDeveloper to user pas in org myfirst-org / space dev as pas...
OK
TIP: Use 'cf target -o "myfirst-org" -s "dev"' to target new space
- Target the new space
pasapicella@pas-macbook:~$ cf target -o myfirst-org -s dev
api endpoint: https://api.system.pas-apples.online
api version: 2.94.0
user: pas
org: myfirst-org
space: dev
Typically we would assign other users to the spaces using "cf set-space-role .."
pasapicella@pas-macbook:~$ cf set-space-role --help
NAME:
set-space-role - Assign a space role to a user
USAGE:
cf set-space-role USERNAME ORG SPACE ROLE
ROLES:
'SpaceManager' - Invite and manage users, and enable features for a given space
'SpaceDeveloper' - Create and manage apps and services, and see logs and reports
'SpaceAuditor' - View logs, reports, and settings on this space
SEE ALSO:
space-users
More Information
Creating and Managing Users with the UAA CLI (UAAC)
https://docs.pivotal.io/pivotalcf/1-12/uaa/uaa-user-management.html
Creating and Managing Users with the cf CLI
https://docs.pivotal.io/pivotalcf/1-12/adminguide/cli-user-management.html
- Create a new ADMIN user which will be used to configure Apps Manager ORGS and spaces for the developers
- Create an ORG
- Create at least one Quota maybe more to control memory limit and application instances within an ORG
- Assign the quota to your ORG
--> Create a new ADMIN user which will be used to configure Apps Manager ORGS and spaces for the developers
1. Login to Ops Manager VM using SSH for example
2. Target the UAA server as shown below
Eg: $ uaac target uaa.YOUR-DOMAIN
ubuntu@opsmanager-pcf:~$ uaac target uaa.system.YYYY --skip-ssl-validation
Unknown key: Max-Age = 2592000
Target: https://uaa.system.YYYY
3. Authenticate and obtain an access token for the admin client from the UAA server
Note: Record the uaa:admin:client_secret from your deployment manifest
ubuntu@opsmanager-pcf:~$ uaac token client get admin -s PASSWD
Successfully fetched token via client credentials grant.
Target: https://uaa.system.YYYY
Context: admin, from client admin
4. Use the uaac contexts command to display the users and applications authorized by the UAA server, and the permissions granted to each user and application. Ensure in the "scope" field that "scim.write" exists
ubuntu@opsmanager-pcf:~$ uaac contexts
[0]*[https://uaa.system.YYYY]
skip_ssl_validation: true
[0]*[admin]
client_id: admin
access_token: .....
token_type: bearer
expires_in: 43199
scope: clients.read password.write clients.secret clients.write uaa.admin scim.write scim.read
jti: b1bf094a5c4640dbac4abc5f3bf15b08
5. Run the following command to create an admin user
ubuntu@opsmanager-pcf:~$ uaac user add apples -p PASSWD --emails papicella@pivotal.io
user account successfully added
6. Run uaac member add GROUP NEW-ADMIN-USERNAME to add the new admin to the groups cloud_controller.admin, uaa.admin, scim.read, and scim.write
ubuntu@opsmanager-pcf:~$ uaac member add cloud_controller.admin apples
success
ubuntu@opsmanager-pcf:~$ uaac member add uaa.admin apples
success
ubuntu@opsmanager-pcf:~$ uaac member add scim.read apples
success
ubuntu@opsmanager-pcf:~$ uaac member add scim.write apples
success
--> Create an ORG
1. Login using the new admin user "apples"
pasapicella@pas-macbook:~$ cf login -u apples -p PASSWD -o system -s system
API endpoint: https://api.system.YYYY
Authenticating...
OK
Targeted org system
Targeted space system
API endpoint: https://api.system.YYYY (API version: 2.94.0)
User: papicella@pivotal.io
Org: system
Space: system
2. Create an ORG as follows
pasapicella@pas-macbook:~$ cf create-org myfirst-org
Creating org myfirst-org as apples...
OK
Assigning role OrgManager to user apples in org myfirst-org ...
OK
TIP: Use 'cf target -o "myfirst-org"' to target new org
--> Create at least one Quota maybe more to control memory limit and application instances within an ORG
1. Here we create what I call a medium-quota which allows 20G of memory, 2 service instances, each application instance can be no more then 1G of memory and only 20 Application Instances can be created using this quota.
pasapicella@pas-macbook:~$ cf create-quota medium-quota -m 20G -i 1G -a 20 -s 2 -r 1000 --allow-paid-service-plans
Creating quota medium-quota as apples...
OK
pasapicella@pas-macbook:~$ cf quota medium-quota
Getting quota medium-quota info as apples...
OK
Total Memory 20G
Instance Memory 1G
Routes 1000
Services 2
Paid service plans allowed
App instance limit 20
Reserved Route Ports 0
--> Assign the quota to your ORG
1. Assign the newly created quota to the ORG we created above
pasapicella@pas-macbook:~$ cf set-quota myfirst-org medium-quota
Setting quota medium-quota to org myfirst-org as apples...
OK
pasapicella@pas-macbook:~$ cf org myfirst-org
Getting info for org myfirst-org as apples...
name: myfirst-org
domains: apps.pas-apples.online
quota: medium-quota
spaces:
isolation segments:
Finally we can add a space to the ORG and assign privileges to a user called "pas" as shown below
- Set OrgManager role to the user "pas"
pasapicella@pas-macbook:~$ cf set-org-role pas myfirst-org OrgManager
Assigning role OrgManager to user pas in org myfirst-org as apples...
OK
- Logout as "apples" admin user as "pas" can now do his own admin for the ORG " myfirst-org"
pasapicella@pas-macbook:~$ cf logout
Logging out...
OK
- Login as pas and target the ORG
pasapicella@pas-macbook:~$ cf login -u pas -p PASSWD -o myfirst-org
API endpoint: https://api.system.YYYY
Authenticating...
OK
Targeted org myfirst-org
API endpoint: https://api.system.YYYY (API version: 2.94.0)
User: pas
Org: myfirst-org
Space: No space targeted, use 'cf target -s SPACE'
- Create a space which will set space roles for the user "pas"
pasapicella@pas-macbook:~$ cf create-space dev
Creating space dev in org myfirst-org as pas...
OK
Assigning role RoleSpaceManager to user pas in org myfirst-org / space dev as pas...
OK
Assigning role RoleSpaceDeveloper to user pas in org myfirst-org / space dev as pas...
OK
TIP: Use 'cf target -o "myfirst-org" -s "dev"' to target new space
- Target the new space
pasapicella@pas-macbook:~$ cf target -o myfirst-org -s dev
api endpoint: https://api.system.pas-apples.online
api version: 2.94.0
user: pas
org: myfirst-org
space: dev
Typically we would assign other users to the spaces using "cf set-space-role .."
pasapicella@pas-macbook:~$ cf set-space-role --help
NAME:
set-space-role - Assign a space role to a user
USAGE:
cf set-space-role USERNAME ORG SPACE ROLE
ROLES:
'SpaceManager' - Invite and manage users, and enable features for a given space
'SpaceDeveloper' - Create and manage apps and services, and see logs and reports
'SpaceAuditor' - View logs, reports, and settings on this space
SEE ALSO:
space-users
More Information
Creating and Managing Users with the UAA CLI (UAAC)
https://docs.pivotal.io/pivotalcf/1-12/uaa/uaa-user-management.html
Creating and Managing Users with the cf CLI
https://docs.pivotal.io/pivotalcf/1-12/adminguide/cli-user-management.html
Thursday, 5 October 2017
Pivotal Cloud Foundry 1.12 on Google Cloud Platform with VM labels
Once PCF is installed on GCP it's worth noting that viewing the "Compute Engine" labels gives you as indication of what VM each CF service is associated with. The screen shots below show's this.
Monday, 25 September 2017
Updating Cloud Foundry CLI using Brew
Need to upgrade the CF CLI using brew it's as simple as below. Go to love brew
pasapicella@pas-macbook:~$ brew upgrade cf-cli
==> Upgrading 1 outdated package, with result:
cloudfoundry/tap/cf-cli 6.31.0
==> Upgrading cloudfoundry/tap/cf-cli
Warning: Use cloudfoundry/tap/cloudfoundry-cli instead of deprecated pivotal/tap/cloudfoundry-cli
==> Downloading https://cli.run.pivotal.io/stable?release=macosx64-binary&version=6.31.0&source=homebrew
==> Downloading from https://s3-us-west-1.amazonaws.com/cf-cli-releases/releases/v6.31.0/cf-cli_6.31.0_osx.tgz
######################################################################## 100.0%
==> Caveats
Bash completion has been installed to:
/usr/local/etc/bash_completion.d
==> Summary
🍺 /usr/local/Cellar/cf-cli/6.31.0: 6 files, 17.6MB, built in 16 seconds
pasapicella@pas-macbook:~$ cf --version
cf version 6.31.0+b35df905d.2017-09-15
pasapicella@pas-macbook:~$ brew upgrade cf-cli
==> Upgrading 1 outdated package, with result:
cloudfoundry/tap/cf-cli 6.31.0
==> Upgrading cloudfoundry/tap/cf-cli
Warning: Use cloudfoundry/tap/cloudfoundry-cli instead of deprecated pivotal/tap/cloudfoundry-cli
==> Downloading https://cli.run.pivotal.io/stable?release=macosx64-binary&version=6.31.0&source=homebrew
==> Downloading from https://s3-us-west-1.amazonaws.com/cf-cli-releases/releases/v6.31.0/cf-cli_6.31.0_osx.tgz
######################################################################## 100.0%
==> Caveats
Bash completion has been installed to:
/usr/local/etc/bash_completion.d
==> Summary
🍺 /usr/local/Cellar/cf-cli/6.31.0: 6 files, 17.6MB, built in 16 seconds
pasapicella@pas-macbook:~$ cf --version
cf version 6.31.0+b35df905d.2017-09-15
Friday, 15 September 2017
Using Cloud Foundry CUPS to inject Spring Security credentials into a Spring Boot Application
The following demo shows how to inject the Spring Security
username/password credentials from a User Provided service on PCF,
hence using the VCAP_SERVICES env variable to inject the values required
to protect the application using HTTP Basic Authentication
while running in PCF. Spring Boot automatically converts this data into a
flat set of properties so you can easily get to the data
as shown below.
The demo application can be found as follows
https://github.com/papicella/springsecurity-cf-cups
The application.yml would access the VCAP_SERVICES CF env variable using the the Spring Boot flat set of properties as shown below.
eg:
VCAP_SERVICES
System-Provided:
{
"VCAP_SERVICES": {
"user-provided": [
{
"credentials": {
"password": "myadminpassword",
"username": "myadminuser"
},
"label": "user-provided",
"name": "my-cfcups-service",
"syslog_drain_url": "",
"tags": [],
"volume_mounts": []
}
]
}
}
...
application.yml
spring:
application:
name: security-cf-cups-demo
security:
user:
name: ${vcap.services.my-cfcups-service.credentials.username:admin}
password: ${vcap.services.my-cfcups-service.credentials.password:password}
The demo application can be found as follows
https://github.com/papicella/springsecurity-cf-cups
The application.yml would access the VCAP_SERVICES CF env variable using the the Spring Boot flat set of properties as shown below.
eg:
VCAP_SERVICES
System-Provided:
{
"VCAP_SERVICES": {
"user-provided": [
{
"credentials": {
"password": "myadminpassword",
"username": "myadminuser"
},
"label": "user-provided",
"name": "my-cfcups-service",
"syslog_drain_url": "",
"tags": [],
"volume_mounts": []
}
]
}
}
...
application.yml
spring:
application:
name: security-cf-cups-demo
security:
user:
name: ${vcap.services.my-cfcups-service.credentials.username:admin}
password: ${vcap.services.my-cfcups-service.credentials.password:password}
Thursday, 14 September 2017
Oracle 12c Service Broker for Pivotal Cloud Foundry
The following example is a PCF 2.0 Service Broker written as a Spring Boot application. This is just an example and should
be evolved to match a production type setup in terms oracle requirements. This service broker simple creates USERS and
assigns then 20M of quota against a known TABLESPACE
It's all documented as follows
https://github.com/papicella/oracle-service-broker
It's all documented as follows
https://github.com/papicella/oracle-service-broker
Monday, 4 September 2017
Introducing Pivotal MySQL*Web, Pivotal’s New Open Source Web-Based Administration UI for MySQL for Pivotal Cloud Foundry
Recently Pivotal announced "Pivotal MySQL*Web" on it's main blog page. You can read more about it here which was an Open Source project I created a while ago for Pivotal MySQL instances on Pivotal Cloud Foundry
https://content.pivotal.io/blog/introducing-pivotal-mysql-web-pivotal-s-new-open-source-web-based-administration-ui-for-mysql-for-pivotal-cloud-foundry
https://content.pivotal.io/blog/introducing-pivotal-mysql-web-pivotal-s-new-open-source-web-based-administration-ui-for-mysql-for-pivotal-cloud-foundry
Couchbase Service Broker for Pivotal Cloud Foundry
The following example is a PCF 2.0 Service Broker written as a Spring Boot application for Couchbase 4.6.x. This is just an example and should be evolved to match a production type setup in terms of bucket creation and access for created service instances.
It's all documented as follows
https://github.com/papicella/couchbase-service-broker
More Information
https://docs.pivotal.io/tiledev/service-brokers.html
It's all documented as follows
https://github.com/papicella/couchbase-service-broker
More Information
https://docs.pivotal.io/tiledev/service-brokers.html
Thursday, 27 July 2017
Accessing Pivotal Cloud Foundry droplet file system when "cf ssh" isn't enabled
In order to view your application layout you can simply use "cf ssh" to log into the container and then view the files created as part of the droplet. The problem is "cf ssh" isn't always enabled bye the Ops team so what is your alternative in cloud foundry?
You can use "cf curl" to invoke an endpoint using the application GUID as shown in the steps below.
** cf ssh demo **
pasapicella@pas-macbook:~/temp/droplets$ cf ssh pas-swagger-demo
vcap@ef9e4e93-0df9-47a7-5351-dccf:~$ ls -lartF
total 16
-rw-r--r-- 1 vcap vcap 675 Apr 9 2014 .profile
-rw-r--r-- 1 vcap vcap 3637 Apr 9 2014 .bashrc
-rw-r--r-- 1 vcap vcap 220 Apr 9 2014 .bash_logout
drwxr-xr-x 2 vcap vcap 6 Jun 14 03:32 deps/
drwxr-xr-x 1 vcap root 72 Jun 14 03:32 app/
-rw-r--r-- 1 vcap vcap 1087 Jun 14 03:32 staging_info.yml
drwxr-xr-x 2 vcap vcap 6 Jun 14 03:32 logs/
drwx------ 1 vcap vcap 76 Jun 14 03:32 ./
drwxr-xr-x 1 root root 18 Jul 26 23:45 ../
drwxr-xr-x 4 vcap vcap 92 Jul 26 23:48 tmp/
vcap@ef9e4e93-0df9-47a7-5351-dccf:~$
** Steps **
You can use "cf curl" to invoke an endpoint using the application GUID as shown in the steps below.
** cf ssh demo **
pasapicella@pas-macbook:~/temp/droplets$ cf ssh pas-swagger-demo
vcap@ef9e4e93-0df9-47a7-5351-dccf:~$ ls -lartF
total 16
-rw-r--r-- 1 vcap vcap 675 Apr 9 2014 .profile
-rw-r--r-- 1 vcap vcap 3637 Apr 9 2014 .bashrc
-rw-r--r-- 1 vcap vcap 220 Apr 9 2014 .bash_logout
drwxr-xr-x 2 vcap vcap 6 Jun 14 03:32 deps/
drwxr-xr-x 1 vcap root 72 Jun 14 03:32 app/
-rw-r--r-- 1 vcap vcap 1087 Jun 14 03:32 staging_info.yml
drwxr-xr-x 2 vcap vcap 6 Jun 14 03:32 logs/
drwx------ 1 vcap vcap 76 Jun 14 03:32 ./
drwxr-xr-x 1 root root 18 Jul 26 23:45 ../
drwxr-xr-x 4 vcap vcap 92 Jul 26 23:48 tmp/
vcap@ef9e4e93-0df9-47a7-5351-dccf:~$
** Steps **
1. Download droplet as follows
Format:
cf curl /v2/apps/`cf app {appname} --guid`/droplet/download > droplet.tar.gz
Example:
pasapicella@pas-macbook:~/temp/droplets$ cf curl /v2/apps/`cf app pas-swagger-demo --guid`/droplet/download > droplet.tar.gz
To determine the app name you can either use Applications manager UI or use "cf apps" to get the app name
2. This will take some time due to the size of the droplet but when done verify you have this on the file system
pasapicella@pas-macbook:~/temp/droplets$ ls -la
total 150736
drwxr-xr-x 3 pasapicella staff 102 Jul 27 14:20 .
drwxr-xr-x 23 pasapicella staff 782 Jul 27 14:19 ..
-rw-r--r-- 1 pasapicella staff 77173173 Jul 27 14:23 droplet.tar.gz
3. Gunzip followed by tar -xvf and you will then have a file system replicator of what your application droplet looks like in CF
pasapicella@pas-macbook:~/temp/droplets$ d
total 313408
drwxr-xr-x 2 pasapicella staff 68 Jun 14 13:32 deps/
drwxr-xr-x 6 pasapicella staff 204 Jun 14 13:32 app/
drwxr-xr-x 2 pasapicella staff 68 Jun 14 13:32 tmp/
-rw-r--r-- 1 pasapicella staff 1087 Jun 14 13:32 staging_info.yml
drwxr-xr-x 2 pasapicella staff 68 Jun 14 13:32 logs/
drwxr-xr-x 23 pasapicella staff 782 Jul 27 14:19 ../
-rw-r--r-- 1 pasapicella staff 160460800 Jul 27 14:23 droplet.tar
drwxr-xr-x 8 pasapicella staff 272 Jul 27 14:25 ./
You really only want to do this to see how your application was staged on the file system as the buildpack may have changed some files or added files based on what you deployed. This is not how you would debug an application but rather view what the file system looks like for your application itself and what content exists in the files should the buildpack have changed file content for example.
Monday, 24 July 2017
Pivotal Cloud Foundry Isolation Segments Applications demo
PCF Isolation Segments Tile allows operators to isolate deployment workloads into dedicated resource pools called isolation segments. You can read more about how to install the Tile at the following location.
https://docs.pivotal.io/pivotalcf/1-11/opsguide/installing-pcf-is.html
In this demo I will show how you can configure your ORGS/spaces to use an isolation segment and then finally show that it is indeed using the isolated Diego Cells assigned to the segment tile at install time.
1. Determine the isolation segment name as per the Ops Manager title
2. Ensure you have CF CLI version as 6.26 or higher as shown below
pasapicella@pas-macbook:~$ cf --version
cf version 6.28.0+9e024bdbd.2017-06-27
3. First, you need to register an Isolation Segment with the cloud controller.
Note: you will need a prividledged user such as admin to perform this task
$ pasapicella@pas-macbook:~$ cf create-isolation-segment segment1
Creating isolation segment segment1 as admin...
OK
4. After registering the Isolation Segment, you can associate it with an Org
pasapicella@pas-macbook:~$ cf enable-org-isolation pas-org segment1
Enabling isolation segment segment1 for org pas-org as admin...
OK
5. List isolation segments against targeted ORGS as follows
pasapicella@pas-macbook:~$ cf isolation-segments
Getting isolation segments as admin...
OK
name orgs
shared
segment1 pas-org
https://docs.pivotal.io/pivotalcf/1-11/opsguide/installing-pcf-is.html
In this demo I will show how you can configure your ORGS/spaces to use an isolation segment and then finally show that it is indeed using the isolated Diego Cells assigned to the segment tile at install time.
1. Determine the isolation segment name as per the Ops Manager title
2. Ensure you have CF CLI version as 6.26 or higher as shown below
pasapicella@pas-macbook:~$ cf --version
cf version 6.28.0+9e024bdbd.2017-06-27
3. First, you need to register an Isolation Segment with the cloud controller.
Note: you will need a prividledged user such as admin to perform this task
$ pasapicella@pas-macbook:~$ cf create-isolation-segment segment1
Creating isolation segment segment1 as admin...
OK
4. After registering the Isolation Segment, you can associate it with an Org
pasapicella@pas-macbook:~$ cf enable-org-isolation pas-org segment1
Enabling isolation segment segment1 for org pas-org as admin...
OK
5. List isolation segments against targeted ORGS as follows
pasapicella@pas-macbook:~$ cf isolation-segments
Getting isolation segments as admin...
OK
name orgs
shared
segment1 pas-org
6. After associating an Isolation Segments with an Org, you then set it on a Space
- First target the ORG you wish to use
pasapicella@pas-macbook:~$ cf target -o pas-org
api endpoint: https://api.yyyy.pcfdemo.yyy
api version: 2.82.0
user: admin
org: pas-org
No space targeted, use 'cf target -s SPACE'
- Now set the isolation segment on your desired space
pasapicella@pas-macbook:~$ cf set-space-isolation-segment isolation-segment-space segment1
Updating isolation segment of space isolation-segment-space in org pas-org as admin...
OK
In order to move running applications to this isolation segment, they must be restarted.
7. Log in as a user of the ORG/SPACE now rather then ad admin user. We will be using a
non privileged user now to push our app who has access to the ORG/Space we targeted
above. Below just shows we have switched to a different user here.
pasapicella@pas-macbook:~$ cf target
api endpoint: https://api.yyyy.pcfdemo.yyy
api version: 2.82.0
user: pas
org: pas-org
space: isolation-segment-space
8. Push an application to the space "isolation-segment-space"
pasapicella@pas-macbook:~/piv-projects/PivotalSpringBootJPA$ cf push -f manifest-inmemory-db.yml
Using manifest file manifest-inmemory-db.yml
Creating app pas-albums in org pas-org / space isolation-segment-space as pas...
OK
....
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
1 of 1 instances running
App started
OK
...
state since cpu memory disk details
#0 running 2017-07-24 02:33:38 PM 225.3% 330.1M of 512M 162.8M of 1G
9. Identify IP address of diego CELL the app is running in
pasapicella@pas-macbook:~/piv-projects/PivotalSpringBootJPA$ cf curl /v2/apps/$(cf app pas-albums --guid)/stats | jq '.[].stats.host'
"10.10.10.71"
With this information you can verify the IP address of the Diego Cells you have for the Isolation Segment as follows using Pivotal Ops Manager Tile tabs. From the images below it's clear 10.10.10.71 is a diego cell IP address of our isolation segment.
More Information
Wednesday, 14 June 2017
Swagger UI with Spring Boot 1.5.x
I recently created this demo / blog entry on using HTTPIE with Spring Boot Rest Repositories as shown below.
http://theblasfrompas.blogspot.com.au/2017/05/using-httpie-with-spring-boot-rest.html
I decided to take that same example and add Swagger UI to the RESTful endpoints. The full source code is here.
https://github.com/papicella/httpie-springboot
In short what you need is the following maven dependancies and that will add all you need. I found it works much cleaner if you use the same version of both these dependancies for some reason
Finally a Class file describing the config and enabling Swagger is required as follows
The GitHub repo also included a Pivotal Cloud Foundry manifest.yml file to make it easy to deploy to Pivotal Cloud Foundry. The example uses a static hostname BUT can easily be changed to use a random-route or alter the hostname itself.
applications:
- name: pas-swagger-demo
memory: 1G
instances: 1
hostname: pas-swagger-demo
path: ./target/httpie-springboot-0.0.1-SNAPSHOT.jar
env:
JAVA_OPTS: -Djava.security.egd=file:///dev/urando
Then it's the simple "cf push"
$ cf push
pasapicella@pas-macbook:~/piv-projects/httpie-springboot$ cf push
Using manifest file /Users/pasapicella/piv-projects/httpie-springboot/manifest.yml
Creating app pas-swagger-demo in org apples-pivotal-org / space development as papicella@pivotal.io...
OK
Creating route pas-swagger-demo.cfapps.io...
OK
..
Showing health and status for app pas-swagger-demo in org apples-pivotal-org / space development as papicella@pivotal.io...
OK
requested state: started
instances: 1/1
usage: 1G x 1 instances
urls: pas-swagger-demo.cfapps.io
last uploaded: Wed Jun 14 03:32:31 UTC 2017
stack: cflinuxfs2
buildpack: container-certificate-trust-store=2.0.0_RELEASE java-buildpack=v3.15-offline-https://github.com/cloudfoundry/java-buildpack.git#a3a9e61 java-main java-opts open-jdk-like-jre=1.8.0_121 open-jdk-like-memory-calculator=2.0.2_RELEASE spring-auto-reconfigur...
state since cpu memory disk details
#0 running 2017-06-14 01:33:40 PM 291.5% 510.9M of 1G 154.9M of 1G
The application is running on Pivotal Web Services as follows:
http://pas-swagger-demo.cfapps.io/swagger-ui.html
http://theblasfrompas.blogspot.com.au/2017/05/using-httpie-with-spring-boot-rest.html
I decided to take that same example and add Swagger UI to the RESTful endpoints. The full source code is here.
https://github.com/papicella/httpie-springboot
In short what you need is the following maven dependancies and that will add all you need. I found it works much cleaner if you use the same version of both these dependancies for some reason
<dependency> <groupId>io.springfox</groupId> <artifactId>springfox-swagger2</artifactId> <version>2.6.1</version> </dependency> <dependency> <groupId>io.springfox</groupId> <artifactId>springfox-swagger-ui</artifactId> <version>2.6.1</version> </dependency>
Finally a Class file describing the config and enabling Swagger is required as follows
package pivotal.io.boot.httpie.demo; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import springfox.documentation.builders.RequestHandlerSelectors; import springfox.documentation.service.ApiInfo; import springfox.documentation.service.Contact; import springfox.documentation.spi.DocumentationType; import springfox.documentation.spring.web.plugins.Docket; import springfox.documentation.swagger2.annotations.EnableSwagger2; import static springfox.documentation.builders.PathSelectors.regex; @Configuration @EnableSwagger2 public class SwaggerConfig { @Bean public Docket swaggerSpringMvcPlugin() { return new Docket(DocumentationType.SWAGGER_2) .select() .apis(RequestHandlerSelectors.basePackage("pivotal.io.boot.httpie.demo")) .paths(regex("/api/employee/emps.*")) .build() .apiInfo(metaData()); } private ApiInfo metaData() { ApiInfo apiInfo = new ApiInfo( "Spring Boot Employee REST API", "Spring Boot Employee REST API", "1.0", "Terms of service", new Contact("Pas Apicella", "https://www.blogger.com/profile/09389663166398991762", "papicella@pivotal.io"), "Apache License Version 2.0", "https://www.apache.org/licenses/LICENSE-2.0"); return apiInfo; } }
The GitHub repo also included a Pivotal Cloud Foundry manifest.yml file to make it easy to deploy to Pivotal Cloud Foundry. The example uses a static hostname BUT can easily be changed to use a random-route or alter the hostname itself.
applications:
- name: pas-swagger-demo
memory: 1G
instances: 1
hostname: pas-swagger-demo
path: ./target/httpie-springboot-0.0.1-SNAPSHOT.jar
env:
JAVA_OPTS: -Djava.security.egd=file:///dev/urando
Then it's the simple "cf push"
$ cf push
pasapicella@pas-macbook:~/piv-projects/httpie-springboot$ cf push
Using manifest file /Users/pasapicella/piv-projects/httpie-springboot/manifest.yml
Creating app pas-swagger-demo in org apples-pivotal-org / space development as papicella@pivotal.io...
OK
Creating route pas-swagger-demo.cfapps.io...
OK
..
Showing health and status for app pas-swagger-demo in org apples-pivotal-org / space development as papicella@pivotal.io...
OK
requested state: started
instances: 1/1
usage: 1G x 1 instances
urls: pas-swagger-demo.cfapps.io
last uploaded: Wed Jun 14 03:32:31 UTC 2017
stack: cflinuxfs2
buildpack: container-certificate-trust-store=2.0.0_RELEASE java-buildpack=v3.15-offline-https://github.com/cloudfoundry/java-buildpack.git#a3a9e61 java-main java-opts open-jdk-like-jre=1.8.0_121 open-jdk-like-memory-calculator=2.0.2_RELEASE spring-auto-reconfigur...
state since cpu memory disk details
#0 running 2017-06-14 01:33:40 PM 291.5% 510.9M of 1G 154.9M of 1G
The application is running on Pivotal Web Services as follows:
http://pas-swagger-demo.cfapps.io/swagger-ui.html
Monday, 22 May 2017
Using HTTPIE with Spring Boot Rest Repositories
I recently got introduced to HTTPIE as a command line alternative to CURL for testing RESTful api endpoints created using @RestController annotated classes. For more information on httpie follow this link
Before we test this out lets create a very basic Spring Boot Application with classes/interfaces to verify HTTPIE. The following assumes you have a Spring Boot application already created and it has maven dependancies as follows to enable JPA, Rest Repositories, H2 and Web support
Note: We are using Spring Boot 1.5.3 here
1. Create classes/interfaces as follows
Employee.java
EmployeeRepository.java
EmployeeRest.java
2. Run the Spring Boot Application which will run on port localhost:8080
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v1.5.3.RELEASE)
2017-05-22 13:39:22.910 INFO 8875 --- [ main] p.i.b.h.d.HttpieSpringbootApplication : Starting HttpieSpringbootApplication on pas-macbook with PID 8875 (/Users/pasapicella/pivotal/DemoProjects/spring-starter/pivotal/httpie-springboot/target/classes started by pasapicella in /Users/pasapicella/pivotal/DemoProjects/spring-starter/pivotal/httpie-springboot)
Now we can test HTTPIE and here are some endpoints
3. Here are some examples with output
** All Employees **
pasapicella@pas-macbook:~$ http http://localhost:8080/api/employee/emps
HTTP/1.1 200
Content-Type: application/json;charset=UTF-8
Date: Mon, 22 May 2017 01:26:43 GMT
Transfer-Encoding: chunked
[
{
"firstName": "pas",
"id": 1,
"job": "CEO",
"lastName": "Apicella"
},
{
"firstName": "lucia",
"id": 2,
"job": "CIO",
"lastName": "Apicella"
},
{
"firstName": "lucas",
"id": 3,
"job": "MANAGER",
"lastName": "Apicella"
},
{
"firstName": "siena",
"id": 4,
"job": "CLERK",
"lastName": "Apicella"
}
]
Before we test this out lets create a very basic Spring Boot Application with classes/interfaces to verify HTTPIE. The following assumes you have a Spring Boot application already created and it has maven dependancies as follows to enable JPA, Rest Repositories, H2 and Web support
Note: We are using Spring Boot 1.5.3 here
<parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>1.5.3.RELEASE</version> <relativePath/> <!-- lookup parent from repository --> </parent> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <java.version>1.8</java.version> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-rest</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-jpa</artifactId> </dependency> <dependency> <groupId>org.hibernate</groupId> <artifactId>hibernate-entitymanager</artifactId> </dependency> <dependency> <groupId>com.h2database</groupId> <artifactId>h2</artifactId> <scope>runtime</scope> </dependency> </dependencies>
1. Create classes/interfaces as follows
Employee.java
package pivotal.io.boot.httpie.demo; import javax.persistence.Entity; import javax.persistence.GeneratedValue; import javax.persistence.GenerationType; import javax.persistence.Id; @Entity public class Employee { @Id @GeneratedValue (strategy = GenerationType.AUTO) private Long id; private String firstName; private String lastName; private String job; public Employee() { } public Employee(String firstName, String lastName, String job) { this.firstName = firstName; this.lastName = lastName; this.job = job; } public Long getId() { return id; } public void setId(Long id) { this.id = id; } public String getFirstName() { return firstName; } public void setFirstName(String firstName) { this.firstName = firstName; } public String getLastName() { return lastName; } public void setLastName(String lastName) { this.lastName = lastName; } public String getJob() { return job; } public void setJob(String job) { this.job = job; } @Override public String toString() { return "Employee{" + "id=" + id + ", firstName='" + firstName + '\'' + ", lastName='" + lastName + '\'' + ", job='" + job + '\'' + '}'; } }
EmployeeRepository.java
package pivotal.io.boot.httpie.demo; import org.springframework.data.jpa.repository.JpaRepository; public interface EmployeeRepository extends JpaRepository <Employee, Long> { }
EmployeeRest.java
package pivotal.io.boot.httpie.demo; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.*; import java.util.List; @RestController @RequestMapping ("/api/employee") public class EmployeeRest { private static Log logger = LogFactory.getLog(EmployeeRest.class); @Autowired private EmployeeRepository employeeRepository; @GetMapping("/emps") public List<Employee> allEmployees() { return employeeRepository.findAll(); } @GetMapping("/emp/{employeeId}") public Employee findEmployee (@PathVariable Long employeeId) { Employee emp = employeeRepository.findOne(employeeId); return emp; } @PostMapping("/emps") public Employee createEmployee(@RequestBody Employee employee) { return employeeRepository.save(employee); } @DeleteMapping("/emps/{employeeId}") public void deleteEmployee(@PathVariable Long employeeId) { Employee emp = employeeRepository.findOne(employeeId); employeeRepository.delete(emp); logger.info("Employee with id " + employeeId + " deleted..."); } }
2. Run the Spring Boot Application which will run on port localhost:8080
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v1.5.3.RELEASE)
2017-05-22 13:39:22.910 INFO 8875 --- [ main] p.i.b.h.d.HttpieSpringbootApplication : Starting HttpieSpringbootApplication on pas-macbook with PID 8875 (/Users/pasapicella/pivotal/DemoProjects/spring-starter/pivotal/httpie-springboot/target/classes started by pasapicella in /Users/pasapicella/pivotal/DemoProjects/spring-starter/pivotal/httpie-springboot)
...
2017-05-22 13:39:25.948 INFO 8875 --- [ main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8080 (http)
2017-05-22 13:39:25.952 INFO 8875 --- [ main] p.i.b.h.d.HttpieSpringbootApplication : Started HttpieSpringbootApplication in 3.282 seconds (JVM running for 3.676)
3. Here are some examples with output
** All Employees **
pasapicella@pas-macbook:~$ http http://localhost:8080/api/employee/emps
HTTP/1.1 200
Content-Type: application/json;charset=UTF-8
Date: Mon, 22 May 2017 01:26:43 GMT
Transfer-Encoding: chunked
[
{
"firstName": "pas",
"id": 1,
"job": "CEO",
"lastName": "Apicella"
},
{
"firstName": "lucia",
"id": 2,
"job": "CIO",
"lastName": "Apicella"
},
{
"firstName": "lucas",
"id": 3,
"job": "MANAGER",
"lastName": "Apicella"
},
{
"firstName": "siena",
"id": 4,
"job": "CLERK",
"lastName": "Apicella"
}
]
** Find Employee by {employeeId} **
pasapicella@pas-macbook:~$ http http://localhost:8080/api/employee/emp/1
HTTP/1.1 200
Content-Type: application/json;charset=UTF-8
Date: Mon, 22 May 2017 01:31:32 GMT
Transfer-Encoding: chunked
{
"firstName": "pas",
"id": 1,
"job": "CEO",
"lastName": "Apicella"
}
** POST new employee **
pasapicella@pas-macbook:~$ http POST http://localhost:8080/api/employee/emps firstName=john lastName=black job=CLERK
HTTP/1.1 200
Content-Type: application/json;charset=UTF-8
Date: Mon, 22 May 2017 02:32:34 GMT
Transfer-Encoding: chunked
{
"firstName": "john",
"id": 5,
"job": "CLERK",
"lastName": "black"
}
** POST with updated employee object **
pasapicella@pas-macbook:~$ http POST http://localhost:8080/api/employee/emps id:=5 firstName=john lastName=black job=CLEANER
HTTP/1.1 200
Content-Type: application/json;charset=UTF-8
Date: Mon, 22 May 2017 02:36:06 GMT
Transfer-Encoding: chunked
{
"firstName": "john",
"id": 5,
"job": "CLEANER",
"lastName": "black"
}
** Delete employee with {employeeId} 5 **
pasapicella@pas-macbook:~$ http DELETE http://localhost:8080/api/employee/emps/5
HTTP/1.1 200
Content-Length: 0
Date: Mon, 22 May 2017 02:36:56 GMT
Tuesday, 2 May 2017
Binding a Spring Cloud Task to a Pivotal Cloud Foundry Database Service
I previously blogged about how to create and deploy a Spring Cloud Task to Pivotal Cloud Foundry (PCF) as shown below.
http://theblasfrompas.blogspot.com.au/2017/03/run-spring-cloud-task-from-pivotal.html
Taking that same example I have used the Spring Cloud Connectors to persist the log output to a database table to avoid looking through log files to view the output. Few things have to change to make this happen as detailed below.
1. We need to change the manifest.yml to include a MySQL service instance as shown below
applications:
- name: springcloudtask-date
memory: 750M
instances: 1
no-route: true
health-check-type: none
path: ./target/springcloudtasktodaysdate-0.0.1-SNAPSHOT.jar
services:
- pmysql-test
env:
JAVA_OPTS: -Djava.security.egd=file:///dev/urando
2. Alter the project dependancies to include Spring Data JPA libraries to persist the log output to a table. Spring Cloud Connectors will automatically pick up the bound MySQL instance and connect for us when we push the application to PCF
https://github.com/papicella/SpringCloudTaskTodaysDate
3. A Entity class, Spring JPA repository interface and a JPA task Configurer has been created for persisting the log output as shown in the code below.
TaskRunOutput.java
TaskRepository.java
JpaTaskConfigurer.java
4. Now as per the previous blog execute the task and verify it completes without error. The screen shot below shows how the "Tasks" tab shows this
Note: You would need to PUSH the application to Pivotal Cloud Foundry before you can execute it which is shown on the original blog entry
5. Now if you follow this blog entry below you can deploy a Web Based interface for Pivotal MySQL instance to view the table and it's output
http://theblasfrompas.blogspot.com.au/2017/04/accessing-pivotal-mysql-service.html
With Pivotal MySQL*Web installed the output can be viewed as shown below.
http://theblasfrompas.blogspot.com.au/2017/03/run-spring-cloud-task-from-pivotal.html
Taking that same example I have used the Spring Cloud Connectors to persist the log output to a database table to avoid looking through log files to view the output. Few things have to change to make this happen as detailed below.
1. We need to change the manifest.yml to include a MySQL service instance as shown below
applications:
- name: springcloudtask-date
memory: 750M
instances: 1
no-route: true
health-check-type: none
path: ./target/springcloudtasktodaysdate-0.0.1-SNAPSHOT.jar
services:
- pmysql-test
env:
JAVA_OPTS: -Djava.security.egd=file:///dev/urando
2. Alter the project dependancies to include Spring Data JPA libraries to persist the log output to a table. Spring Cloud Connectors will automatically pick up the bound MySQL instance and connect for us when we push the application to PCF
https://github.com/papicella/SpringCloudTaskTodaysDate
<dependencies> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-task</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-jpa</artifactId> </dependency> <dependency> <groupId>org.hibernate</groupId> <artifactId>hibernate-entitymanager</artifactId> </dependency> <dependency> <groupId>com.h2database</groupId> <artifactId>h2</artifactId> </dependency> <dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <scope>runtime</scope> </dependency> </dependencies>
3. A Entity class, Spring JPA repository interface and a JPA task Configurer has been created for persisting the log output as shown in the code below.
TaskRunOutput.java
package pas.au.pivotal.pa.sct.demo; import javax.persistence.*; @Entity @Table (name = "TASKRUNOUTPUT") public class TaskRunOutput { @Id @GeneratedValue(strategy = GenerationType.AUTO) private Long id; private String output; public TaskRunOutput() { } public TaskRunOutput(String output) { this.output = output; } public Long getId() { return id; } public void setId(Long id) { this.id = id; } public String getOutput() { return output; } public void setOutput(String output) { this.output = output; } @Override public String toString() { return "TaskRunOutput{" + "id=" + id + ", output='" + output + '\'' + '}'; } }
TaskRepository.java
package pas.au.pivotal.pa.sct.demo; import org.springframework.data.jpa.repository.JpaRepository; public interface TaskRepository extends JpaRepository <TaskRun, Long> { }
JpaTaskConfigurer.java
package pas.au.pivotal.pa.sct.demo.configuration; import java.text.SimpleDateFormat; import java.util.Date; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import pas.au.pivotal.pa.sct.demo.TaskRunOutput; import pas.au.pivotal.pa.sct.demo.TaskRunRepository; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.cloud.task.configuration.DefaultTaskConfigurer; import org.springframework.cloud.task.listener.annotation.BeforeTask; import org.springframework.cloud.task.repository.TaskExecution; import org.springframework.orm.jpa.JpaTransactionManager; import org.springframework.stereotype.Component; import org.springframework.transaction.PlatformTransactionManager; @Component public class JpaTaskConfigurer extends DefaultTaskConfigurer { private static final Log logger = LogFactory.getLog(JpaTaskConfigurer.class); @Autowired private PlatformTransactionManager transactionManager; @Autowired private TaskRunRepository taskRunRepository; @Override public PlatformTransactionManager getTransactionManager() { if(this.transactionManager == null) { this.transactionManager = new JpaTransactionManager(); } return this.transactionManager; } @BeforeTask public void init(TaskExecution taskExecution) { String execDate = new SimpleDateFormat().format(new Date()); taskRunRepository.save(new TaskRunOutput("Executed at " + execDate)); logger.info("Executed at : " + execDate); } }
4. Now as per the previous blog execute the task and verify it completes without error. The screen shot below shows how the "Tasks" tab shows this
Note: You would need to PUSH the application to Pivotal Cloud Foundry before you can execute it which is shown on the original blog entry
5. Now if you follow this blog entry below you can deploy a Web Based interface for Pivotal MySQL instance to view the table and it's output
http://theblasfrompas.blogspot.com.au/2017/04/accessing-pivotal-mysql-service.html
With Pivotal MySQL*Web installed the output can be viewed as shown below.
Subscribe to:
Posts (Atom)