Thursday, 23 November 2017

Taking Pivotal Cloud Foundry Small Footprint for a test drive

Pivotal Cloud Foundry (PCF) now has a small footprint edition. It features a deployment configuration with as few as 6 VMs. Review the documentation for download and installation instructions as follows

http://docs.pivotal.io/pivotalcf/1-12/customizing/small-footprint.html

There was also a Pivotal blog post on this as follows:

https://content.pivotal.io/blog/big-things-come-in-small-packages-getting-started-with-pivotal-cloud-foundry-small-footprint

As you can see from this image it's considerably smaller control plane that's obvious.



It is important to understand what the limitations of such an install are as per the docs link below.

http://docs.pivotal.io/pivotalcf/1-12/customizing/small-footprint.html#limits

Installing the small footprint looks identical from the Operations Manager UI in fact it's still labelled ERT and from the home page of Operations Manager UI your wouldn't even know you had the small footprint



If you dig a bit further and click on the "ERT tile" and then select "Resource Config" left hand link you will then clearly know it's the Small Footprint PCF install.


I choose to use internal MySQL database and if I didn't then it could be scaled back even more then the default 7 VM's I ended up with.

Lastly I was very curious to find out what jobs are placed on which service VM's. Here is what it looked like for me when I logged into bosh director and run some bosh CLI commands
  
ubuntu@ip-10-0-0-241:~$ bosh2 -e aws vms --column=Instance --column="Process State" --column=AZ --column="VM Type"
Using environment '10.0.16.5' as user 'director' (bosh.*.read, openid, bosh.*.admin, bosh.read, bosh.admin)

Task 73. Done

Deployment 'cf-a96683b17697c86b8c90'

Instance                                             Process State  AZ               VM Type
backup-prepare/16356c40-1f20-42f0-8f2e-de45549be797  running        ap-southeast-2a  t2.micro
blobstore/b6e22107-018b-425d-8fe4-ab47eeaf2c75       running        ap-southeast-2a  m4.large
compute/5439f18f-c842-40a2-b6f3-faf6b6848716         running        ap-southeast-2a  r4.xlarge
control/68979d93-d12b-4d87-b110-d3d41a48b261         running        ap-southeast-2a  r4.xlarge
database/a3efedaa-4df6-48f5-9f20-61cf3d9f3c1b        running        ap-southeast-2a  r4.large
mysql_monitor/d40ef638-d2d0-488e-b937-99a7f5b5b334   running        ap-southeast-2a  t2.micro
router/f9573547-c5a1-43d4-be02-38a1d9e9c73e          running        ap-southeast-2a  t2.micro

7 vms

Succeeded
  
ubuntu@ip-10-0-0-241:~$ bosh2 -e aws instances --ps --column=Instance --column=Process
Using environment '10.0.16.5' as user 'director' (bosh.*.read, openid, bosh.*.admin, bosh.read, bosh.admin)

Task 68. Done

Deployment 'cf-a96683b17697c86b8c90'

Instance                                                          Process
autoscaling-register-broker/9feaef45-994e-472c-8ca3-f0c39467dd6b  -
autoscaling/184df31d-a64c-49e0-8b6b-27eafdb31ca0                  -
backup-prepare/16356c40-1f20-42f0-8f2e-de45549be797               -
~                                                                 service-backup
blobstore/b6e22107-018b-425d-8fe4-ab47eeaf2c75                    -
~                                                                 blobstore_nginx
~                                                                 blobstore_url_signer
~                                                                 consul_agent
~                                                                 metron_agent
~                                                                 route_registrar
bootstrap/0dc22a1f-a1ee-4a03-85c6-fed08f37c44a                    -
compute/5439f18f-c842-40a2-b6f3-faf6b6848716                      -
~                                                                 consul_agent
~                                                                 garden
~                                                                 iptables-logger
~                                                                 metron_agent
~                                                                 netmon
~                                                                 nfsv3driver
~                                                                 rep
~                                                                 route_emitter
~                                                                 silk-daemon
~                                                                 vxlan-policy-agent
control/68979d93-d12b-4d87-b110-d3d41a48b261                      -
~                                                                 adapter
~                                                                 auctioneer
~                                                                 bbs
~                                                                 cc_uploader
~                                                                 cloud_controller_clock
~                                                                 cloud_controller_ng
~                                                                 cloud_controller_worker_1
~                                                                 cloud_controller_worker_local_1
~                                                                 cloud_controller_worker_local_2
~                                                                 consul_agent
~                                                                 doppler
~                                                                 file_server
~                                                                 locket
~                                                                 loggregator_trafficcontroller
~                                                                 metron_agent
~                                                                 nginx_cc
~                                                                 policy-server
~                                                                 reverse_log_proxy
~                                                                 route_registrar
~                                                                 routing-api
~                                                                 scheduler
~                                                                 silk-controller
~                                                                 ssh_proxy
~                                                                 statsd_injector
~                                                                 syslog_drain_binder
~                                                                 tps_watcher
~                                                                 uaa
database/a3efedaa-4df6-48f5-9f20-61cf3d9f3c1b                     -
~                                                                 cluster_health_logger
~                                                                 consul_agent
~                                                                 galera-healthcheck
~                                                                 gra-log-purger-executable
~                                                                 mariadb_ctrl
~                                                                 metron_agent
~                                                                 mysql-diag-agent
~                                                                 mysql-metrics
~                                                                 nats
~                                                                 route_registrar
~                                                                 streaming-mysql-backup-tool
~                                                                 switchboard
mysql-rejoin-unsafe/01a0aec3-b103-4c09-bc69-cabc61c513cc          -
mysql_monitor/d40ef638-d2d0-488e-b937-99a7f5b5b334                -
~                                                                 replication-canary
nfsbrokerpush/829f0292-59ab-4824-8b0b-c4af4bddbce0                -
notifications-ui/50972440-36cd-499d-ad7c-eef4df7e604b             -
notifications/968d625e-af63-4e2f-a59c-f6b789ef1cff                -
push-apps-manager/3d9760d7-6f09-453c-a052-32604b6a3235            -
push-pivotal-account/5a8782ad-82eb-4469-8242-f0873bc4a587         -
push-usage-service/9b781db1-171b-4abe-92f4-7445fd3d487f           -
router/f9573547-c5a1-43d4-be02-38a1d9e9c73e                       -
~                                                                 consul_agent
~                                                                 gorouter
~                                                                 metron_agent
smoke-tests/a8e2ff97-bae1-4594-90fc-ec8c430fd620                  -

77 instances

Succeeded

Now, with Small Footprint, you have yet another way to bring PCF to your organization!

Sunday, 19 November 2017

Using Spring Boot Actuator endpoint for Spring Boot application health check type on PCF

An application health check is a monitoring process that continually checks the status of a running Cloud Foundry application. When deploying an app, a developer can configure the health check type (port, process, or HTTP), a timeout for starting the application, and an endpoint (for HTTP only) for the application health check.

To use the HTTP option your manifest.yml would look like this

---
applications:
- name: pas-cf-manifest
  memory: 756M
  instances: 1
  hostname: pas-cf-manifest
  path: ./target/demo-0.0.1-SNAPSHOT.jar
  health-check-type: http
  health-check-http-endpoint: /health
  stack: cflinuxfs2
  timeout: 80
  env:
    JAVA_OPTS: -Djava.security.egd=file:///dev/urandom
    NAME: Apples

Using a HTTP endpoint such as "/health" is possible once you add the Spring Boot Actuator maven dependency as follows

  
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

More Information

https://docs.run.pivotal.io/devguide/deploy-apps/healthchecks.html

Saturday, 28 October 2017

Testing network connectivity from Cloud Foundry Application Instances

This app below simply tests whether a host:port is accessible from a CF app instance. For example can my application instance access my Oracle Database Instance running outside of PCF given application instances need network access to the database database for example.

You can use bosh2 ssh to get to the Diego Cells if you have access to the environment or even "cf ssh" if that has been enabled.

GitHub URL:

https://github.com/papicella/cloudfoundry-socket-test

Success


pasapicella@pas-macbook:~$ http http://pas-cf-sockettest.cfapps.io/www.google.com.au/80
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 81
Content-Type: application/json;charset=UTF-8
Date: Wed, 25 Oct 2017 08:38:33 GMT
X-Vcap-Request-Id: 8fd05c77-f680-4966-558b-c45e71825fa0

{
    "errorMessage": "N/A",
    "hostname": "www.google.com.au",
    "port": "80",
    "res": "SUCCESS"
}

Failure

pasapicella@pas-macbook:~$ http http://pas-cf-sockettest.cfapps.io/10.0.0.10/8080
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 110
Content-Type: application/json;charset=UTF-8
Date: Wed, 25 Oct 2017 11:52:18 GMT
X-Vcap-Request-Id: 91ad2359-7d95-49c6-4538-548e480b7820

{
    "errorMessage": "Connection refused (Connection refused)",
    "hostname": "10.0.0.10",
    "port": "8080",
    "res": "FAILED"
}

Sunday, 22 October 2017

Just installed Pivotal Cloud Foundry, what's next should I login to Apps Manager?

I get this question often from customers. Pivotal Cloud Foundry has just been installed and the API endpoint to target the instance is working fine. In short we want to do the following before we get developers onto the platform to ensure we no longer using the UAA server admin login details from the CLI or Apps Manager UI.

  • Create a new ADMIN user which will be used to configure Apps Manager ORGS and spaces for the developers
  • Create an ORG
  • Create at least one Quota maybe more to control memory limit and application instances within an ORG
  • Assign the quota to your ORG
Steps

--> Create a new ADMIN user which will be used to configure Apps Manager ORGS and spaces for the developers

1. Login to Ops Manager VM using SSH for example
2. Target the UAA server as shown below

Eg: $ uaac target uaa.YOUR-DOMAIN

ubuntu@opsmanager-pcf:~$ uaac target uaa.system.YYYY --skip-ssl-validation
Unknown key: Max-Age = 2592000

Target: https://uaa.system.YYYY

3. Authenticate and obtain an access token for the admin client from the UAA server

Note: Record the uaa:admin:client_secret from your deployment manifest

ubuntu@opsmanager-pcf:~$ uaac token client get admin -s PASSWD

Successfully fetched token via client credentials grant.
Target: https://uaa.system.YYYY
Context: admin, from client admin

4. Use the uaac contexts command to display the users and applications authorized by the UAA server, and the permissions granted to each user and application. Ensure in the "scope" field that "scim.write" exists

ubuntu@opsmanager-pcf:~$ uaac contexts

[0]*[https://uaa.system.YYYY]
  skip_ssl_validation: true

  [0]*[admin]
      client_id: admin
      access_token: .....
      token_type: bearer
      expires_in: 43199
      scope: clients.read password.write clients.secret clients.write uaa.admin scim.write scim.read
      jti: b1bf094a5c4640dbac4abc5f3bf15b08

5. Run the following command to create an admin user

ubuntu@opsmanager-pcf:~$ uaac user add apples -p PASSWD --emails papicella@pivotal.io
user account successfully added

6. Run uaac member add GROUP NEW-ADMIN-USERNAME to add the new admin to the groups cloud_controller.admin, uaa.admin, scim.read, and scim.write

ubuntu@opsmanager-pcf:~$ uaac member add cloud_controller.admin apples
success
ubuntu@opsmanager-pcf:~$ uaac member add uaa.admin apples
success
ubuntu@opsmanager-pcf:~$ uaac member add scim.read apples
success
ubuntu@opsmanager-pcf:~$ uaac member add scim.write apples
success

--> Create an ORG

1. Login using the new admin user "apples"

pasapicella@pas-macbook:~$ cf login -u apples -p PASSWD -o system -s system
API endpoint: https://api.system.YYYY
Authenticating...
OK

Targeted org system

Targeted space system

API endpoint:   https://api.system.YYYY (API version: 2.94.0)
User:           papicella@pivotal.io
Org:            system
Space:          system

2. Create an ORG as follows

pasapicella@pas-macbook:~$ cf create-org myfirst-org
Creating org myfirst-org as apples...
OK

Assigning role OrgManager to user apples in org myfirst-org ...
OK

TIP: Use 'cf target -o "myfirst-org"' to target new org

--> Create at least one Quota maybe more to control memory limit and application instances within an ORG

1. Here we create what I call a medium-quota which allows 20G of memory, 2 service instances, each application instance can be no more then 1G of memory and only 20 Application Instances can be created using this quota.

pasapicella@pas-macbook:~$ cf create-quota medium-quota -m 20G -i 1G -a 20 -s 2 -r 1000 --allow-paid-service-plans
Creating quota medium-quota as apples...
OK

pasapicella@pas-macbook:~$ cf quota medium-quota
Getting quota medium-quota info as apples...
OK

Total Memory           20G
Instance Memory        1G
Routes                 1000
Services               2
Paid service plans     allowed
App instance limit     20
Reserved Route Ports   0

--> Assign the quota to your ORG

1. Assign the newly created quota to the ORG we created above

pasapicella@pas-macbook:~$ cf set-quota myfirst-org medium-quota
Setting quota medium-quota to org myfirst-org as apples...
OK

pasapicella@pas-macbook:~$ cf org myfirst-org
Getting info for org myfirst-org as apples...

name:                 myfirst-org
domains:              apps.pas-apples.online
quota:                medium-quota
spaces:
isolation segments:

Finally we can add a space to the ORG and assign privileges to a user called "pas" as shown below

- Set OrgManager role to the user "pas"

pasapicella@pas-macbook:~$ cf set-org-role pas myfirst-org OrgManager
Assigning role OrgManager to user pas in org myfirst-org as apples...
OK

- Logout as "apples" admin user as "pas" can now do his own admin for the ORG " myfirst-org"

pasapicella@pas-macbook:~$ cf logout
Logging out...
OK

- Login as pas and target the ORG

pasapicella@pas-macbook:~$ cf login -u pas -p PASSWD -o myfirst-org
API endpoint: https://api.system.YYYY
Authenticating...
OK

Targeted org myfirst-org

API endpoint:   https://api.system.YYYY (API version: 2.94.0)
User:           pas
Org:            myfirst-org
Space:          No space targeted, use 'cf target -s SPACE'

- Create a space which will set space roles for the user "pas"

pasapicella@pas-macbook:~$ cf create-space dev
Creating space dev in org myfirst-org as pas...
OK
Assigning role RoleSpaceManager to user pas in org myfirst-org / space dev as pas...
OK
Assigning role RoleSpaceDeveloper to user pas in org myfirst-org / space dev as pas...
OK

TIP: Use 'cf target -o "myfirst-org" -s "dev"' to target new space

- Target the new space

pasapicella@pas-macbook:~$ cf target -o myfirst-org -s dev
api endpoint:   https://api.system.pas-apples.online
api version:    2.94.0
user:           pas
org:            myfirst-org
space:          dev

Typically we would assign other users to the spaces using "cf set-space-role .."

pasapicella@pas-macbook:~$ cf set-space-role --help
NAME:
   set-space-role - Assign a space role to a user

USAGE:
   cf set-space-role USERNAME ORG SPACE ROLE

ROLES:
   'SpaceManager' - Invite and manage users, and enable features for a given space
   'SpaceDeveloper' - Create and manage apps and services, and see logs and reports
   'SpaceAuditor' - View logs, reports, and settings on this space

SEE ALSO:
   space-users

More Information

Creating and Managing Users with the UAA CLI (UAAC)
https://docs.pivotal.io/pivotalcf/1-12/uaa/uaa-user-management.html

Creating and Managing Users with the cf CLI
https://docs.pivotal.io/pivotalcf/1-12/adminguide/cli-user-management.html

Thursday, 5 October 2017

Pivotal Cloud Foundry 1.12 on Google Cloud Platform with VM labels

Once PCF is installed on GCP it's worth noting that viewing the "Compute Engine" labels gives you as indication of what VM each CF service is associated with. The screen shots below show's this.



Monday, 25 September 2017

Updating Cloud Foundry CLI using Brew

Need to upgrade the CF CLI using brew it's as simple as below. Go to love brew

pasapicella@pas-macbook:~$ brew upgrade cf-cli
==> Upgrading 1 outdated package, with result:
cloudfoundry/tap/cf-cli 6.31.0
==> Upgrading cloudfoundry/tap/cf-cli
Warning: Use cloudfoundry/tap/cloudfoundry-cli instead of deprecated pivotal/tap/cloudfoundry-cli
==> Downloading https://cli.run.pivotal.io/stable?release=macosx64-binary&version=6.31.0&source=homebrew
==> Downloading from https://s3-us-west-1.amazonaws.com/cf-cli-releases/releases/v6.31.0/cf-cli_6.31.0_osx.tgz
######################################################################## 100.0%
==> Caveats
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d
==> Summary
🍺  /usr/local/Cellar/cf-cli/6.31.0: 6 files, 17.6MB, built in 16 seconds

pasapicella@pas-macbook:~$ cf --version
cf version 6.31.0+b35df905d.2017-09-15

Friday, 15 September 2017

Using Cloud Foundry CUPS to inject Spring Security credentials into a Spring Boot Application

The following demo shows how to inject the Spring Security username/password credentials from a User Provided service on PCF, hence using the VCAP_SERVICES env variable to inject the values required to protect the application using HTTP Basic Authentication while running in PCF. Spring Boot automatically converts this data into a flat set of properties so you can easily get to the data as shown below.

The demo application can be found as follows

https://github.com/papicella/springsecurity-cf-cups

The application.yml would access the VCAP_SERVICES CF env variable using the the Spring Boot flat set of properties as shown below.

eg:

VCAP_SERVICES

System-Provided:
{
 "VCAP_SERVICES": {
  "user-provided": [
   {
    "credentials": {
     "password": "myadminpassword",
     "username": "myadminuser"
    },
    "label": "user-provided",
    "name": "my-cfcups-service",
    "syslog_drain_url": "",
    "tags": [],
    "volume_mounts": []
   }
  ]
 }
}
...

application.yml

spring:
  application:
    name: security-cf-cups-demo
security:
  user:
    name: ${vcap.services.my-cfcups-service.credentials.username:admin}
    password: ${vcap.services.my-cfcups-service.credentials.password:password}

Thursday, 14 September 2017

Oracle 12c Service Broker for Pivotal Cloud Foundry

The following example is a PCF 2.0 Service Broker written as a Spring Boot application. This is just an example and should be evolved to match a production type setup in terms oracle requirements. This service broker simple creates USERS and assigns then 20M of quota against a known TABLESPACE

It's all documented as follows

https://github.com/papicella/oracle-service-broker




Monday, 4 September 2017

Introducing Pivotal MySQL*Web, Pivotal’s New Open Source Web-Based Administration UI for MySQL for Pivotal Cloud Foundry

Recently Pivotal announced "Pivotal MySQL*Web" on it's main blog page. You can read more about it here which was an Open Source project I created a while ago for Pivotal MySQL instances on Pivotal Cloud Foundry

https://content.pivotal.io/blog/introducing-pivotal-mysql-web-pivotal-s-new-open-source-web-based-administration-ui-for-mysql-for-pivotal-cloud-foundry


Couchbase Service Broker for Pivotal Cloud Foundry

The following example is a PCF 2.0 Service Broker written as a Spring Boot application for Couchbase 4.6.x. This is just an example and should be evolved to match a production type setup in terms of bucket creation and access for created service instances.

It's all documented as follows

https://github.com/papicella/couchbase-service-broker




More Information

https://docs.pivotal.io/tiledev/service-brokers.html