Feed aggregator

Oracle API Platform Cloud Service: using the Developer Portal for discovering APIs via the API Catalog and subscribing applications to APIs

Amis Blog - Sun, 2018-04-22 14:22

At the Oracle Partner PaaS Summer Camps VII 2017 in Lisbon last year, at the end of august, I attended the API Platform Cloud Service & Integration Cloud Service bootcamp.

In a series of article’s I will give a high level overview of what you can do with Oracle API Platform Cloud Service.

At the Summer Camp a pre-built Oracle VM VirtualBox APIPCS appliance (APIPCS_17_3_3.ova) was provided to us, to be used in VirtualBox. Everything needed to run a complete demo of API Platform Cloud Service is contained within Docker containers that are staged in that appliance. The version of Oracle API Platform CS, used within the appliance, is Release 17.3.3 — August 2017.

See https://docs.oracle.com/en/cloud/paas/api-platform-cloud/whats-new/index.html to learn about the new and changed features of Oracle API Platform CS in the latest release.

In this article in the series about Oracle API Platform CS, the focus will be on the Developer Portal, discovering APIs via the API Catalog and subscribing applications to APIs. As a follow-up from my previous article, at the end the focus is on validating the “Key Validation” policy of the “HumanResourceService”API.
[https://technology.amis.nl/2018/04/14/oracle-api-platform-cloud-service-using-the-management-portal-and-creating-an-api-including-some-policies/]

Be aware that the screenshot’s in this article and the examples provided, are based on a demo environment of Oracle API Platform CS and were created by using the Oracle VM VirtualBox APIPCS appliance mentioned above.

This article only covers part of the functionality of Oracle API Platform CS. For more detail I refer you to the documentation: https://cloud.oracle.com/en_US/api-platform.

Short overview of Oracle API Platform Cloud Service

Oracle API Platform Cloud Service enables companies to thrive in the digital economy by comprehensively managing the full API lifecycle from design and standardization to documenting, publishing, testing and managing APIs. These tools provide API developers, managers, and users an end-to-end platform for designing, prototyping. Through the platform, users gain the agility needed to support changing business demands and opportunities, while having clear visibility into who is using APIs for better control, security and monetization of digital assets.
[https://cloud.oracle.com/en_US/api-platform/datasheets]

Architecture

Management Portal:
APIs are managed, secured, and published using the Management Portal.
The Management Portal is hosted on the Oracle Cloud, managed by Oracle, and users granted
API Manager privileges have access.

Gateways:
API Gateways are the runtime components that enforce all policies, but also help in
collecting data for analytics. The gateways can be deployed anywhere – on premise, on Oracle
Cloud or to any third party cloud providers.

Developer Portal:
After an API is published, Application Developers use the Developer Portal to discover, register, and consume APIs. The Developer Portal can be customized to run either on the Oracle Cloud or directly in the customer environment on premises.
[https://cloud.oracle.com/opc/paas/datasheets/APIPCSDataSheet_Jan2018.pdf]

Oracle Apiary:
In my article “Oracle API Platform Cloud Service: Design-First approach and using Oracle Apiary”, I talked about using Oracle Apiary and interacting with its Mock Server for the “HumanResourceService” API, I created earlier.

The Mock Server for the “HumanResourceService” API is listening at:
http://private-b4874b1-humanresourceservice.apiary-mock.com
[https://technology.amis.nl/2018/01/31/oracle-api-platform-cloud-service-design-first-approach-using-oracle-apiary/]

Roles

Within Oracle API Platform CS roles are used.

Roles determine which interfaces a user is authorized to access and the grants they are eligible to receive.
[https://docs.oracle.com/en/cloud/paas/api-platform-cloud/apfad/api-platform-cloud-service-roles-resources-actions-and-grants.html]

  • Administrator
    System Administrators responsible for managing the platform settings. Administrators possess the rights of all other roles and are eligible to receive grants for all objects in the system.
  • API Manager
    People responsible for managing the API lifecycle, which includes designing, implementing, and versioning APIs. Also responsible for managing grants and applications, providing API documentation, and monitoring API performance.
  • Application Developer
    API consumers granted self-service access rights to discover and register APIs, view API documentation, and manage applications using the Developer Portal.
  • Gateway Manager
    Operations team members responsible for deploying, registering, and managing gateways. May also manage API deployments to their gateways when issued the Deploy API grant by an API Manager.
  • Gateway Runtime
    This role indicates a service account used to communicate from the gateway to the portal. This role is used exclusively for gateway nodes to communicate with the management service; users assigned this role can’t sign into the Management Portal or the Developer Portal.
  • Service Manager
    People responsible for managing resources that define backend services. This includes managing service accounts and services.
  • Plan Manager
    People responsible for managing plans.

Within the Oracle VM VirtualBox APIPCS appliance the following users (all with password welcome1) are present and used by me in this article:

User Role api-manager-user APIManager api-gateway-user GatewayManager app-dev-user ApplicationDeveloper

Publish an API, via the Management Portal (api-manager-user)

Start the Oracle API Platform Cloud – Management Portal as user api-manager-user.

Navigate to tab “Publication” of the “HumanResourceService” API (which I created earlier).
[https://technology.amis.nl/2018/04/14/oracle-api-platform-cloud-service-using-the-management-portal-and-creating-an-api-including-some-policies/]

Publish an API to the Developer Portal when you want application developers to discover and consume it.

Each published API has a details page on the Developer Portal. This page displays basic information about the API, an overview describing the purpose of the API, and documentation for using the API. This page is not visible on the Developer Portal until you publish it.
[https://docs.oracle.com/en/cloud/paas/api-platform-cloud/apfad/publishing-apis.html#GUID-145F0AAE-872B-4577-ACA6-994616A779F1]

The tab “Publication” contains the following parts:

  • API Portal URL
  • Developer Portal API Overview
  • Documentation

Next I will explain (in reversed order) these parts in more detail.

As you can see, for some of the parts we can use HTML, Markdown or Apiary.

Remark:
Markdown is a lightweight markup language with plain text formatting syntax.
[https://en.wikipedia.org/wiki/Markdown]

Part “Documentation” of the tab “Publication”

You can provide HTML or Markdown documentation by uploading a file, manually entering text, or providing a URL to the documentation resource. After you have added the documentation, it appears on the Documentation tab of the API detail page in the Developer Portal.
[https://docs.oracle.com/en/cloud/paas/api-platform-cloud/apfad/publishing-apis.html#GUID-9FD22DC2-18A9-4338-91E7-70726C906B91]

It is also possible to add documentation from Oracle Apiary to an API.

Adding documentation to the API can help users understand its purpose and how it was configured.

Note:
Swagger or API Blueprint documentation can only be added to an Oracle Apiary Pro account. To add documentation, the team must have ownership of the API in Oracle Apiary. API definitions owned by personal accounts cannot be used. To transfer ownership of an API from a personal account to a team account, see the Oracle Apiary documentation.
[https://docs.oracle.com/en/cloud/paas/api-platform-cloud/apfad/publishing-apis.html#GUID-A7E68AA0-396D-400C-933C-1C4CD3DDD832]

So let’s see how I tried using documentation from Oracle Apiary.

Using Oracle Apiary for documentation

I clicked on button “Apiary”. In my case the following screen appeared:

Next, I clicked on button “Go To Apiary”.

Then I clicked on button “Sign in”.

After a successful sign in (for example by using Email/Password), the following screen appeared (with the “HumanResourceService” API visible):

Next, I clicked on button “Create a team”. The following pop-up appeared:

Because I use a Free (personal) Account for Oracle Apiary, I am not able to create a team.

Remember the note (see above) saying: “Swagger or API Blueprint documentation can only be added to an Oracle Apiary Pro account. To add documentation, the team must have ownership of the API in Oracle Apiary. API definitions owned by personal accounts cannot be used.”.
[https://docs.oracle.com/en/cloud/paas/api-platform-cloud/apfad/publishing-apis.html#GUID-A7E68AA0-396D-400C-933C-1C4CD3DDD832]

So, for me, the path of using documentation from Oracle Apiary came to an end.

As an alternative, in this article, I used Markdown for documentation. But before explaining that in more detail, I want to give you an impression of screenshot’s of what happens when you click on button “Apiary” and have an Apiary account with the right privileges to add documentation to an API.

Remark:
The screenshot’s that follow, are taken from the API Platform Cloud Service bootcamp, I attended at the Oracle Partner PaaS Summer Camps VII 2017 in Lisbon last year.

So, when you click on button “Apiary”, the following screen appears:

A list of API’s is visible where you can choose one to connect. For example: “TicketService27” API.

After a click on button “Connect”, the “Documentation” part of tab “Publication” looks like:

Using Markdown for documentation

For reasons mentioned above, as an alternative for using Oracle Apiary, in this article, I used Markdown for documentation. Markdown is new to me, so (in this article) I will only demonstrate it with a simplified version of the documentation (available in Apiary).

Click on button “Markdown”.

Next, click on tab “Text” and enter the following text:

# HumanResourceService

Human Resource Service is an API to manage Human Resources.

## Employees Collection [/employees]

### Get all employees [GET /employees]

Get all employees.

### Get an employee [GET /employees/{id}]

Get a particular employee by providing an identifier.

### Create an employee [POST /employees]

Create an employee, by using post with the complete payload.

### Update an employee [PUT /employees/{id}]

Update an employee, by using put with the a payload containing: last_name, job_id, salary and department_id.

## Departments Collection [/departments]

### Get a department [GET /department/{id}]

Get a particular department by providing an identifier.

### Get a department and employee [GET /departments/{department_id}/employees/{employee_id}]

Get a particular department by providing a department identifier and a particular employee within that department by providing an employee identifier.

After a click on button “OK”, the “Documentation” part of tab “Publication” looks like:

In the pop-up, click on button “Save Changes”.

Part “Developer Portal API Overview” of the tab “Publication”

You can provide overview text for an API, describing its features and other information a developer should know about its use, in either HTML or Markdown.

You can upload a file, enter text manually, or provide a link to HTML or Markdown to use as overview text. This text appears on the API’s detail page in the Developer Portal.
[https://docs.oracle.com/en/cloud/paas/api-platform-cloud/apfad/publishing-apis.html#GUID-D1BF7E3E-03C9-42AE-9808-EC9BC77D3E61]

For the “Developer Portal API Overview” part, I chose to use HTML (because in this article, up to now, examples of using Markdown and Apiary were already provided).

Once again I will only demonstrate it with a simplified version of the documentation (available in Apiary).

Click on button “HTML”.

Next, click on tab “Text” and enter the following text:

<h1>HumanResourceService</h1>

Human Resource Service is an API to manage Human Resources.

It provides CRUD methods for the resources <b>Employees</b> and <b>Departments</b>.

After a click on button “OK”, the “Developer Portal API Overview” part of tab “Publication” looks like:

In the pop-up, click on button “Save Changes”.

Part “API Portal URL” of the tab “Publication”

Before publishing to the Developer Portal, each API has to be configured with its own unique Vanity Name. A vanity name is the URI path of an API’s details page when it is published to the Developer Portal.

On the Publication tab, enter the path at which this API will be discoverable in the Developer Portal in the API Portal URL field. This is also called the API’s vanity name.

Note:
An API’s vanity name must be unique, regardless of case. You can’t have APIs with vanity names of Creditcheck and creditcheck. You must enter the vanity name exactly (matching case) in the URL to navigate to an API’s details page in the Developer Portal. For example, navigating to https://<host>:<port>/developers/apis/Creditcheck opens the page for an API with a vanity name of Creditcheck; https://<host>:<port>/developers/apis/creditcheck doesn’t open this page and returns a 404 because the segment in the URL does not match the vanity name exactly.

Only valid URI simple path names are supported. Characters such as “?”, “/”, and “&” are not supported in vanity names. Test_2 is a supported vanity name, but Test/2 is not.
[https://docs.oracle.com/en/cloud/paas/api-platform-cloud/apfad/publishing-apis.html#GUID-C9034B10-72EA-4046-A8B8-B5B1AE087180]

The default API’s vanity name, is derived from the API name:

<not published>/HumanResourceService

Publish the “HumanResourceService” API to the Developer Portal

So now that we have all the documentation in place, Notice that the button “Preview” appeared.

Clicking on button “Preview” generates an error:

Remember that I am using a demo environment of Oracle API Platform CS by using the Oracle VM VirtualBox APIPCS appliance. This seems to be a bug in that environment. So what should have been visible was something like:

Here you can see on the left, that the tab “Overview” is selected . There is also a tab “Documentation”.

Remark:
Please see the screenshot’s later on in this article for the “HumanResourceService”API in the “Developer Portal” (tab APIs), with regard to the tabs “Overview” and “Documentation”. These show the same information as in the Preview context.

Next, click on button “Publish to Portal”.

Notice that the > icon “Launch Developer Portal in another browser window” appeared and also that the API Portal URL is changed to:

http://apics.oracle.com:7201/developers/apis/HumanResourceService

In the top part of the screen you can see that the “HumanResourceService”API is “Published’.

It’s time to launch the Developer Portal.

Click on the icon “Launch Developer Portal in another browser window”.

Sign in to the Oracle API Platform Cloud – Developer Portal as user app-dev-user

After a successful sign in as user app-dev-user, the next screen appears (with tab “APIs” selected):

The “Developer Portal” is the web page where you discover APIs, subscribe to APIs and get the necessary information to invoke them. When you access the “Developer Portal”, the API Catalog page appears. All the APIs that have been published to the “Developer Portal” are listed. Use the API Catalog page to find APIs published to the “Developer Portal”.

In the “Developer Portal” screen above there are no APIs, or they are not visible for the current user. So we have to go back to the Oracle API Platform Cloud – Management Portal (as an API Manager). There we can grant the privileges needed for an Application Developer to see the API. How you do this is described later on in this article.

For now we continue as if the correct privileges were already in place. Therefor the “HumanResourceService” API is visible.

Click on the “HumanResourceService” API.

Here you can see on the left, that the tab “Overview” is selected.

For now I will give you a short overview of screenshot’s of each of the tabs on the left.

Tab “Overview” of the “HumanResourceService” API

Remember that we used HTML code for the “Developer Portal API Overview” part of the tab “Publication”?
So here you can see the result.

Tab “Documentation” of the “HumanResourceService” API

Remember that we used Markdown code for the “Documentation” part of the tab “Publication”?
So here you can see the result.

Remark:
If I had an Apiary account with the right privileges to add documentation to an API and used Apiary for documentation, the tab “Documentation” would have looked like:

Discover APIs

In the API Catalog page, you can search for an API by entering keywords in the field at the top of the catalog. The list is narrowed to the APIs that have that word in the name or the description. If you enter multiple words, the list contains all APIs with either of the words; APIs with both words appear at the top of the list. If a keyword or keywords have been applied to the list, they appear in a bar at the top of the page. Filters can also be applied to the list. You can also sort the list for example in alphabetical order or by newest to oldest API.
[Oracle Partner PaaS Summer Camps VII 2017, APIPCS bootcamp, Lab_APIPCS_Design_and_Implement.pdf]

Subscribe an application to the “HumanResourceService” API

In the “Developer Portal” screen if we navigate, in the API Catalog page, to the “HumanResourceService” API, and if the user has the correct privileges, a button “Subscribe” is visible. In the Oracle API Platform Cloud – Management Portal (as an API Manager) we can grant the privileges needed for an Application Developer to register an application to the API. How you do this is described later on in this article.

For now we continue as if the correct privileges were already in place.

Click on button “Subscribe”.

Next, click on button “Create New Application”. Enter the following values:

Application Name HumanResourceWebApplication Description Web Application to manage Human Resources. Application Types Web Application Contact information: First Name FirstName Last Name LastName Email Email@company.com Phone 123456789 Company Company

Click on button “Save”.

For a short while a pop-up “The application ‘HumanResourceWebApplication’ was created.” appears.

So now we have an application, we can subscribe it, to the “HumanResourceService” API.

Notice that an Application Key was generated, with as value:

fb3138d1-0636-456e-96c4-4e21b684f45e

Remark:
You can reissue a key for an application in case it has been compromised, Application keys are established at the application level. If you reissue an application’s key, the old key is invalidated. This affects all APIs (that have the key validation policy applied) to which an application is registered. Every request to these APIs must use the new key to succeed. Requests using the old key are rejected. APIs without the key validation policy are not affected as these do not require a valid application key to pass requests.
[https://docs.oracle.com/en/cloud/paas/api-platform-cloud/apfad/reissuing-application-key.html#GUID-4E570C15-C289-4B6D-870C-F7ADACC1F6DD]

Next, click on button “Subscribe API”.

For a short while a pop-up “API ‘HumanResourceService’ was subscribed to application ‘HumanResourceWebApplication’.” appears.

A request to register the application to the API is sent to the API Manager. So now we have to wait for the approval of the API Manager. How you do this is described later on in this article.

In the API Catalog page, when viewing an API you can see which applications are subscribed to it.

In the My Applications page, when viewing an application you can see which APIs it subscribed to.

After a click on the “HumanResourceWebApplication” application, the next screen appears (with tab “Overview” selected):

First l will give you a short overview of screenshot’s of each of the tabs on the left. Some of these I will explain in more detail as I will walk you through some of the functionality of Oracle API Platform CS.

Tab “Overview” of the “HumanResourceWebApplication” application

Tab “Subscribed APIs” of the “HumanResourceWebApplication” application

Tab “Grants” of the “HumanResourceWebApplication” application

Application grants are issued per application.

The following tabs are visible and can be chosen:

  • Manage Application
    People issued this grant can view, modify and delete this application. API Manager users issued this grant can also issue grants for this application to others.
  • View all details
    People issued this grant can see all details about this application in the Developer Portal.

See for more information: https://docs.oracle.com/en/cloud/paas/api-platform-cloud/apfad/managing-application-grants.html

Tab “Analytics” of the “HumanResourceWebApplication” application

Create an Application in the “My Applications” page

Click on button “New Application”.

In the same way as described before I created several applications (one at a time) with minimum information (Application Name, Application Types, First Name, Last Name and Email).

In the My Applications page, the list of applications then looks like:

In the table below I summarized the applications that I created:

Application Name Application Types Application Key DesktopApp_A_Application Desktop App e194833d-d5ac-4c9d-8143-4cf3a3e81fea DesktopApp_B_Application Desktop App fd06c3b5-ab76-4e89-8c5a-e4b8326c360b HumanResourceWebApplication Web Application fb3138d1-0636-456e-96c4-4e21b684f45e MobileAndroid_A_Application Mobile – Android fa2ed56f-da3f-49ea-8044-b16d9ca75087 MobileAndroid_B_Application Mobile – Android 385871a2-7bb8-4281-9a54-c0319029e691 Mobile_iOS_A_Application Mobile – iOS 7ebb4cf8-5a3f-4df5-82ad-fe09850f0e50

In the API Catalog page, navigate to the “HumanResourceService” API. Here you can see that there is already one subscribed application.

Click on button “Subscribe”.

Next, select the “MobileAndroid_B_Application” application.

For a short while a pop-up “API ‘HumanResourceService’ was subscribed to application ‘ MobileAndroid_B_Application ‘.” appears.

In the API Catalog page, when viewing an API you can see which applications are subscribed to it.

Here we can see the status “Pending”. A request to register the “MobileAndroid_B_Application” application to the “HumanResourceService” API is sent to the API Manager. So now we have to wait for the approval of the API Manager. Repeat the steps described in this article, to approve the request, by switching to an API Manager.

In the screen below, we can see the end result:

Edit the Key Validation Policy, via the Management Portal (api-manager-user)

In the top right of the Oracle API Platform Cloud – Management Portal sign in as user api-manager-user.

Navigate to tab “API Implementation” of the “HumanResourceService” API.

Hoover over the “Key Validation” policy and then, on the right, click on icon “Edit policy details”.

Click on button “Apply”.

Next, click on button “Save Changes”.

I applied this policy as an active policy, represented as a solid line around the policy.

Redeploy the API, via the Management Portal (api-manager-user)

Navigate to tab “Deployments” of the “HumanResourceService” API, and then hover over the “Production Gateway” gateway and then, on the right, hover over the icon “Redeploy”.

Next, click on icon “Latest Iteration”. Also approve the request, by switching to a Gateway Manager.
How you do this, is described in my previous article “Oracle API Platform Cloud Service: using the Management Portal and creating an API (including some policies)”.
[https://technology.amis.nl/2018/04/14/oracle-api-platform-cloud-service-using-the-management-portal-and-creating-an-api-including-some-policies/]

So now the “HumanResourceService” API is redeployed on the “Production Gateway” gateway (Node 1).

It is time to invoke the API.

Validating the “Key Validation” policy, via Postman

As described in my previous article, in Postman, I created requests within the collection named “HumanResourceServiceCollection”.
[https://technology.amis.nl/2018/04/14/oracle-api-platform-cloud-service-using-the-management-portal-and-creating-an-api-including-some-policies/]

Then again I invoked two request, to validate them against the “Key Validation” policy.

Invoke method “GetEmployee” of the “HumanResourceService” API

From Postman I invoked the request named “GetEmployeeRequest” (with method “GET” and URL “http://apics.oracle.com:8001/HumanResourceService/1/employees/100”) and a response with “Status 401 Unauthorized” is shown:

After providing the Value fb3138d1-0636-456e-96c4-4e21b684f45e (being the Application Key of the “HumanResourceWebApplication” application) for the Header Key “application-key”, a response with “Status 200 OK” is shown:

After providing the Value e194833d-d5ac-4c9d-8143-4cf3a3e81fea (being the Application Key of the “DesktopApp_A_Application” application) for the Header Key “application-key”, a response with “Status 401 Unauthorized” is shown:

Invoke method “GetDepartmentEmployee” of the “HumanResourceService” API

From Postman I invoked the request named “GetDepartmentEmployeeRequest” (with method “GET” and URL “http://apics.oracle.com:8001/HumanResourceService/1/departments/30/employees/119”) and a response with “Status 401 Unauthorized” is shown:

After providing the Value 385871a2-7bb8-4281-9a54-c0319029e691 (being the Application Key of the “MobileAndroid_B_Application” application) for the Header Key “application-key”, a response with “Status 200 OK” is shown:

Tab “Analytics” of the “Production Gateway” gateway

In the top right of the Oracle API Platform Cloud – Management Portal sign in as user api-gateway-user and click on the “Production Gateway” gateway and navigate to the tab “Analytics”.

In this tab the requests I sent, are visible at “Total Requests”.

If we look, for example, at “Requests By Resource”, the requests are also visible.

Next, click on icon “Applications (4 Active)” and if we look, for example, at “Active Applications”, we can see that there were in total 3 request rejections (because of policy “Key Validation”).

If we look, for example, at “Requests By API”, the requests are also visible.

There were 2 request that had no Header Key “application-key” at all. As you can see in the graph above, these were rejected and were administrated under “Unknown Application (No Key).

There was 1 request that had a Value e194833d-d5ac-4c9d-8143-4cf3a3e81fea for the Header Key “application-key”. As you can see in the graph above, this request was rejected and was administrated under the “DesktopApp_A_Application” application. Remember that this application was not registered to the “HumanResourceService” API.

The other 2 request were accepted, because they had a valid Value for the Header Key and the corresponding applications were registered to the “HumanResourceService” API.

So the “Key Validation” policy is working correct.

Sign in to the Oracle API Platform Cloud – Management Portal as user api-manager-user

Go back to the Oracle API Platform Cloud – Management Portal and, if not already done, sign in as user api-manager-user. Navigate to tab “Grants” of the “HumanResourceService” API.

API grants are issued per API.

The following tabs are visible and can be chosen:

  • Manage API
    Users issued this grant are allowed to modify the definition of and issue grants for this API.
  • View all details
    Users issued this grant are allowed to view all information about this API in the Management Portal.
  • Deploy API
    Users issued this grant are allowed to deploy or undeploy this API to a gateway for which they have deploy rights. This allows users to deploy this API without first receiving a request from an API Manager.
  • View public details
    Users issued this grant are allowed to view the publicly available details of this API on the Developer Portal.
  • Register
    Users issued this grant are allowed to register applications for this plan.
  • Request registration
    Users issued this grant are allowed to request to register applications for this plan.

Users and groups issued grants for a specific API have the privileges to perform the associated actions on that API. See for more information: https://docs.oracle.com/en/cloud/paas/api-platform-cloud/apfad/managing-api-grants.html.

“View public details” grant

To view an API, the Application Developer must have the “View public details” grant or another grant that implies these privileges.

Click on tab “View public details”.

Next, click on button “Add Grantee”.

Select “app-dev-user” and click on button “Add”.

So now, the user app-dev-user (with Role ApplicationDeveloper) is granted the “View public details” privilege.

Remark:
In practice you would probably grant to a group instead of to a single user.

“Request registration” grant

To register an API, the Application Developer must have the “Request registration” grant or another grant that implies these privileges.

Click on tab “Request registration”.

Next, click on button “Add Grantee”.

Select “app-dev-user” and click on button “Add”.

So now, the user app-dev-user (with Role ApplicationDeveloper) is granted the “Request registration” privilege.

Remark:
In practice you would probably grant to a group instead of to a single user.

Be aware that you could also grant the “Register” privilege, so approval of the API Manager (for registering an application to an API) is not needed anymore in that case. This makes sense if it concerns a development environment, for example. Since the Oracle VM VirtualBox APIPCS appliance is using a “Production Gateway” gateway, in this article, I chose for the request and approve mechanism.

Approve a request for registering an application to an API, via the Management Portal (api-manager-user)

On the left, click on tab “Registrations” and then click on tab “Requesting”.

Hover over the “HumanResourceWebApplication” application, then click on button “Approve”.

In the pop-up, click on button “Yes”.

Then you can see on the tab “Registered”, that the registration is done.

After a click on the top right icon “Expand”, more details are shown:

So now the “HumanResourceWebApplication” application is registered to the “HumanResourceService” API.

Summary

As a follow up from my previous articles about Oracle API Platform Cloud Service, in this article the focus is on using the Developer Portal, discovering APIs via the API Catalog and subscribing applications to APIs.

I activated the Key Validation (Security) policy, which I created in my previous article, and redeployed the API to a gateway and validated that this policy worked correct, using requests which I created in Postman.
[https://technology.amis.nl/2018/04/14/oracle-api-platform-cloud-service-using-the-management-portal-and-creating-an-api-including-some-policies/]

While using the Management Portal and Developer Portal in this article, I focused on the roles “API Manager” and “Application Developer”. For example, the user api-manager-user had to approve a request from the app-dev-user to register an application to an API.

At the API Platform Cloud Service bootcamp (at the Oracle Partner PaaS Summer Camps VII 2017 in Lisbon last year, at the end of august), I (and many others) got hands-on experience with the API Platform Cloud Service. There we created an API with more policies than described in this article.

It became obvious that the API Platform Cloud Service is a great API Management solution and that with the help of policies your are able to secure, throttle, route, manipulate, or log requests before they reach the backend service.

The post Oracle API Platform Cloud Service: using the Developer Portal for discovering APIs via the API Catalog and subscribing applications to APIs appeared first on AMIS Oracle and Java Blog.

Oracle VM Server: How to add ISO images to your repository

Dietrich Schroff - Sun, 2018-04-22 09:57
After  setting up
you have to upload some ISO images to your repository, so that your virtual machines can boot with these ISO images.

So inside Oracle VM Manager go to the "Repositories" tab and use the "import iso icon" (the blue arrow):

The only way to import ISO images is to give a URL to Oracle VM Manager:


And after pushing the ok button you can monitor the download inside the job summary tab:



Inside the repository tab you will find the ISO image after a successful download:

This ISO image can now be used inside the configuration of your VMs. (Take a look here)


Domain Indexes -- 3 : CTXCAT Index

Hemant K Chitale - Sat, 2018-04-21 11:14
In previous posts in December 2017, I had demonstrated a CONTEXT Index.

A CONTEXT Index is used for full-text retrieval from large pieces of text (or document formats stored in LOBs)

A CTXCAT Index is best suited for small fragments of text that are to be indexed with other relational data.

Before I begin with the CTXCAT index, in addition to the CTXAPP role (that I had granted during the earlier demonstration), the account also needs the CREATE TRIGGER privilege.

SQL> grant ctxapp to ctxuser;

Grant succeeded.

SQL> grant create trigger to ctxuser;

Grant succeeded.

SQL>


I can now proceed with the CTXUSER demonstration.

SQL> connect ctxuser/ctxuser
Connected.
SQL> create table books
2 (book_id integer primary key,
3 book_title varchar2(250) not null,
4 book_author varchar2(80),
5 book_subject varchar2(25),
6 shelf_id integer)
7 /

Table created.

SQL>
SQL> insert into books values
2 (1,'A Study In Scarlet','Arthur Conan Doyle','Mystery',1);

1 row created.

SQL> insert into books values
2 (2,'The Sign Of Four','Arthur Conan Doyle','Mystery',1);

1 row created.

SQL> insert into books values
2 (3,'Murder On The Orient Express','Agatha Christie','Mystery',1);

1 row created.

SQL> insert into books values
2 (4,'A Brief History of Time','Stephen Hawking','Science - Physics',2);

1 row created.

SQL>
SQL> insert into books values
2 (5,'2001: A Space Odyssey','Arthur C Clarke','Science Fiction',3);

1 row created.

SQL>
SQL> commit;

Commit complete.

SQL>


Next, I specify what is called an Index Set -- which specifies the structured columns that are to be included in the CTXCAT Index.  I then define the CTXCAT Index on the BOOK_TITLE column.

SQL> begin
2 ctx_ddl.create_index_set('books_set');
3 ctx_ddl.add_index('books_set','book_subject');
4 ctx_ddl.add_index('books_set','shelf_id');
5 end;
6 /

PL/SQL procedure successfully completed.

SQL>
SQL> create index books_title_index
2 on books (book_title)
3 indextype is ctxsys.ctxcat
4 parameters ('index set books_set')
5 /

Index created.

SQL>


Now, I can use the Index to query the table, using the CATSEARCH clause instead of the CONTAINS clause. My query includes both BOOK_TITLE and SHELF_ID

SQL> select book_title,book_author,book_subject,shelf_id
2 from books
3 where catsearch (book_title,'History','shelf_id=1') > 0
4 /

no rows selected

SQL> select book_title,book_author,book_subject,shelf_id
2 from books
3 where catsearch (book_title,'History','shelf_id>1') > 0
4 /

BOOK_TITLE
--------------------------------------------------------------------------------
BOOK_AUTHOR
--------------------------------------------------------------------------------
BOOK_SUBJECT SHELF_ID
------------------------- ----------
A Brief History of Time
Stephen Hawking
Science - Physics 2


SQL>


The CTXCAT Index that I built on BOOK_TITLE also includes BOOK_SUBJECT and SHELF_ID as indexed columns by virtue of the INDEX_SET called "BOOKS_SET".

Now, I add another row and verify if I need to Sync the index (as I had to do with the CONTEXT Index earlier).

SQL> insert into books
2 values
3 (6,'The Selfish Gene','Richard Dawkins','Evolution',2);

1 row created.

SQL> commit;
SQL> select book_title,book_author,book_subject,shelf_id
2 from books
3 where catsearch (book_title,'Gene','book_subject > ''S'' ') > 0
4 /

no rows selected

SQL> select book_title,book_author,book_subject,shelf_id
2 from books
3 where catsearch (book_title,'Gene','book_subject > ''E'' ') > 0
4 /

BOOK_TITLE
--------------------------------------------------------------------------------
BOOK_AUTHOR
--------------------------------------------------------------------------------
BOOK_SUBJECT SHELF_ID
------------------------- ----------
The Selfish Gene
Richard Dawkins
Evolution 2


SQL>


Note, specifically, how I could use the BOOK_SUBJECT in the query as if looking up a separate index on BOOK_SUBJECT.
The new book was included in the index without a call to CTX_DDL.SYNC_INDEX as would be required for the CONTEXT IndexType.

The portion of the query that is on the BOOK_TITLE column does a Text search on this column but the portions on BOOK_SUBJECT an SHELF_ID behave as with regular indexes.


(I know  that some readers will dispute the subject categorization "Evolution"  but I deliberately threw that in so that I  could show a query that uses a predicate filter not on "Science").

.
.
.




Categories: DBA Blogs

Oracle VM Server: my first vm: Error: HVM guest support is unavailable

Dietrich Schroff - Sat, 2018-04-21 09:47
All my tests with Oracle VM Server are running inside Oracle Virtualbox. If you want to do some tests yourself with this setup, you can easily get this error message after powering on your VM:

Server error: Command: ['xm', 'create', '/OVS/Repositories/0004fb0000030000dad74d9c43176d2e/VirtualMachines/0004fb00000600005e79798ecb1a63cf/vm.cfg'] failed (1): stderr: Error: HVM guest support is unavailable: is VT/AMD-V supported by your CPU and enabled in your BIOS?
stdout: Using config file "/OVS/Repositories/0004fb0000030000dad74d9c43176d2e/VirtualMachines/0004fb00000600005e79798ecb1a63cf/vm.cfg".To get your system running, you have to change this for your Oracle VM Server node on VirtualBox:


After that you get:

Server error: Command: ['xm', 'create', '/OVS/Repositories/0004fb0000030000dad74d9c43176d2e/VirtualMachines/0004fb0000060000b5dca8dccb8b74f6/vm.cfg'] failed (1): stderr: Error: Boot loader didn't return any data!
stdout: Using config file "/OVS/Repositories/0004fb0000030000dad74d9c43176d2e/VirtualMachines/0004fb0000060000b5dca8dccb8b74f6/vm.cfg".But this is due to specifying no boot media at your VM. Therefore you have to add an ISO image to your Oracle VM Server repository.

Can I do it with PostgreSQL? – 19 – Create user … identified by values

Yann Neuhaus - Sat, 2018-04-21 06:39

Puh, that last post in this series is already half a year old. Time is moving too fast :( Today, while being at a customer again, this question came up: Can I do something comparable in PostgreSQL to what I can do in Oracle, which is: Create a user and provide the hashed password so that the password is the same on the source and the target (which implies not knowing the password at all)? In Oracle you can find the hashed passwords in user$ where can I find that in PostgreSQL? Lets go.

When we look at the “create user” command there is no option which seems to do that:

postgres=# \h create user
Command:     CREATE USER
Description: define a new database role
Syntax:
CREATE USER name [ [ WITH ] option [ ... ] ]

where option can be:

      SUPERUSER | NOSUPERUSER
    | CREATEDB | NOCREATEDB
    | CREATEROLE | NOCREATEROLE
    | INHERIT | NOINHERIT
    | LOGIN | NOLOGIN
    | REPLICATION | NOREPLICATION
    | BYPASSRLS | NOBYPASSRLS
    | CONNECTION LIMIT connlimit
    | [ ENCRYPTED ] PASSWORD 'password'
    | VALID UNTIL 'timestamp'
    | IN ROLE role_name [, ...]
    | IN GROUP role_name [, ...]
    | ROLE role_name [, ...]
    | ADMIN role_name [, ...]
    | USER role_name [, ...]
    | SYSID uid

Maybe we can just pass the hashed password? Lets try be creating a new user:

postgres=# create user u with login password 'u';
CREATE ROLE

The hashed passwords in PostgreSQL are stored in pg_shadow:

postgres=# select passwd from pg_shadow where usename = 'u';
               passwd                
-------------------------------------
 md56277e2a7446059985dc9bcf0a4ac1a8f
(1 row)

Lets use that hash and create a new user:

postgres=# create user w login encrypted password 'md56277e2a7446059985dc9bcf0a4ac1a8f';
CREATE ROLE

Can we login as w using “u” as a password?

postgres@pgbox:/home/postgres/ [PG10] psql -X -h 192.168.22.99 -p $PGPORT -U w postgres -W
Password for user u: 
psql: FATAL:  no pg_hba.conf entry for host "192.168.22.99", user "w", database "postgres", SSL off

Ok, makes sense. After fixing that:

postgres@pgbox:/home/postgres/ [PG10] psql -X -h 192.168.22.99 -p $PGPORT -U w postgres -W
Password for user w: 
psql: FATAL:  password authentication failed for user "w"

So obviously this is not the way to do it. Do we have the same hashes in pg_shadow?

postgres=# select usename,passwd from pg_shadow where usename in ('w','u');
 usename |               passwd                
---------+-------------------------------------
 u       | md56277e2a7446059985dc9bcf0a4ac1a8f
 w       | md56277e2a7446059985dc9bcf0a4ac1a8f
(2 rows)

Hm, exactly the same. Why can’t we login then? The answer is in the documentation:”Because MD5-encrypted passwords use the role name as cryptographic salt, …”. We can verify that be re-creating the “w” user using the same password as that of user “u”:

postgres=# drop user w;
DROP ROLE
postgres=# create user w login password 'u';
CREATE ROLE
postgres=# select usename,passwd from pg_shadow where usename in ('w','u');
 usename |               passwd                
---------+-------------------------------------
 u       | md56277e2a7446059985dc9bcf0a4ac1a8f
 w       | md53eae63594a41739e87141e8333d15f73
(2 rows)

The hashed values are not the same anymore. What of course is working is to re-create the user with that hash:

postgres=# drop role w;
DROP ROLE
postgres=# create user w login password 'md53eae63594a41739e87141e8333d15f73';
CREATE ROLE

Now we should be able to login with the password ‘u':

postgres@pgbox:/home/postgres/ [PG10] psql -X -h 192.168.22.99 -p $PGPORT -U w postgres -W
Password for user w: 
psql (10.0 dbi services build)
Type "help" for help.

postgres=> 

Fine. Another way of getting the password hashes is to use pg_dumpall using the “–globals-only” switch:

postgres@pgbox:/home/postgres/ [PG10] pg_dumpall --globals-only > a.sql
postgres@pgbox:/home/postgres/ [PG10] grep -w w a.sql 
CREATE ROLE w;
ALTER ROLE w WITH NOSUPERUSER INHERIT NOCREATEROLE NOCREATEDB LOGIN NOREPLICATION NOBYPASSRLS PASSWORD 'md53eae63594a41739e87141e8333d15f73';

Hope that helps.

 

Cet article Can I do it with PostgreSQL? – 19 – Create user … identified by values est apparu en premier sur Blog dbi services.

GCP - How to manage SSH keys on VM Instance?

Surachart Opun - Fri, 2018-04-20 23:13
On Google Cloud Platform, adding SSH keys in Metadata (project-wide public SSH keys). It can help to ssh to every VM instances on Compute Engine easily but it's not a good idea. We are able to do for test, but should not use on Production. We should add SSH Key in OS login. 
https://cloud.google.com/compute/docs/instances/adding-removing-ssh-keys#risks



Question:
How can we block SSH Keys from Metadata(project-wide public SSH keys) on VM instance?
Answer: We can block by checking "Block project-wide SSH keys" on each instance.

In case we have SSH Key on metadata. We are able to ssh by using private key and login like.

So, we block it... On "Compute Engine" - "VM Instances", click [instance name] and "Edit". To check "Block project-wide SSH keys" and "Save".


VM instance should refuse Key what 's not in SSH  Keys of VM instance. (You can remove SSH Keys of project owner on instance, but it will be automatic added when you click "SSH" on GUI).

Additional, we should review and remove SSH Keys in metadata(project-wide public SSH keys), if we ensure we have not used. (Don't remove ssh key of project owner).

After removing, We would like to add SSH Key and don't want to add it in OS login. We are able to add it in SSH Keys on Instance like.


Assume: username is "myuser".

First of all, we have to generate Private and Public Keys. Example uses "PuTTY Key Generator". Because I use "Putty.


Then "Save private key" (We have to use when putty to server) and "Save public key".

To use public key on VM instance, click "Add item".



Example: It's [public key] [username]  

 then "Save".
Note: In picture, it's highlight about [username]

Open "putty", select "Private key file for authentication", fill in ip address and connect.

it's easy, right?
myuser@centos7:~$ id
uid=1003(myuser) gid=1004(myuser) groups=1004(myuser),4(adm),30(dip),44(video),46(plugdev),1000(google-sudoers)If we use command "id [user in project-wide SSH keys], we still see it, but it's unable to ssh on this VM instance.
myuser@centos7:~$ id opun
uid=1001(opun) gid=1002(opun) groups=1002(opun),4(adm),30(dip),44(video),46(plugdev),
Reference:
Categories: DBA Blogs

Relocate Goldengate Processes to Other Node with agctl

Pakistan's First Oracle Blog - Fri, 2018-04-20 22:00
Oracle Grid Infrastructure Agents can be used to manage Oracle Goldengate through Oracle GI. agctl is the utility to add, modify and relocate the goldengate. These Oracle GI agents can also be used with other products like weblogic, mysql etc. 


Frits has a good article about installation and general commands regarding GI agents for a non-clustered environment.

Following is the command to relocate Goldengate processes to other node with agctl. 


[gi@hostname ~]$ agctl status goldengate [service_name]
[gi@hostname ~]$ agctl config goldengate [service_name] [gi@hostname ~]$ agctl relocate goldengate [service_name] --node [node_name] [gi@hostname ~]$ agctl config goldengate [service_name] [gi@hostname ~]$ agctl status goldengate [service_name]

Hope that helps.
Categories: DBA Blogs

utl_http.begin_request results in protocol error when url size is big

Tom Kyte - Fri, 2018-04-20 17:26
Hi, while using utl_http package, we are able to make calls to a 3rd party webservice and all was going good till we hit transaction which resulted in big URL size - for ex one transaction had multiple rejections and url size is bigger than normal ...
Categories: DBA Blogs

Upgrade to 12c - High Fetch time vs. Low execution time

Tom Kyte - Fri, 2018-04-20 17:26
Hi Tom, We are migrating our databases from Oracle 11.2.0.3 to Oracle 12.1.0.2.0R1 on Exadata and after we did this, we are seeing extreme slowness in loading 3 of our application screens, even though the queries are running as or more efficiently...
Categories: DBA Blogs

BI Publisher Desktop 12.2 Certified with E-Business Suite

Steven Chan - Fri, 2018-04-20 11:14

[Contributing Author: Pieter Breugelmans]

You can use Oracle Business Intelligence Publisher (BI Publisher) to create and manage reports for E-Business Suite data. Oracle BI Publisher Desktop 12.2.1.3.0 is certified with Oracle E-Business Suite Release 12.2 and 12.1 for the following Microsoft Office releases:

  • Microsoft Office 2016
  • Microsoft Office 2013
  • Microsoft Office 2010

What does this certification cover?

Oracle BI Publisher Desktop consists of client-side tools to assist in the design and testing of layout templates for Oracle E-Business Suite Release 12.2 and 12.1. These layout templates are executed by Oracle BI Publisher on the Oracle E-Business Suite application tier as illustrated in the diagram below. The desktop utility consists of the following tools:

  • Template Builder Add-in for Microsoft Word - to build RTF layout templates
  • Template Builder Add-in for Microsoft Excel - to build Excel layout templates
  • Template Viewer - to test and debug all supported Oracle BI Publisher layout template types

For details, see:

Related Articles

 

Categories: APPS Blogs

Partner Webcast – Oracle Container Native Application Development Platform – Use with Kubernetes

Containerization of cloud applications rapidly becomes the right (only) way to deploy complex systems architected with microservice approach in mind. Containers solve one of the fundamental issues of...

We share our skills to maximize your revenue!
Categories: DBA Blogs

Partitioning vs Indexing

Tom Kyte - Thu, 2018-04-19 23:06
Hi Tom, i have a question in partitioning by list of a table. I have a set of tables which need to be hystoricized once a new record is inserted: then i have a STATUS column which flag an active status (AT) and a historic one (ST). To match this re...
Categories: DBA Blogs

Oralce Open v$open_cursor counts simple "updates" as open with the use of a cursor (open, execute, fetch, close, commit)

Tom Kyte - Thu, 2018-04-19 23:06
I am checking for open cursors while running our client server application (application info below) with the query below and noticed that a simple ?update? without the use of any cursors shows as open cursor. When another ?update? is issued its repla...
Categories: DBA Blogs

Performance issue/session getting hang

Tom Kyte - Thu, 2018-04-19 23:06
Hi Tom, I have a table having around 5 million records. Table Structure : DESC RPT_MSG_CHANGE <code>Name Null Type ---------- -------- -------------- OID NOT NULL NUMBER PRODUCT NOT NULL VARCHAR2(20)...
Categories: DBA Blogs

Quarterly EBS Upgrade Recommendations: April 2018 Edition

Steven Chan - Thu, 2018-04-19 11:50

We've previously provided advice on the general priorities for applying EBS updates and creating a comprehensive maintenance strategy.   

Here are our latest upgrade recommendations for E-Business Suite updates and technology stack components.  These quarterly recommendations are based upon the latest updates to Oracle's product strategies, latest support timelines, and newly-certified releases

You can research these yourself using this Note:

Upgrade Recommendations for April 2018

  EBS 12.2  EBS 12.1  EBS 12.0  EBS 11.5.10 Check your EBS support status and patching baseline

Apply the minimum 12.2 patching baseline
(EBS 12.2.3 + latest technology stack updates listed below)

In Premier Support to September 30, 2023

Apply the minimum 12.1 patching baseline
(12.1.3 Family Packs for products in use + latest technology stack updates listed below)

In Premier Support to December 31, 2021

In Sustaining Support. No new patches available.

Upgrade to 12.1.3 or 12.2

Before upgrading, 12.0 users should be on the minimum 12.0 patching baseline

In Sustaining Support. No new patches available.

Upgrade to 12.1.3 or 12.2

Before upgrading, 11i users should be on the minimum 11i patching baseline

Apply the latest EBS suite-wide RPC or RUP

12.2.7
Sept. 2017

12.1.3 RPC5
Aug. 2016

12.0.6

11.5.10.2
Use the latest Rapid Install

StartCD 51
Feb. 2016

StartCD 13
Aug. 2011

12.0.6


11.5.10.2

Apply the latest EBS technology stack, tools, and libraries

AD/TXK Delta 10
Sept. 2017

FND
Apr. 2017

EBS 12.2.6 OAF Update 11
Apr. 2018

EBS 12.2.5 OAF Update 19
Jan. 2018

EBS 12.2.4 OAF Update 18
Dec. 2017

ETCC
Jan. 2018

Web Tier Utilities 11.1.1.9

Daylight Savings Time DSTv28
Nov. 2016

Upgrade to JDK 7

Web ADI Bundle 5
Jan. 2018

Report Manager Bundle 5
Jan. 2018

FND
Apr. 2017

OAF Bundle 5
Jun. 2016

JTT Update 4
Oct. 2016

Daylight Savings Time DSTv28
Nov. 2016

Upgrade to JDK 7

 

 

Apply the latest security updates

Apr. 2018 Critical Patch Update

SHA-2 PKI Certificates

SHA-2 Update for Web ADI & Report Manager

Migrate from SSL or TLS 1.0 to TLS 1.2

Sign JAR files

Apr. 2018 Critical Patch Update

SHA-2 PKI Certificates

SHA-2 Update for Web ADI & Report Manager

Migrate from SSL or TLS 1.0 to TLS 1.2

Sign JAR files

Oct. 2015 Critical Patch Update April 2016 Critical Patch Update Use the latest certified desktop components

Use the latest JRE 1.8, 1.7, or 1.6 release that meets your requirements.

Switch to Java Web Start

Upgrade to IE 11

Upgrade to Firefox ESR 52

Upgrade Office 2003 and Office 2007 to later Office versions (e.g. Office 2016)

Upgrade Windows XP and Vista and Win 10v1507 to later versions (e.g. Windows 10v1607)

Use the latest JRE 1.8, 1.7, or 1.6 release that meets your requirements

Switch to Java Web Start

Upgrade to IE 11

Upgrade to Firefox ESR 52

Upgrade Office 2003 and Office 2007 to later Office versions (e.g. Office 2016)

Upgrade Windows XP and Vista and Win 10v1507 to later versions (e.g. Windows 10v1607)

    Upgrade to the latest database Database 11.2.0.4 or 12.1.0.2 Database 11.2.0.4 or 12.1.0.2 Database 11.2.0.4 or 12.1.0.2 Database 11.2.0.4 or 12.1.0.2 If you're using Oracle Identity Management

Upgrade to Oracle Access Manager 11.1.2.3

Upgrade to Oracle Internet Directory 11.1.1.9

Migrate from Oracle SSO to OAM 11.1.2.3

Upgrade to Oracle Internet Directory 11.1.1.9

    If you're using Oracle Discoverer

Migrate to Oracle
Business Intelligence Enterprise Edition (OBIEE), Oracle Business
Intelligence Applications (OBIA).

Discoverer 11.1.1.7 is in Sustaining Support as of June 2017

Migrate to Oracle
Business Intelligence Enterprise Edition (OBIEE), Oracle Business
Intelligence Applications (OBIA).

Discoverer 11.1.1.7 is in Sustaining Support as of June 2017

    If you're using Oracle Portal Migrate to Oracle WebCenter  11.1.1.9 Migrate to Oracle WebCenter 11.1.1.9 or upgrade to Portal 11.1.1.6 (End of Life Jun. 2017).

 

 
Categories: APPS Blogs

15 Minutes to get a Kafka Cluster running on Kubernetes – and start producing and consuming from a Node application

Amis Blog - Thu, 2018-04-19 11:07

imageFor  workshop I will present on microservices and communication patterns I need attendees to have their own local Kafka Cluster. I have found a way to have them up and running in virtually no time at all. Thanks to the combination of:

  • Kubernetes
  • Minikube
  • The Yolean/kubernetes-kafka GitHub Repo with Kubernetes yaml files that creates all we need (including Kafka Manager)

Prerequisites:

  • Minikube and Kubectl are installed
  • The Minikube cluster is running (minikube start)

In my case the versions are:

Minikube: v0.22.3, Kubectl Client 1.9 and (Kubernetes) Server 1.7:

image

The steps I went through:

Git Clone the GitHub Repository: https://github.com/Yolean/kubernetes-kafka 

From the root directory of the cloned repository, run the following kubectl commands:

(note: I did not know until today that kubectl apply –f can be used with a directory reference and will then apply all yaml files in that directory. That is incredibly useful!)

kubectl apply -f ./configure/minikube-storageclass-broker.yml
kubectl apply -f ./configure/minikube-storageclass-zookeeper.yml

(note: I had to comment out the reclaimPolicy attribute in both files – probably because I am running a fairly old version of Kubernetes)

kubectl apply -f ./zookeeper

kubectl apply -f ./kafka

(note: I had to change API version in 50pzoo and 51zoo as well as in 50kafka.yaml from apiVersion: apps/v1beta2 to apiVersion: apps/v1beta1 – see https://github.com/kubernetes/kubernetes/issues/55894 for details; again, I should upgrade my Kubernetes version)

To make Kafka accessible from the minikube host (outside the K8S cluster itself)

kubectl apply -f ./outside-services

This exposes Services as type NodePort instead of ClusterIP, making them available for client applications that can access the Kubernetes host.

I also installed (Yahoo) Kafka Manager:

kubectl apply -f ./yahoo-kafka-manager

(I had to change API version in kafka-manager from apiVersion: apps/v1beta2 to apiVersion: apps/v1beta1 )

At this point, the Kafka Cluster is running. I can check the pods and services in the Kubernetes Dashboard as well as through kubectl on the command line. I can get the Port at which I can access the Kafka Brokers:

image

And I can access the Kafka Manager at the indicated Port.

image

Initially, no cluster is visible in Kafka Manager. By providing the Zookeeper information highlighted in the figure (zookeeper.kafka:2181) I can make the cluster visible in this user interface tool.

Finally the eating of the pudding: programmatic production and consumption of messages to and from the cluster. Using the world’s simplest Node Kafka clients, it is easy to see the stuff is working. I am impressed.

I have created the Node application and its package.json file. Then added the kafka-node dependency (npm install kafka-node –save). Next I created the producer:

// before running, either globally install kafka-node  (npm install kafka-node)
// or add kafka-node to the dependencies of the local application

var kafka = require('kafka-node')
var Producer = kafka.Producer
KeyedMessage = kafka.KeyedMessage;

var client;
KeyedMessage = kafka.KeyedMessage;

var APP_VERSION = "0.8.5"
var APP_NAME = "KafkaProducer"

var topicName = "a516817-kentekens";
var KAFKA_BROKER_IP = '192.168.99.100:32400';

// from the Oracle Event Hub - Platform Cluster Connect Descriptor
var kafkaConnectDescriptor = KAFKA_BROKER_IP;

console.log("Running Module " + APP_NAME + " version " + APP_VERSION);

function initializeKafkaProducer(attempt) {
  try {
    console.log(`Try to initialize Kafka Client at ${kafkaConnectDescriptor} and Producer, attempt ${attempt}`);
    const client = new kafka.KafkaClient({ kafkaHost: kafkaConnectDescriptor });
    console.log("created client");
    producer = new Producer(client);
    console.log("submitted async producer creation request");
    producer.on('ready', function () {
      console.log("Producer is ready in " + APP_NAME);
    });
    producer.on('error', function (err) {
      console.log("failed to create the client or the producer " + JSON.stringify(err));
    })
  }
  catch (e) {
    console.log("Exception in initializeKafkaProducer" + JSON.stringify(e));
    console.log("Try again in 5 seconds");
    setTimeout(initializeKafkaProducer, 5000, ++attempt);
  }
}//initializeKafkaProducer
initializeKafkaProducer(1);

var eventPublisher = module.exports;

eventPublisher.publishEvent = function (eventKey, event) {
  km = new KeyedMessage(eventKey, JSON.stringify(event));
  payloads = [
    { topic: topicName, messages: [km], partition: 0 }
  ];
  producer.send(payloads, function (err, data) {
    if (err) {
      console.error("Failed to publish event with key " + eventKey + " to topic " + topicName + " :" + JSON.stringify(err));
    }
    console.log("Published event with key " + eventKey + " to topic " + topicName + " :" + JSON.stringify(data));
  });

}

//example calls: (after waiting for three seconds to give the producer time to initialize)
setTimeout(function () {
  eventPublisher.publishEvent("mykey", { "kenteken": "56-TAG-2", "country": "nl" })
}
  , 3000)

and ran the producer:

image

The create the consumer:

var kafka = require('kafka-node');

var client;

var APP_VERSION = "0.8.5"
var APP_NAME = "KafkaConsumer"

var eventListenerAPI = module.exports;

var kafka = require('kafka-node')
var Consumer = kafka.Consumer

// from the Oracle Event Hub - Platform Cluster Connect Descriptor

var topicName = "a516817-kentekens";

console.log("Running Module " + APP_NAME + " version " + APP_VERSION);
console.log("Event Hub Topic " + topicName);

var KAFKA_BROKER_IP = '192.168.99.100:32400';

var consumerOptions = {
    kafkaHost: KAFKA_BROKER_IP,
    groupId: 'local-consume-events-from-event-hub-for-kenteken-applicatie',
    sessionTimeout: 15000,
    protocol: ['roundrobin'],
    fromOffset: 'earliest' // equivalent of auto.offset.reset valid values are 'none', 'latest', 'earliest'
};

var topics = [topicName];
var consumerGroup = new kafka.ConsumerGroup(Object.assign({ id: 'consumerLocal' }, consumerOptions), topics);
consumerGroup.on('error', onError);
consumerGroup.on('message', onMessage);

consumerGroup.on('connect', function () {
    console.log('connected to ' + topicName + " at " + consumerOptions.host);
})

function onMessage(message) {
    console.log('%s read msg Topic="%s" Partition=%s Offset=%d'
    , this.client.clientId, message.topic, message.partition, message.offset);
}

function onError(error) {
    console.error(error);
    console.error(error.stack);
}

process.once('SIGINT', function () {
    async.each([consumerGroup], function (consumer, callback) {
        consumer.close(true, callback);
    });
});

and ran the consumer – which duly consumed the event published by the publisher. It is wonderful.

image

Resources

The main resources is the GitHub Repo: https://github.com/Yolean/kubernetes-kafka . Absolutely great stuff.

Also useful: npm package kafka-node – https://www.npmjs.com/package/kafka-node

Documentation on Kubernetes: https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/#section-2 – with references to Kubectl and Minikube – and the Katakoda playground: https://www.katacoda.com/courses/kubernetes/playground

The post 15 Minutes to get a Kafka Cluster running on Kubernetes – and start producing and consuming from a Node application appeared first on AMIS Oracle and Java Blog.

JavaOne Event Expands with More Tracks, Languages and Communities – and New Name

OTN TechBlog - Thu, 2018-04-19 11:00

The JavaOne conference is expanding to create a new, bigger event that’s inclusive to more languages, technologies and developer communities. Expect more talks on Go, Rust, Python, JavaScript, and R along with more of the great Java technical content that developers have come to expect. We’re calling the new event Oracle Code One, October 22-25 at Moscone West in San Francisco.

Oracle Code One will include a Java technical keynote with the latest information on the Java platform from the architects of the Java team.  It will also have the latest details on Java 11, advances in OpenJDK, and other core Java development.  We are planning dedicated tracks for server side Java EE technology including Jakarta EE (now part of the Eclipse Foundation), Spring, and the latest advances in Java microservices and containers.  Also a wealth of community content on client development, JVM languages, IDEs, test frameworks, etc.

As we expand, developers can also expect additional leading edge topics such as chatbots, microservices, AI, and blockchain. There will also be sessions around our modern open source developer technologies including Oracle JET, Project Fn and OpenJFX.

Finally, one of the things that will continue to make this conference so great is the breadth of community run activities such as Oracle Code4Kids workshops for young developers, IGNITE lightning talks run by local JUG leaders, and an array of technology demos and community projects showcased in the Developer Lounge.  Expect a grand finale with the Developer Community Keynote to close out this week of fun, technology, and community.

Today, we are launching the call for papers for Oracle Code One and you can apply now to be part of any of the 11 tracks of content for Java developers, database developers, full stack developers, DevOps practitioners, and community members.  

I hope you are as excited about this expansion of JavaOne as I am and will join me at the inaugural year of Oracle Code One!

Please submit your abstracts here for consideration:
https://www.oracle.com/code-one/index.html

Long Raw to BLOB

Tom Kyte - Thu, 2018-04-19 04:46
Hi Tom, We are using an Oracle 8.1.7 database. Is there a way in PL/SQL or Java Stored Procedure to convert a Long Raw into a BLOB? Thanks, Firas Khasawneh
Categories: DBA Blogs

Remote and Programmatic Manipulation of Docker Containers from a Node application using Dockerode

Amis Blog - Thu, 2018-04-19 02:23

imageIn previous articles, I have talked about using Docker Containers in smart testing strategies by creating a container image that contains the baseline of the application and the required test setup (test data for example). For each test instead of doing complex setup actions and finishing of with elaborate tear down steps, simply spinning up a container at the beginning and tossing it away at the end.

I have shown how that can be done through the command line – but that of course is not a workable procedure. In this article I will provide a brief introduction of programmatic manipulation of containers. By providing access to the Docker Daemon API from remote clients (step 1) and by leveraging the npm package Dockerode (step 2) it becomes quite simple from a straightforward Node application to create, start and stop containers – as well as build, configure, inspect, pause them and manipulate in other ways. This opens up the way for build jobs to programmatically run tests by starting the container, running the tests against that container and killing and removing the container after the test. Combinations of containers that work together can be managed just as easily.

As I said, this article is just a very lightweight introduction.

Expose Docker Daemon API to remote HTTP clients

The step that to me longest was exposing the Docker Daemon API. Subsequent versions of Docker used different configurations for this and apparently different Linux distributions also have different approaches. I was happy to find this article: https://www.ivankrizsan.se/2016/05/18/enabling-docker-remote-api-on-ubuntu-16-04 that describes for Ubuntu 16.x as Docker Host how to enable access to the API.

Edit file /lib/systemd/system/docker.service – add -H tcp://0.0.0.0:4243 to the entry that describes how to start the Docker Daemon in order to have it listen to incoming requests at port 4243 (note: other ports can be used just as well).

Reload (systemctl daemon-reload) to apply the changed file configuration

Restart the Docker Service: service docker restart

And we are in business.image

A simple check to see if HTTP requests on port 4243 are indeed received and handled: execute this command on the Docker host itself:

curl http://localhost:4243/version

image

The next step is the actual remote access. From a browser running on a machine that can ping successfully to the Docker Host – in my case that is the Virtual Box VM spun up by Vagrant, at IP 192.168.188.108 as defined in the Vagrantfile – open this URL: http://192.168.188.108:4243/version. The result should be similar to this:

image

Get going with Dockerode

To get started with npm package Dockerode is not any different really from any other npm package. So the steps to create a simple Node application that can list, start, inspect and stop containers in the remote Docker host are as simple as:

Use npm init to create the skeleton for a new Node application

Use

npm install dockerode –save

to retrieve Dockerode and create the dependency in package.json.

Create file index.js. Define the Docker Host IP address (192.168.188.108 in my case) and the Docker Daemon Port (4243 in my case) and write the code to interact with the Docker Host. This code will list all containers. Then it will inspect, start and stop a specific container (with identifier starting with db8). This container happens to run an Oracle Database – although that is not relevant in the scope of this article.

var Docker = require('dockerode');
var dockerHostIP = "192.168.188.108"
var dockerHostPort = 4243

var docker = new Docker({ host: dockerHostIP, port: dockerHostPort });

docker.listContainers({ all: true }, function (err, containers) {
    console.log('Total number of containers: ' + containers.length);
    containers.forEach(function (container) {
        console.log(`Container ${container.Names} - current status ${container.Status} - based on image ${container.Image}`)
    })
});

// create a container entity. does not query API
async function startStop(containerId) {
    var container = await docker.getContainer(containerId)
    try {
        var data = await container.inspect()
        console.log("Inspected container " + JSON.stringify(data))
        var started = await container.start();
        console.log("Started "+started)
        var stopped = await container.stop();
        console.log("Stopped "+stopped)
    } catch (err) {
        console.log(err);
    };
}
//invoke function
startStop('db8')

The output in Visual Studio Code looks like this:

SNAGHTML26a0b0e

And the action can be tracked on the Docker host like this (to prove it is real…)image

Resources

Article by Ivan Krizsan on configuring the Docker Daemon on Ubuntu 16.x – my life safer: https://www.ivankrizsan.se/2016/05/18/enabling-docker-remote-api-on-ubuntu-16-04

GitHub Repo for Dockerode – with examples and more: https://github.com/apocas/dockerode

Presentation at DockerCon 2016 that gave me the inspiration to use Dockerode: https://www.youtube.com/watch?v=1lCiWaLHwxo 

Docker docs on Configuring the Daemon – https://docs.docker.com/install/linux/linux-postinstall/#configure-where-the-docker-daemon-listens-for-connections


The post Remote and Programmatic Manipulation of Docker Containers from a Node application using Dockerode appeared first on AMIS Oracle and Java Blog.

Garbage First in JDeveloper

Darwin IT - Thu, 2018-04-19 01:07
At my current customer we work with VDI's: Virtual Desktop Images, that at several times a day very, very slow. Even so slow that it more or less stalls for a minute or two.

JDeveloper is not known as a Ferrari under the IDE's. One of the causes is that by default heap settings is very poor: 128M-800M. Especially when you use it in  SOA or BPM Quickstart then at startup it will need to grow several times. But very soon working in it you'll get out of memory errors.

Because of the VDI's I did several changes to try to improve performance.
Main thing is set Xms and Xmx both at 2048M. I haven't found needing more up to this day.

But I found using the Garbage First collector gives me a slightly better performance.

To set it, together with the heap, add/change the following options in the ide.conf in ${JDEV_HOME}\jdeveloper\ide\bin\:
# Set the default memory options for the Java VM which apply to both 32 and 64-bit VM's.
# These values can be overridden in the user .conf file, see the comment at the top of this file.
#AddVMOption -Xms128M
#AddVMOption -Xmx800M
AddVMOption -Xms2048M
AddVMOption -Xmx2048M
AddVMOption -XX:+UseG1GC
AddVMOption -XX:MaxGCPauseMillis=200

Find more on the command line options in this G1GC tutorial.

You can also use the ParNew incombination with the ParOld or ConcMarkSeep collector, as suggested in this blog. But from Java9 onwards G1GC is the default, and I expect that it better fits the behavior of JDeveloper, as in SOASuite and OSB installations.

Pages

Subscribe to Oracle FAQ aggregator