Removing unused files from Oracle SOA Suite MDS repository

In this post we will see how to remove unused files from oracle soa suite MDS repository.

For removing the files you need to connect to your server using wlst scripts.

we have to use the deleteMetaData function of wlst to remove the files.

Follow the below steps to remove the files from repository.

Login to your unix box and navigate to the below path.

$FMW_HOME/ORACLE_SOA1/common/bin/wlst.sh

[testuser@soaserver]$ cd /u01/oracle/SOAInstall/soa/common/bin
[testuser@soaserver]$ ./wlst.sh

wls:/fmw_domain/serverConfig>connect(‘username,’pwd’, ‘hostname:7001’)

wls:/fmw_domain/serverConfig>deleteMetadata(application=’soa-infra’,server=’soa_server1′,docs=’/apps/wsdl/oracle/customer/*’)

Executing operation: deleteMetadata. 

Operation “deleteMetadata” completed. Summary of “deleteMetadata” operation is:

List of documents successfully deleted:

/apps/wsdl/oracle/customer/custfile.wsdl

1 documents successfully deleted.

How to export and import MDS metadata in SOA Suite 12c Em console

In this post we will see how to export and import MDS metadata from oracle enterprise manager console to the file system.

Export –>

Login to Em console and click on SOA-Infra and MDS Configuration.

Select the option “Export metadata document to an archive on the machine where this web browser is running” and Click on Export.

Once after you press OK, it will take some time to export the meta data to a file.

If you open the file , it will show you fault policies, wsdl, xml schema and all configuration settings of your server.

Import –>

Login to Em console and click on SOA-Infra and MDS Configuration.

Select the option “Import metadata documents from an archive on the machine where this browser is running ” and Click on browse to select the file and click on Import.

Exporting MDS metadata using WLST

Here we will see , how to export MDS metadata using wlst commands.

login to your unix box  and launch wlst.

[testuser@soaserver]$ cd /u01/oracle/SOAInstall/soa/common/bin

[testuser@soaserver]$ ./wlst.sh

wls:/fmw_domain/serverConfig>connect(‘username,’pwd’, ‘hostname:7001’)

wls:/fmw_domain/serverConfig>listApplications()

wls:/fmw_domain/serverConfig>exportMetadata(application=’soa-infra’, server=’soa_server1′, toLocation=’/u01/oracle/MyMetadata’)

Removing Files & Folders from SOA MDS

There are scripts available over internet to remove the artifacts from MDS. But there are couple of other methods as well to achieve this without using script. And they are -

Remove entry from MDS_PATHS table -
Connect to SOA_MDS and search the table MDS_PATHS. This table holds the information of all the folder that are registered in MDS. Deleting an entry from this table will remove the artifact from the MDS as well. But the catch with this method is even if you remove the entry in this table there are various other related tables where the entry will still exist.

WLST deleteMetadata method -
WLST has deleteMetadata method that can be used to delete the file/folder from the MDS. It can be done through command line or from EM console as well.

Doing it command line way -

run (UNIX) MIDDLEWARE_HOME/ORACLE_SOA1/common/bin/wlst.sh

offline>connect('user','password','hostname:7001')
wls:/soa_domain/serverConfig> deleteMetadata(application='soa-infra',server='soa
_server1',docs='/apps/wsdl/*')

Executing operation: deleteMetadata.

Operation "deleteMetadata" completed. Summary of "deleteMetadata" operation is:

List of documents successfully deleted:
/apps/wsdl/test.wsdl

1 documents successfully deleted.

Doing it via EM console -

Right click soa-infra and goto MDS configuration as shown in image below.

 

 select "Runtime Configuration"

Then goto operations and click deleteMetadata.

Edit the docs and provide the similar arguments as shown in command line and click invoke. That should remove the folder as passed in the argument.

Oracle MDS to collaborate XML objects across Oracle SOA-projects

Background

How to collaborate/share a predefined xml schema or wsdl or edl (in fact, any kind of static or application resources) across multiple oracle soa-projects ?  Example: if a group of techies working on different Oracle fmw/soa-projects and needs to share a common/canonical/predefined "employee" schema, then what can be their approach?   

The answer can be - there are many ways thru which a predefined "employee" schema (here, I am taking "employee" schema as an example) can be shared across multiple Oracle soa-projects.  One of the common practice is - copy that employee-schema (meaning import the schema from local file system) into all the required soa-projects in jdeveloper. In many scenarios, this approach works fine, but when you need to change the schema, you will be required to update every copy :(

So the better approach is to have a single copy of employee-schema and make a reference (instead copy or import) of it in all the required soa-projects and composites.  But how ???  

Solution thru MDS

There are two ways which allows you to share the common artifacts across  Oracle soa-projects, those are MDS (metadata service) and OER. For now, my topic is on MDS (in my later posts, I can provide some details on setup and usage of OER).  MDS supports two types of repositories, those are file based and database repositories.  In this blog, I am providing the details of database based MDS repository.

Key steps in defining and using MDS Database repository are -

• Create common files (with a proper directory structure) in local Jdeveloper.
• Deploy/export that directory along with files to MDS (FMW database)
• Refer those common files (from MDS) in Jdeveloper and soa-projects.

Stage1: Create predefined/common files (or directory) in local file system.

• create a directory called "apps" (you can create with any name) in <jdeveloper_home>/jdeveloper/integration/seed
• keep all the files (wsdl, xml schemas, edl, property files - what not any file you can think of) in this directory, which you want to share across all the SOA projects and composites. 

Stage2: Deploy that directory (or files) to MDS

• Create a dummy SOA Project and define the deployment profile (at project level) to include (contribute) the above created directory (in our case it is "apps"). Projects -> Deployment -> new -> Archive type is Jar -> In the Deployment profile properties -> select Contributors -> select the required directories and files -> and OK.

• Make sure that corresponding Application of above dummy project should also have a deployment profile of Archive type "SOA Bundle", and previously created dummy project profile as a dependent. Application Menu -> Application properties -> Deployment -> new -> Archite type : SOA Bundle -> select previously created dummy project's deployment profile. thats it...

• Once the dummy project's and application's deployment profiles are created, deploy the application as a SAR file in to your local system and deploy it to Weblogic server.

Stage3: Refer those common files (from MDS) in Jdeveloper and soa-projects

• In Jdeveloper, create a SOA_MDS connection from the Items list. This will launch MDS Connection wizard.  Specify some needed details such as connection location, connection type (in our case it is MDS Database) and provide the necessary weblogic server details such as servername, portname, userid/password and other connection details.  thats it.. you are all set with the MDS repository access in Jdeveloper.

• Once MDS connection is established, always refer the files directly from MDS-Repository instead create a local copies at project level.  Example: We can import schemas (or any files or application resources) from mrs in a similar fashion to how we import from local file system.  Instead selecting the file system (browse button), launch SOA Resource browser -> resource Palette -> select schema that you wish to import -> make sure to "De SELECT" copy to Project option.  This will refer the schema directly from MDS and the schemaLocation attribute contains "oramds:/apps/<<file/schema name>>"

This MetadataService (MDS) allows you to share (centralize) common artifacts such as schemas, wsdls, edls and some application resources across SOA-Compisites.

SOA SUITE 12c Data Base Schema Dehydration Store States

Hi All,

As me, you might be stumbling to find out what, similar to List of all states from COMPOSITE_INSTANCE, CUBE_INSTANCE, DLV_MESSAGE  in 11g, the important tables would be for SOA Suite 12c and relevant states.

Basically this is described very well in the Oracle Support Document 2065869.1: 12c SOA composite <schema> States

For the benefit of every one, I have also copy pasted the content from the obove Oracle Note to here (copy right of the below details are with Oracle Support):

=======================================================================

Details

Based on 11g:

Tables COMPOSITE_INSTANCE and MEDIATOR_INSTANCE does not apply to 12c.

Tables CUBE_INSTANCE, DLV_MESSAGE and DLV_TYPE did not have any change.

The FLOW_INSTANCE is available only in 12c.

CUBE_INSTANCE States

State	Description
0	STATE_INITIATED
1	STATE_OPEN_RUNNING
2	STATE_OPEN_SUSPENDED
3	STATE_OPEN_FAULTED
4	STATE_CLOSED_PENDING_CANCEL
5	STATE_CLOSED_COMPLETED
6	STATE_CLOSED_FAULTED
7	STATE_CLOSED_CANCELLED
8	STATE_CLOSED_ABORTED
9	STATE_CLOSED_STALE
10	STATE_CLOSED_ROLLED_BACK

DLV_MESSAGE States

State	Description
0	STATE_UNRESOLVED
1	STATE_RESOLVED
2	STATE_HANDLED
3	STATE_CANCELLED
4	STATE_MAX_RECOVERED

DLV_TYPE States

State	Description
1	Invoke Message
2	DLV Message

MEDIATOR States

Mediator state is now derived from SCA_ASYNC_CPNT_STATE table. The state column can take one of the following values:

State	Description
1	RUNNING
2	COMPLETED
3	FAULTED
4	ABORTED
5	STALE

FLOW_INSTANCE States

State	Description
-1	NOT_SET
1	SUSPENDED
2	TERMINATED
3	STALE

  

The Flow State is derived from the Flow Instance table using the following columns:

• Active_Component_Instances
• Unhandled_Faults
• Recoverable_Faults
• Admin_State

Completed Flow
0 Active_Component_Instances

Completed Faulted Flow
0 Active_Component_Instances
1 or more Unhandled_Faults

Recoverable Flows
1 or more Active_Component_Instances
1 or more Recoverable_Faults

Completed Successfully
0 Active_Component_Instances
0 Unhandled_Faults
0 Recoverable_Faults

  

======================================================================

Further to the above Info, I also found a very useful blog that provides more knowledge / usage to the above table. 

Oracle BPEL 12c Data model changes and FAQ

Hope this was helpful to you.

Steps to delete data from MDS ( soa-inrfa ) 11g

In my previous blog I explained how to add content in MDS repository . Removing content is very simple compared with adding.I don't have any idea about how to remove data in MDS ( GUI mode ).


WSLT ( weblogic scripting tool )


$ oracle/Middleware/oracle_common/common/bin

$ wlst.sh

Make a connection to weblogic server .

wls:/offline> connect('weblogic','********','t3://localhost:7001')

already MDS Contains fault-policies.xml

/apps/com/xsd/fault-policies.xml 


wls:/offline> deleteMetadata(application='soa-infra',server='AdminServer',docs='/apps/com/xsd/fault-policies.xml')

application -- > soa-infra
server        -->  Adminsever is not mandatory . eg : soa_sever1  ( Managed servers )
docs         -->   /apps    by default , then file location.

wls:/offline> deleteMetadata(application='soa-infra',server='AdminServer',docs='/apps/**')

** will delete all the files, so be careful while using this command

Done ............

Oracle Metadata Services (MDS) repository Oracle SOA-MDS / BPM-MDS 11g

MDS  - Matadata Service.

Why MDS ?

 From Development to Production moving complete code to server from Jdveloper and maintain complete code base version .

Two types of MDS :

     

  1. File system MDS :

                             

 Advantage :  File system MDS are useful in terms of maintain complete xsd,wsdl,xml,etc  in same system .so when ever developers want to change any elements in xsd,wsdl,xml,etc are very frequently.

If I use file system MDS during development time  , then how weblogic server will understand  the local file system reference during run time ???

simple create jar ,then  convert into soa-bundle and deploy to the server. so in adf-config.xml file system MDS URL are not hard coded . I would suggest to use file system MDS.

Dis-advantage :   Multiple developers not able to use during development .

 2. Database MDS :

Advantage :  During development time multiple developers can share the same wsdl,xml,xsd  from Database MDS.

Dis-advantage :  MDS URL hard code in adf-config.xml ,when we generate configuration plan. those MDS URL are not  a part of any reference or as a service.

Example :  Business Analysist  create BPM template and those templates are saved in MDS repository . Developer job to check out the complete template  code and develop the process .

I listed few usage of MDS .

Step to create File system MDS Connection :

 create folder name apps in below location  :

              Jdeveloper11.6\jdeveloper\integration\seed

apps --> place xsd, wsdl, xml , etc ..

Resource Palette - > IDE Connection - > SOA - MDS 

now  make reference to  file system MDS in adf-config.xml . 

ApplicationResource - > Descriptors - > ADF  META-INF - > adf-config.xml 

 <?xml version="1.0" encoding="windows-1252" ?>

<adf-config xmlns="http://xmlns.oracle.com/adf/config"
            xmlns:config="http://xmlns.oracle.com/bc4j/configuration"
            xmlns:adf="http://xmlns.oracle.com/adf/config/properties"
            xmlns:sec="http://xmlns.oracle.com/adf/security/config">
  <adf-adfm-config xmlns="http://xmlns.oracle.com/adfm/config">
    <defaults useBindVarsForViewCriteriaLiterals="true"/>
    <startup>
      <amconfig-overrides>
        <config:Database jbo.locking.mode="optimistic"/>
      </amconfig-overrides>
    </startup>
  </adf-adfm-config>
  <adf:adf-properties-child xmlns="http://xmlns.oracle.com/adf/config/properties">
    <adf-property name="adfAppUID" value="BPELAsResApplication.Krishna"/>
  </adf:adf-properties-child>
  <sec:adf-security-child xmlns="http://xmlns.oracle.com/adf/security/config">
    <CredentialStoreContext credentialStoreClass="oracle.adf.share.security.providers.jps.CSFCredentialStore"
                            credentialStoreLocation="../../src/META-INF/jps-config.xml"/>
  </sec:adf-security-child>
  <adf-mds-config xmlns="http://xmlns.oracle.com/adf/mds/config">
    <mds-config xmlns="http://xmlns.oracle.com/mds/config">
      <persistence-config>
        <metadata-namespaces>
          <namespace metadata-store-usage="mstore-usage_1" path="/soa/shared"/>
          <namespace metadata-store-usage="mstore-usage_2" path="/apps"/>
          </metadata-namespaces>
        <metadata-store-usages>
          <metadata-store-usage id="mstore-usage_1">
            <metadata-store class-name="oracle.mds.persistence.stores.file.FileMetadataStore">
              <property value="${oracle.home}/integration"
                        name="metadata-path"/>
              <property value="seed" name="partition-name"/>
            </metadata-store>
          </metadata-store-usage>
          <metadata-store-usage id="mstore-usage_2">
            <metadata-store class-name="oracle.mds.persistence.stores.file.FileMetadataStore">
              <property value="${oracle.home}/integration"
                        name="metadata-path"/>
              <property value="seed" name="partition-name"/>
            </metadata-store>
          </metadata-store-usage>
         </metadata-store-usages>
      </persistence-config>
    </mds-config>
  </adf-mds-config>
</adf-config>

as I mentioned earlier  during development time apps folder reference in local system , but when the same deployed in weblogic server. Reference inside adf-config.xml automatically resolve during runtime.

Step to create SOA_Bundle ( MDS weblogic server ) and Database MDS :

yet to define .......

How to Undeploy Composite Manually

Sometime I noticed that you can’t undeploy the composite from SOA EM console.It is little bit irritating and can be of various reason like not deployed properly , database reference is null etc.

But I figured out undeployment procedure of SOA composite manually apart from wizard based.It worked fine for me.Here are the steps,

Login to EM console and go to mds configuration,

Export metadata,

soa-infra_metadata.zip file will be downloaded.Unzip the same and you will find deployed-composites folder and underneath deployed-composites.xml like below,

Delete the entry of whatever the composite you want to undeploy.You can delete the respective composite from your partition folder. Then zip it again and import the MDS from EM.

You need to restart the server and after that your composite will be gone !!!