Monitoring Certificate Expiry Dates of a JAVA Keystore (.JKS) File in a WebLogic Domain (Shell Script, Scheduling, Alerting)

| December 31, 2017 | 3 Comments

One of my client asked me how to monitor expiry dates of JKS certificate entries in a WebLogic domain. If an imported certificate entry expires in a JKS (Java Keystore File) then the HTTPS transactions is going to fail because of SSL handshake exception/failure.

So, a proactive administrator should monitor these certificates and should warn the 3rd party end point URL’s administrator or update/import the certificate with the new one before the failure. Especially, if you are working with OSB (Oracle Service Bus) and SOA Suite 11g/12c WebLogic domains it is very crucial to monitor imported JKS certificate entries.

 

Monitoring Certificate Expiry Dates of a JAVA Keystore (.JKS) File in a WebLogic Domain (Shell Script, Scheduling, Alerting)

 

Monitoring Certificate Expiry Dates of a JAVA Keystore (.JKS) File in a WebLogic Domain (Shell Script, Scheduling, Alerting)

How to monitor imported certificates (CRT, CER) expiry / expiration date in a JKS (Java Keystore) file and create remaining day alerts/notifications?

There are too many alternatives to do that. I have found some other scripts available but all of them based on JKS alias entries. Then decided to write a new one from scratch to iterate all JKS aliases in the JKS file.

I’ll explain two method for doing this JKS certificate monitoring in WebLogic.

 

  • Shell scripting and OS Cron Job
  • Shell script and WLSDM Scheduler

 

At the end of post I’ll put download link of the script and you can download it. Go to my GitHub project page to download entire project.

Step by Step Script Usage and Integration via Server CRON Job:

1. Download monitorJKS folder and upload to your server (Do it at the end, firstly read the whole tutorial)

2. Edit environment variables according to your server and WebLogic installation paths.

 

3. Variable Info:

3.1.  Change “__thresholdDay” value according to your preference. By default, I have set it with 7 which means it will start to send ALERT emails 7 days before the certificate expiration.

3.2. Do not forget about to set your local  “__scriptPath” value.

4. Comment out below section for the SMTP email delivery. (Line 58)

5. Create your CRON job for /path/to/monitorJKS/monitorJKSforWebLogic_Lnx.sh script. Running once in a day is just enough. Schedule for every midnight.

 

Step by Step Script Usage and Integration via WLSDM Scheduler:

Scheduling this script via WLSDM for WebLogic is more exciting and smarter. Because, I am going to use stdErr functionality of Shell and WLSDM interface.

1. Copy monitorJKSforWebLogic_Lnx.sh to server.

2. Go to “WLSDM Console > Configuration > Monitoring & Diagnostics > User Defined Actions / Scripts (tab)”  page.

WLSDM Console Menu (Portlet in WebLogic Console)

WLSDM Console Menu (Portlet in WebLogic Console)

3. In the page operations menu (on top-right of every page) click on “New User Defined Action / Script” button.

4. In the modal window add shell script file as below and save.

WLSDM: New User Defined Action / Script

WLSDM: New User Defined Action / Script (Click to enlarge)

5. Switch to “Scheduled Jobs / Downtime” tab.

6. In the page operation menu (on top-right of every page) click on “New Scheduled Jobs / Downtime” button.

7. In the modal window add new CRON job as below and save. (you can choose every minute for testing the cron job temporarily)

WLSDM: New Scheduled Job / Downtime

WLSDM: New Scheduled Job / Downtime (Click to enlarge)

 

That’s it! The integration and monitoring of JKS certificates expiry date is done. Let’s test it and see the results.

I chose every minute to test the script and understand that WLSDM scheduler is working properly. The threshold in the script is  100 days which is a very high value; in a production system it must be something like 5 or 7 days. In my test case the threshold is:

__thresholdDay="100" 

Let’s see the WLSDM scheduler output (enlarge below screen capture). Here is the job records and its results. It demonstrates all the JKS monitoring processes passed successfully and we will not get any email alert from WLSDM.

WLSDM for WebLogic: Scheduled Job Execution History (click to enlarge)

WLSDM for WebLogic: Scheduled Job Execution History (click to enlarge)

Drill down each monitoring cycle detail by clicking “View Output/Details” lens in the “Scheduled Job Execution History” list. WLSDM reflects all the “SHELL/BASH stdOut” and “echo/printf” message values in this modal window page. It is possible to see Linux/Unix/Windows shell/bash (terminal) scripts’ stdOut and stdErr values in this page. Click below screen capture to see a successful JKS monitoring output for my shell script/action which is monitorJKS.sh.

WLSDM CRON Job Detail Modal Window: Monitoring JKS certificate expiry date

WLSDM CRON Job Detail Modal Window: Monitoring JKS certificate expiry date (click to enlarge)

As you see the remaining days for the alias volthreadcom is 124 days. The threshold is 100 which means remaining days are more than threshold; so the result is as expected: –> Script executed successfully! Certificates are OK.

For the second scenario; I am going to change the threshold from 100 to 150. In this case volthreadcom JKS certificate entry would be in trouble :) I am expecting to get a failed job result. Because the shell script will terminate itself with error (check “exit 1″ in the script)

After updating threshold to 150; the WLSDM cron job switched to failed status immediately. Check below screen captures!

WLSDM for WebLogic: Failed Scheduler CRON Jobs List (click to enlarge)

WLSDM for WebLogic: Failed Scheduler CRON Jobs List (click to enlarge)

Click “View Details/Output” button to see what’s going on for the JKS certificate entries.

WLSDM Cron Job Details: View Warning for Certificate Expiry Dates (click to enlarge)

WLSDM Cron Job Details: View Warning for Certificate Expiry Dates (click to enlarge)

YESSS! Here is the most important part. I have received an email for the certificates that’s going to expire according to my threshold.

Click below image to enlarge and check the details for understanding how to receive HTML emails for the certificates going to expire.

JKS Certificate Expiry Date Warning: WLSDM for WebLogic Scheduler HTML Email Notification (click to enlarge)

JKS Certificate Expiry Date Warning: WLSDM for WebLogic Scheduler HTML Email Notification (click to enlarge)

 

Actually I am cheating WLSDM. Because; I know that WLSDM informs administrators about failed cron jobs/scripts by sending HTML email notifications as above. So, when I get an email about current “WLSDM WebLogic scheduled job” probably it means there is a WARNING about JKS certificate expiry date. Check below section in the script especially the (exit 1) part then you will understand what I mean. Otherwise please let me know to think about your case together :)

I have finished the documentation part and reading part. I have also prepared a Youtube tutorial/screencast about monitoring JKS entries. I’ve recorded the entire video on my iMac. The date function/command is different in MacOS (Sierra) than Linux. I have did some tricks for executing my shell script properly for macOS environment (If you are interested knock my door).

If you already read this technical blog post; please also watch the entire technical screencast it will inspire you about your PRODUCTION environment monitoring cases. Especially about WebLogic cron job integrations and definitions.

Youtube Screencast Tutorial: Monitoring Certificate Expiry Date of JKS

(Please share the screencast with other Oracle FMW WebLogic Community. Regards!) –> Sharing is just caring Fevzi Korkutata :)

Source Code (Script and sample output file):

Github URL: https://github.com/admineer/monitorJKSforWebLogic

All the project source and sample output files are available in the Github monitorJKSforWebLogic project. I am also adding the entire script as below. (Copy/Download the script from Github; because the shell script source file is well formated at there)

 

 

 

LinkedInTwitterGoogle+FacebooktumblrEmail
    Share admineer resources...

Tags: , , , , ,

Category: Linux, Middleware, OSB, Shell, SOA, WebLogic

M.Fevzi Korkutata

About the Author ()

M.Fevzi Korkutata: Deep level technical consultant... Oracle ACE in Middleware & SOA expertise. His specialty is large scale and mission critical live production systems and like to work as DevOps. He knows all kind of application servers and its environments. Software product development, operation management, design, implementation, integration... etc. Korkutata working as "Application and Infrastructure Architect"... Likes to work and communicate internationally, stay connected :)

Comments (3)

Trackback URL | Comments RSS Feed

  1. Moumita says:

    Hi,

    Thanks for the article. I have certain queries regarding how to edit environment variables and also how to create the cron job. Can I contact you. If yes, need your contact detail.
    Please advise.

    • Fevzi Korkutata says:

      Hi,
      These queries are common and off topic about using the script. Just google it :) Yes you can reach me by using contact :)

  2. Raghu Kumar says:

    Hi,

    I need one help with the script.
    In my company environment there are multiple JVM’s running on single server each is having a separate JKS file. Is there a way to consolidate all the JKS file and get the result.
    Eg:
    __keystore=”/webdata/dev/PT855H92/webserv/HRGPSD1/piaconfig/keystore/pskey”
    __keystore=”/webdata/sit/PT855H92/webserv/HRGPST1/piaconfig/keystore/pskey”
    __keystore=”/webdata/dev/PT855H92/webserv/HRPSPD1/piaconfig/keystore/pskey”
    __keystore=”/webdata/sit/PT855H92/webserv/HRPSPT1/piaconfig/keystore/pskey”

    Do we have any script to get results from all the keystores at once. Instead of creating separate script for each JKS File.

    Regards,
    Raghu

Leave a Reply to Fevzi Korkutata Cancel reply

Your email address will not be published. Required fields are marked *