Micro Focus Diagnostics – Tips and Tricks

Table of Contents

Micro Focus Diagnostics – Tips and Tricks – June 2021

1. How to restore the symbol table?

Did you by mistake delete your symbol table? No. I will help you restore it, effortlessly. 

Solution

Follow the steps below to restore the corrupted Sleepycat (symbol table)

Follow the below written commands.

  • $ cd lib
  • $ mkdir foo
  • $ mkdir bar
  • $ java -cp “.;je.jar” com.sleepycat.je.util.DbDump -v -h ../archive/mediator-mhaeuptle2/symboltable -d foo -s symboltable -r $ java -cp “.;je.jar” com.sleepycat.je.util.DbLoad -v -f foo/localsymboltable.dump -s symbolfile -h bar 
  • Now remove all *jdb files from symbol table and copy in files from bar

In case that the restoration of the symbol file still fails,  follow the steps below to use the backup (daily or weekly):

Step 1: Stop the server

Step 2: Rename the symboltable directory

Step 3: Create a new symboltable directory and copy all the files with the same timestamp in the filename to the symboltable directory E.g. in a backup, we could end up having files from multiple backups, like 00000000_2009_4_19_0.jdb and 00000000_2009_3_19_0.jdb. Use the latest ones that have 2009_4_19_0 in the name and remove all the others that have 2009_3_19_0 in the name

Step 4: Rename the PathedSymbolTable.db. E.g. if it is named PathedSymbolTable_2009_4_19_0.db, rename it to PathedSymbolTable.db

Step 5: To rename the *.jdb files (timestamp must not be included in the filename), use the following perl script on Unix. For windows, refer to rename.pl:

@files = <*>; foreach $file (@files) {if($file =~ /(.*)_2.*\.jdb/) { # print”$1.jdb\n”; rename($file,”$1.jdb”); } }

Unix:

for i in `ls *.jdb` do
  $F=`cut -f 1 -d'_' $i'
  mv $i $F
done

Step 6: Run this script inside the symboltable directory with perl -w . You can download perl from activestate.com > activeperl > downloads

Step 7: After all files are in the form .jdb and free of the timestamp (like 000003f0.jdb), restart the server.

2. How does the HP Diagnostics Collector get data?

I got complaints from some users that Collector collects the data from Vendor published API’s and reports to the Diagnostics Server. But the question is how does HP Diagnostics Collector get data? Is this something to be taken seriously? 

Solution

The HP Diagnostic collector collects data from vendor published external API’s. Oracle Collector reads the oracle external interface , MQ Collector reads the MQ external interface. In this way the HP Diagnostics Collector gets data. So, metrics that are reported by the collectors are 100% vendor supplied. Hence, we advise you to be careful with this.

3. Is it possible to change Diagnostics profile database?

In this document you will get information about BSM’s database i.e. the BSM’s database cannot be targeted with modification of the configuration of the Database of Diagnostics. When configuring the integration between BSM and Diagnostics, it is possible that the user needs to have the data from Diagnostics into the BSM database.

Solution

After comparing BSM and Diagnostics, they contain different types of databases and to directly save data from Diagnostics into a BSM database is not possible. 

It is possible to fill a BSM database with Diagnostics data, the integration is defined to be able to see data into it.

There is a chance to only select to export data to an external database or connect a probe to BSM server to pick the data, but it is not possible to modify the target BSM database or tables that are involved in the integration with Diagnostics.

The different tables from BSM are dedicated to specific data, and to modify and/ or add tables may affect the rest of data from BSM causing unexpected behaviour. 

4. What are the points files used by Java Probe for Instrumentation?

The java probe uses the default points file for Instrumentation.

Solution

There’s nothing to do about this because it is by default. Java Probe uses 2 points file

1. The auto_detect.points file 

2. The <probename>.points file.

5. How to enable long query logging in Diagnostics?

Long queries are queries that execute for longer than a specified duration. By default this duration is one (1) second in Diagnostics. Long queries logging is a new feature available in Diagnostics 9.26 IP1 or later. A new feature is provided in the HP Diagnostics 9.26 IP1 release to allow for the logging of long queries.

Solution

Any long queries will be logged in the log named “long_queries.log” in the Diagnostics Server log folder located at:

<MercuryDiagnosticsInstall>\Server\log (e.g. C:\MercuryDiagnostics\Server\log)

This duration is specified in milliseconds (ms) using the attribute:

query.delay.to_log

in the “webserver.properties” configuration file located at:

<MercuryDiagnosticsInstall>\Server\etc (e.g. C:\MercuryDiagnostics\Server\etc)

For example a duration of two (2) seconds would be specified as follow:

query.delay.to_log=2000

The default duration specified in the “webserver.properties” configuration file is one (1) second, that is, 1000 ms as follows:

query.delay.to_log=1000

6. Can I collect CPU, Memory and Network from an Oracle Database using “Oracle Collector”?

I want to know whether they can collect CPU, Memory and Network from an Oracle Database using “Oracle Collector”? Also, Can I use HP Diagnostics to collect CPU, Memory and Network from an Oracle Database using “Oracle Collector”?

Solution

You can check C:\\MercuryDiagnostics\\Collector\\probe\\etc\\metrics.xml

for a list of all metrics. Here you will get the answer to all your questions.

7. Profiler does not show

I have installed Java Agent 9.21. It shows available and working in Diagnostics Server but profiler View does not show when using Internet Explorer Language as ‘fr-FR’.

We tried clicking on ‘Show profiler’ which launched a new page , but it still hung on ‘wait status’ during the connection to the profiler. In the probe’s logs there are no errors reported.

Solution

This issue is caused by a bug in Diagnostics when the ‘fr-FR’ language in Internet Explorer is used.

In order to verify this is the cause of the problem, user may try to add it to English instead in:

Internet Explorer > Tools > Options > General > Languages and move it on top to set as default.

After restarting Explorer, if Profiler shows, it is possible to apply a patch to solve this situation and keep Internet Explorer. Support may provide patch ‘HPDiagServer_9.21.147.1634 to resolve this issue and keep ‘fr-FR’ as language in Internet Explorer.

8. HPD Mediator Server – High Disk Utilization – Clean required to clear to space

I need help with the Archive folder. It is getting bigger. I need to understand the purging Policy.

Solution 

persistency.major.0.minor.2 corresponds to one hour data which will be (re)aggregated to create the data in “Days” folder. The retention period for this data is set at 14 days (336) hours to facilitate the reaggregation activity which is done in the background and can need additional time. Sometimes (rarely) a corrupted zip file in the archive can be deleted and the product will recreate the zip file if the source files for the lower (“previous”) major are still available.

In this case the setting “persistency.major.0.minor.2.retention” overrides the setting “persistency.major.0.retention” for 1h data.

A couple of other points to note:

Point 1) Purging of data in the archive will not occur until the configured persistence.purging.threshold is reached. So you may see data files present in the archive that you would expect to be deleted.

Point 2) When the threshold is reached the purger will delete the first available files it finds which in all cases will not be the oldest files. This may be confusing.

Point 3) The threshold can be reduced to “force” purging but if it is too small then the archive will still grow as per the retention policy. This may also be confusing.

Point 4) Trying to adjust the retention policy is risky and inadvisable. Better to allocate enough storage for the probes attached to a Mediator – in my experience 1GB per probe is often enough even though the guide states 4GB. Storage required per probe will depend on the nature of application and how active it is.

9. Diag Mediator will not start – SEVERE archive : Unable to create archive

I found archive errors in log shows. This is because the Diag 9.24 Mediator server attempted to restart the Mediator on Sept 04 on LINUX RH 5.10. The mediator repeats the error and restart attempt even after trying for several times.

I was forced to shut down and leave. This was *not* a recent upgrade, it has been 9.,24 for a couple years, so that 1 log entry suggestion is not applicable. May the archive file or directory be corrupt? Is there a specific file I can delete to get past this issue?

Also just extract of repeating errors ==2018-09-12 13:27:54,456: SEVERE archive: Unable to initialize local symbol table 2018-09-12 13:27:54,456: SEVERE archive: If you recently upgrade HP Diagnostics from 9.23 or earlier to 9.24 or later

2018-09-12 13:27:54,456: SEVERE archive: You need to run sleepycat_upgrade.cmd(sh) under the bin directory before starting the server.
2018-09-12 13:27:54,457: SEVERE archive: Please check Diagnostics_Upgrade_Patch_Install_Instructions.pdf for more detailscom.sleepycat.je.EnvironmentFailureException: (JE 6.0.11) /qual/qophma1/diagnostics/Server/./archive/<FQDN>/symboltable fetchTarget of 0x2df4e/0x10c2a4 parent IN=19275446 IN class=com.sleepycat.je.tree.IN lastFullVersion=0x2ef03/0x4c7082 lastLoggedVersion=0x2ef03/0x4c7082 parent.getDirty()=false state=0 LOG_FILE_NOT_FOUND: Log file missing, log is likely invalid. Environment is invalid and must be closed.Caused by: java.io.FileNotFoundException: /qual/qophma1/diagnostics/Server/./archive/<FQDN>/0002df4e.jdb (No such file or directory) at java.io.RandomAccessFile.open(Native Method)
2018-09-12 13:27:54,475: SEVERE archive: Unable to create archivecom.sleepycat.je.EnvironmentFailureException: (JE 6.0.11) /qual/qophma1/diagnostics/Server/./archive/<FQDN>/symboltable fetchTarget of 0x2df4e/0x10c2a4 parent IN=19275446 IN class=com.sleepycat.je.tree.IN lastFullVersion=0x2ef03/0x4c7082 lastLoggedVersion=0x2ef03/0x4c7082 parent.getDirty()=false state=0 LOG_FILE_NOT_FOUND: Log file missing, log is likely invalid. Environment is invalid and must be closed. Caused by: java.io.FileNotFoundException: /qual/qophma1/diagnostics/Server/./archive<FQDN>symboltable/0002df4e.jdb (No such file or directory) at java.io.RandomAccessFile.open(Native Method)==

Solution

The possible solution for these kinds of errors is to rename the archive. If you do want to keep the historical data, perform below-written steps:

Symbols recovery procedure:

Step 1: Back up the whole /archive up immediately.

Step 2: Make sure the server you are restoring the symbols into is the exact same version as the original one.

Step 3: Make sure the reason why the symbols went corrupt has already been resolved (OOM, OOD, crash, etc).

Step 4: Increase the heap to the safe maximum possible.

Step 5: Set the following in the server.properties file:

+probe.event.listener.disabled=true

+symboltable.backup = false

+persistence.purging.threshold = 500G

+persistence.disable.purging=true

+persistence.disable.reaggregation=true

+symboltable.run.cleaner.on.startup = false

Step 6: Restore the /days symbol table back-up

Step 7: Remove the date in the name of the symbols files:

@files = <*>; foreach $file (@files) {if($file =~ /(.*)_2.*\.jdb/) { # print”$1.jdb\n”; rename($file,”$1.jdb”); } }

Step 8: Start the service

Step 9: If this doesn’t work, do a DBDump/DBLoad:

cd lib

mkdir foo

mkdir bar

9.23:

java -cp “.;je.jar” com.sleepycat.je.util.DbDump -v -h ../archive/mediator-/symboltable -d foo -s localsymboltable -r

java -cp “.;je.jar” com.sleepycat.je.util.DbLoad -v -f foo/localsymboltable.dump -s localsymboltable -h bar

9.24:

java -cp “.;je-6.0.11.jar” com.sleepycat.je.util.DbDump -v -h ../archive/mediator-/symboltable -d foo -s localsymboltable -r

java -cp “.;je-6.0.11.jar” com.sleepycat.je.util.DbLoad -v -f foo/localsymboltable.dump -s localsymboltable -h bar

Note: Symbol file names should already have the date removed from their names.

Step 10: Start the service

Step 11: If issues are still observed, set symboltable.repair.corrupt.symbols=true in the server.properties and restart the server once more

Step 12: If this doesn’t work, try again from step 6 with the /weeks, /months and /years back-ups.

10. LDAP Configuration in Diagnostics

Are you facing problems in LDAP? Need not worry. I will provide you with the correct steps to configure LDAP in the Diagnostics server.

Solution

First I will take a look at how to Enable LDAP Authentication on HPE Diagnostic Tool.

Step 1: Check the Server Installation and Administrator Guide, there will be an appendix named “Configuring Diagnostics to Use JAAS”. They will talk about how I should enable JAAS authentication on the following file.

Step 2: JAAS must be enabled in the /etc/server.properties file by un- commenting the following two lines

authentication.jaas.config.file=jaas.configuration

authentication.jaas.realm=Diagnostics

Step 3: Keep in mind that after any change made in any properties file, DIAG should be restarted.

Step 4: When the Authentication is enabled, we can use the LDAP login with 2 modes (Simplified Mode and Advance Mode). On this step, I will explain the simplified mode.

Configuring LDAP logging.

Edit the Diagnostics JAAS realm (application) block in /etc/jaas.configuration with option values specific to your LDAP server.

// // Simple LDAP Example:
com.mercury.diagnostics.server.jaas.spi.LDAPLoginModule sufficient
//allowAnonymous="true"
baseDN="OU=Diag,DC=te,DC=com"
providerURL="ldap://BSM00.te.com:389"
searchFirst="true"
debug="true"
uidAttribute="CN"
;

Step 5: I can see, this file will have by default configuration from 2 modes, what we need to do is to work with just 1 module, and leave the other documented as the screenshot above.

Now Based on all the attributes of the screenshots from above, you need to start testing with them. (Keep in mind that some attributes are not there since they are not needed, such as the useSSL which you don’t have, so I can avoid it or even comment on it).

Now

JAAS configuration file

baseDN: Used to construct the principal’s DN. If anonymous searches are allowed, then it is also used to specify which base DN to search for users in. (required)

For example, my baseDN is “OU=Diag, DC=te, DC=com”, let’s take a look at my Active Directory Structure.

“te.com” is my Domain, the reason why I need to split is using DC (Domain).

“Diag” is my OU (Organizational Unit) which is assigned to my Domain (te.com)

So, you can have a baseDN as longer as you want, for example this other:

In this case the base DN will be: “OU=TestDiag, OU=Diag, DC=te, DC=com”

providerURL: URL to LDAP server, in my case the LDAP server URL with FQDN is the following

“ldap://BSM00.te.com:389”. Keep in mind that 389 is the LDAP port (non-secure) and 636 is the secure

LDAP port.

If the value is set to true and allowAnonymous is also true, then do an anonymous search for the users, otherwise, construct the user’s principal DN from the uidAttribute.In this case, we have the searchFirst value but, the allowAnonymous value is not declared, so at the end we will need to specify which will be the first user to match between Active Directory and Diagnostics, this will be specified on the uidAttribute. debug: the log level to find extra information on login logs from Diag. (testing purpose) uidAttribute: Used to construct the user principal DN. Here we will have specified which user will match.

In this case we have “CN” as value. This value is indicated on the Diag Guide.

With the searchFirst value set on true, Diagnostics will try to match the first CN (Common Name)

available on active directory, in this case all the users.

This is my User “cuadra” and my group “DiagAdmin” that belongs to the OU (Diag), so Diag will match all

the CN (Common Names) available on the OU specified on the baseDN attribute.

We can see that just with few values, LDAP communication can be established, sometimes we can have

specified deeper which attributes to use. This is a simple authentication.

After all these changes, restart the Diag server to take effect. You can create roles under Diag by following the below written steps:

Step 1: Access to Diag GUI -> Manager Authorization and Authentication -> Edit Enterprise Permissions

Step 2: You need to add the user created from Active Directory.

Step 3: After adding the user as well the permissions, save the changes. This time we will be able to logging

to Diag with the Active Directory Users.

Step 5: To test how Diag finds the specified user, take a look over the server.log (keep in mind to enable DEBUG).

You can see that the Diagnostics checks for the attributes not specified and it assigns the default values, this is how the simplified mode works perfectly.