After migrating from Oracle JAVA to OpenJDK, our custom Atlassian SDK plug-in not working

[Devendar Gangapuram]

Hi Team,

Recently, we have migrated JIRA and BAMBOO from Oracle JAVA to OpenJDK and these two servers are up and running fine. But we have Atlassian JIRA SDK plug-in installed before migration, this plug-in was working fine fine but now it is not. Some one please explain me what could be the issue. 

Thank you in advance,

Dev G

[Nic Brough -Adaptavist-]

Your app may have used features in Java that are used slightly differently in the new implementation.  I've seen this happen with different versions of oracle java and when running the same version of it on Windows and Linux, so it might not be the swap of JDK.

The solution in my case was a slight change in the version of Java once, but the other couple of times, simply recompiling my app using the target version of Java solved it.

[Devendar Gangapuram]

Thank you @Nic Brough -Adaptavist- ,

I have tried recompiling but this time gives me different error like below...

Logs:-

count":0,"min":0,"max":0,"sum":0,"avg":0,"distributionCounter":{}}}, index writer version cache stats: {"put":0,"get":0,"getFound":0,"clear":0}
2021-11-23 00:47:10,191+0100 plugin-transaction-0 INFO [c.a.jira.plugin.PluginTransactionListener] [plugin-transaction] numberStartEvents:715, numberEndEvents:715, numberSendEvents:496, numberEventsInTransactions:13120, numberOfPluginEnableEvents:279
2021-11-23 00:48:00,492+0100 Caesium-1-4 ERROR ServiceRunner [c.a.s.caesium.impl.SchedulerQueueWorker] Unhandled exception thrown by job QueuedJob[jobId=com.atlassian.analytics.client.upload.RemoteFilterRead:job,deadline=1637624880000]
javax.xml.stream.FactoryConfigurationError: Provider for class javax.xml.stream.XMLInputFactory cannot be created
at javax.xml.stream.FactoryFinder.findServiceProvider(FactoryFinder.java:370)
at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:313)
at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:227)
at javax.xml.stream.XMLInputFactory.newInstance(XMLInputFactory.java:154)
at com.amazonaws.util.XmlUtils.createXmlInputFactory(XmlUtils.java:62)
at com.amazonaws.util.XmlUtils.access$000(XmlUtils.java:27)
at com.amazonaws.util.XmlUtils$1.initialValue(XmlUtils.java:35)
at com.amazonaws.util.XmlUtils$1.initialValue(XmlUtils.java:32)
at java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:180)
at java.lang.ThreadLocal.get(ThreadLocal.java:170)
at com.amazonaws.util.XmlUtils.getXmlInputFactory(XmlUtils.java:53)
at com.amazonaws.services.s3.internal.S3ErrorResponseHandler.createException(S3ErrorResponseHandler.java:93)
at com.amazonaws.services.s3.internal.S3ErrorResponseHandler.handle(S3ErrorResponseHandler.java:64)
at com.amazonaws.services.s3.internal.S3ErrorResponseHandler.handle(S3ErrorResponseHandler.java:51)
at com.amazonaws.http.AwsErrorResponseHandler.handleAse(AwsErrorResponseHandler.java:50)
at com.amazonaws.http.AwsErrorResponseHandler.handle(AwsErrorResponseHandler.java:38)
at com.amazonaws.http.AwsErrorResponseHandler.handle(AwsErrorResponseHandler.java:24)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1612)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1302)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4365)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4312)
at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1415)
at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1276)
at com.atlassian.analytics.client.s3.AnalyticsS3Client.getS3ObjectInputStream(AnalyticsS3Client.java:110)
at com.atlassian.analytics.client.eventfilter.reader.RemoteListReader.getRemoteListObject(RemoteListReader.java:29)
at com.atlassian.analytics.client.eventfilter.reader.RemoteListReader.readFilterList(RemoteListReader.java:25)
at com.atlassian.analytics.client.eventfilter.parser.SimpleListParser.readSimpleFilterList(SimpleListParser.java:46)
at com.atlassian.analytics.client.eventfilter.BlacklistFilter.readBlacklist(BlacklistFilter.java:54)
at com.atlassian.analytics.client.eventfilter.BlacklistFilter.readRemoteList(BlacklistFilter.java:46)
at com.atlassian.analytics.client.upload.RemoteFilterRead.runJob(RemoteFilterRead.java:56)
at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:134)
at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:106)
at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:90)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:435)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeLocalJob(CaesiumSchedulerService.java:402)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:380)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:66)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:60)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:35)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Provider for class javax.xml.stream.XMLInputFactory cannot be created
at javax.xml.stream.FactoryFinder.findServiceProvider(FactoryFinder.java:367)
... 46 more
Caused by: java.util.ServiceConfigurationError: javax.xml.stream.XMLInputFactory: Provider com.ctc.wstx.stax.WstxInputFactory not found
at java.util.ServiceLoader.fail(ServiceLoader.java:239)
at java.util.ServiceLoader.access$300(ServiceLoader.java:185)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at javax.xml.stream.FactoryFinder$1.run(FactoryFinder.java:353)
at java.security.AccessController.doPrivileged(Native Method)
at javax.xml.s

Please check once and let me know your comments please.

Thank you,

Dev G

[Devendar Gangapuram]

Hi @Nic Brough -Adaptavist- ,

After migrating from Oracle JAVA to OpenJDK, getting this below  error...

Caused by: java.util.ServiceConfigurationError: javax.xml.stream.XMLInputFactory: Provider com.ctc.wstx.stax.WstxInputFactory not found

Please have a look and help me out to fix this issue as soon as possible. 

Thanks,

 Dev G

[Nic Brough -Adaptavist-]

You'll need to include the missing library in your build, or refactor so that you do it a different way with a library that is suitable for your new version of Java

[Devendar Gangapuram]

Thank you so much for the quick reply Nic,

Unfortunately, I didn't work even after including the dependencies in my POM.xml. I have tried placing with two different decencies like mention in the image but no luck. Please let me know what is the correct dependency to fix this below error. 

ServiceConfigurationError: javax.xml.stream.XMLInputFactory: Provider com.ctc.wstx.stax.WstxInputFactory not found

------------------------------------------

Please check below logs...

message=[INDEX-FIXER] Re-indexing issues modified in the last {1 days, 23 hours, 31 minutes, and 16 seconds}. (Versioning short-circuit checks are enabled.)
2021-11-25 07:28:54,762+0100 Caesium-1-4 ERROR [c.a.s.caesium.impl.SchedulerQueueWorker] Unhandled exception thrown by job QueuedJob[jobId=com.atlassian.analytics.client.upload.RemoteFilterRead:job,deadline=1637821732137]
javax.xml.stream.FactoryConfigurationError: Provider for class javax.xml.stream.XMLInputFactory cannot be created
at javax.xml.stream.FactoryFinder.findServiceProvider(FactoryFinder.java:370)
at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:313)
at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:227)
at javax.xml.stream.XMLInputFactory.newInstance(XMLInputFactory.java:154)
at com.amazonaws.util.XmlUtils.createXmlInputFactory(XmlUtils.java:62)
at com.amazonaws.util.XmlUtils.access$000(XmlUtils.java:27)
at com.amazonaws.util.XmlUtils$1.initialValue(XmlUtils.java:35)
at com.amazonaws.util.XmlUtils$1.initialValue(XmlUtils.java:32)
at java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:180)
at java.lang.ThreadLocal.get(ThreadLocal.java:170)
at com.amazonaws.util.XmlUtils.getXmlInputFactory(XmlUtils.java:53)
at com.amazonaws.services.s3.internal.S3ErrorResponseHandler.createException(S3ErrorResponseHandler.java:93)
at com.amazonaws.services.s3.internal.S3ErrorResponseHandler.handle(S3ErrorResponseHandler.java:64)
at com.amazonaws.services.s3.internal.S3ErrorRespons

message=[INDEX-FIXER] Re-indexing issues modified in the last {1 days, 23 hours, 31 minutes, and 16 seconds}. (Versioning short-circuit checks are enabled.)
2021-11-25 07:28:54,762+0100 Caesium-1-4 ERROR [c.a.s.caesium.impl.SchedulerQueueWorker] Unhandled exception thrown by job QueuedJob[jobId=com.atlassian.analytics.client.upload.RemoteFilterRead:job,deadline=1637821732137]
javax.xml.stream.FactoryConfigurationError: Provider for class javax.xml.stream.XMLInputFactory cannot be created
at javax.xml.stream.FactoryFinder.findServiceProvider(FactoryFinder.java:370)
at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:313)
at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:227)
at javax.xml.stream.XMLInputFactory.newInstance(XMLInputFactory.java:154)
at com.amazonaws.util.XmlUtils.createXmlInputFactory(XmlUtils.java:62)
at com.amazonaws.util.XmlUtils.access$000(XmlUtils.java:27)
at com.amazonaws.util.XmlUtils$1.initialValue(XmlUtils.java:35)
at com.amazonaws.util.XmlUtils$1.initialValue(XmlUtils.java:32)
at java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:180)
at java.lang.ThreadLocal.get(ThreadLocal.java:170)
at com.amazonaws.util.XmlUtils.getXmlInputFactory(XmlUtils.java:53)
at com.amazonaws.services.s3.internal.S3ErrorResponseHandler.createException(S3ErrorResponseHandler.java:93)
at com.amazonaws.services.s3.internal.S3ErrorResponseHandler.handle(S3ErrorResponseHandler.java:64)
at com.amazonaws.services.s3.internal.S3ErrorRespons

--------------

c.a.activeobjects.osgi.ActiveObjectsServiceFactory] getService bundle [com.atlassian.whisper.atlassian-whisper-plugin]
2021-11-25 07:31:14,286+0100 https-openssl-nio-443-exec-7 ERROR JiraAdmins 451x22x1 p636k5 10.43.136.105 /secure/Dashboard.jspa [c.a.g.r.internal.http.HttpClientFetcher] Unable to perform a request to: https://10.45.137.111/rest/gadgets/1.0/g/messagebundle/und/gadget.common%2Cgadget.issuetable%2Cgadget.assignedtome%2Cgadget.issuetable.common
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alert.createSSLException(Alert.java:131)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
a

--------------------------------------

Thank you,

Dev G

[Devendar Gangapuram]

Hi @Nic Brough -Adaptavist- ,

I have just tried roll backing from Open JDK to Oracle JAVA and made necessary changes but getting same errors mentioned in the earlier reply, which means now in both cases(Oracle JAVA & Open JDK) it gives same errors. Could you please check and let me know where I am doing mistake. Quick reply would be appreciated please.

Thanks you,

Dev G

[Devendar Gangapuram]

@Nic Brough -Adaptavist- ,

JIRA server is starting fine but in server logs it shows below error...

Logs:-

INFO      [c.a.jira.upgrade.UpgradeIndexManager] Reindexing is not allowed after this upgrade and there is no immediate reindex requests

java.lang.NullPointerException

[Devendar Gangapuram]

My JIRA server shows below information. Can I perform this re-index process?

[Nic Brough -Adaptavist-]

Yes, you need to, because something has damaged your index beyond repair, and onlt a "full re-index" can fix it (a full re-index deletes the whole index and builds it again from scratch).

The warning message there is just telling you why the background index won't cut it.

[Devendar Gangapuram]

Thank you so much @Nic Brough -Adaptavist-  for the fast reply. 

Do we need to take any back up before proceed this Re-index? also will it impact on our current  JIRA Data?

Thank you,

Dev G

[Devendar Gangapuram]

Hi @Nic Brough -Adaptavist- ,

I have performed the reindex and after that it gives below handshake errors . 

       at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)

        at java.lang.Thread.run(Thread.java:748)

Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

        at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)

        at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)

        at sun.security.validator.Validator.validate(Validator.java:271)

        at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:312)

        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:221)

        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:128)

        at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:636)

        ... 304 more

Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

        at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)

        at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)

        at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)

        at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)

        ... 310 more

java.lang.NullPointerException

        at com.dxc.jira.listener.WorkflowListener.onIssueEvent(WorkflowListener.java:202)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:42)

        at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.lambda$null$0(AsynchronousAbleEventDispatcher.java:37)

        at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.dispatch(AsynchronousAbleEventDispatcher.java:85)

        at com.atlassian.event.internal.LockFreeEventPublisher$Publisher.dispatch(LockFreeEventPublisher.java:220)

        at com.atlassian.event.internal.LockFreeEventPublisher.publish(LockFreeEventPublisher.java:96)

[Nic Brough -Adaptavist-]

Sorry, I've been distracted, and missed your follow up questions

On the indexing - no, the index is a copy of the data in your database (mostly issue data, but some other bits are in there too) in a very different shape, and a full re-index is effectively just building a new copy of it.  There's nothing to back up, as the re-index is just reading data out of the database to create a new index with.  You could take a backup of the current existing index, but the whole point of your re-index run is to create a new valid one, and it's very unlikely that you'd want to roll back to a known-broken index!

And on your new error,  that is an error to do with your certificates, not your code.

The usual culprit is simply a missing or expired certificate, but it can also be down to the certificate authority.

Check the SSL certs you've got installed first - are they the right ones?  And valid?