Upgrading from 7.5 to 8.5 completing final XML restore

[Jeffrey Melies]

We are upgrading our Jira instance from 7.5.x to 8.5.x. 

The upgrade went as expected, we had several people test, made some scriptrunner changes, and now we are making arrangements to move to Production, just doing another XML restore in our soon to be new dev and prod environments so we can ensure all of our documentation steps are accurate, but now I'm getting an error:

error importing data: java.lang.outofmemoryerror: java heap space

See Atlassian-jira.log below. 

I have tried to increase memory to -Xms3072m & -Xmx6608m from 3072m, on both, I also tried -XX:MaxPermSize=768m from 512m with the same results.

Both Source and Destination databases are on the same server, but for obvious reasons different databases.

https-jsse-nio-443-exec-23 WARN anonymous 464x2599x1 pmspf9 xx.xx.xx.xx /rest/capabilities/navigation [c.a.p.n.p.navigation.services.LocalNavigationLinksImpl] Failed to gather navigation links
com.atlassian.cache.CacheException: com.atlassian.activeobjects.internal.ActiveObjectsSqlException: There was a SQL exception thrown by the Active Objects library:
Database:
 - name:Microsoft SQL Server
 - version:12.00.6024
 - minor version:0
 - major version:12
Driver:
 - name:Microsoft JDBC Driver 7.2 for SQL Server
 - version:7.2.1.0

com.microsoft.sqlserver.jdbc.SQLServerException: Invalid column name 'LAST_POLLED'.
 at com.atlassian.cache.memory.DelegatingCachedReference.get(DelegatingCachedReference.java:92)
 at com.atlassian.jira.plugins.dvcs.dao.impl.CachingOrganizationDaoImpl.getAllCachedOrgs(CachingOrganizationDaoImpl.java:211)
 at com.atlassian.jira.plugins.dvcs.dao.impl.CachingOrganizationDaoImpl.getAllByType(CachingOrganizationDaoImpl.java:83)
 at com.atlassian.jira.plugins.dvcs.service.OrganizationServiceImpl.getAll(OrganizationServiceImpl.java:112)
 at com.atlassian.jira.plugins.dvcs.navlinks.BitbucketCloudAccountNavLinkRepository.matching(BitbucketCloudAccountNavLinkRepository.java:86)
 at com.atlassian.jira.plugins.dvcs.navlinks.BitbucketCloudAccountNavLinkRepository.all(BitbucketCloudAccountNavLinkRepository.java:80)
 at com.atlassian.plugins.navlink.producer.navigation.services.LocalNavigationLinksImpl$1.apply(LocalNavigationLinksImpl.java:44)
 at com.atlassian.plugins.navlink.producer.navigation.services.LocalNavigationLinksImpl$1.apply(LocalNavigationLinksImpl.java:39)
 at com.google.common.collect.Lists$TransformingRandomAccessList$1.transform(Lists.java:616)
 at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47)
 at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47)
 at com.google.common.collect.Iterators$ConcatenatedIterator.hasNext(Iterators.java:1332)
 at com.google.common.collect.Iterators.addAll(Iterators.java:357)
 at com.google.common.collect.Sets.newHashSet(Sets.java:276)
 at com.google.common.collect.Sets.newHashSet(Sets.java:259)
 at com.atlassian.plugins.navlink.producer.navigation.services.LocalNavigationLinksImpl.all(LocalNavigationLinksImpl.java:34)
 at com.atlassian.plugins.navlink.producer.navigation.services.LocalNavigationLinkServiceImpl.all(LocalNavigationLinkServiceImpl.java:40)
 at com.atlassian.plugins.navlink.producer.navigation.rest.NavigationServlet.createContext(NavigationServlet.java:94)
 at com.atlassian.plugins.navlink.producer.navigation.rest.NavigationServlet.doGet(NavigationServlet.java:78)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
 at com.atlassian.plugin.servlet.DelegatingPluginServlet.service(DelegatingPluginServlet.java:37)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
 at com.atlassian.plugin.servlet.ServletModuleContainerServlet.service(ServletModuleContainerServlet.java:46)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
 ... 44 filtered
 at com.atlassian.diagnostics.internal.platform.monitor.http.HttpRequestMonitoringFilter.doFilter(HttpRequestMonitoringFilter.java:55)
 ... 3 filtered
 at com.atlassian.web.servlet.plugin.request.RedirectInterceptingFilter.doFilter(RedirectInterceptingFilter.java:21)
 ... 41 filtered
 at com.atlassian.jira.security.JiraSecurityFilter.lambda$doFilter$0(JiraSecurityFilter.java:66)
 ... 1 filtered
 at com.atlassian.jira.security.JiraSecurityFilter.doFilter(JiraSecurityFilter.java:64)
 ... 20 filtered
 at com.atlassian.jira.servermetrics.CorrelationIdPopulatorFilter.doFilter(CorrelationIdPopulatorFilter.java:30)
 ... 12 filtered
 at com.atlassian.jira.servermetrics.MetricsCollectorFilter.doFilter(MetricsCollectorFilter.java:25)
 ... 7 filtered
 at com.atlassian.plugins.navlink.producer.ForwardServletFilter.redirectToTarget(ForwardServletFilter.java:45)
 at com.atlassian.plugins.navlink.producer.ForwardServletFilter.doFilter(ForwardServletFilter.java:35)
 ... 3 filtered
 at com.atlassian.web.servlet.plugin.request.RedirectInterceptingFilter.doFilter(RedirectInterceptingFilter.java:21)
 ... 57 filtered
 at com.atlassian.jira.security.JiraSecurityFilter.lambda$doFilter$0(JiraSecurityFilter.java:66)
 ... 1 filtered
 at com.atlassian.jira.security.JiraSecurityFilter.doFilter(JiraSecurityFilter.java:64)
 ... 16 filtered
 at com.atlassian.plugins.rest.module.servlet.RestSeraphFilter.doFilter(RestSeraphFilter.java:37)
 ... 19 filtered
 at com.atlassian.jira.servermetrics.CorrelationIdPopulatorFilter.doFilter(CorrelationIdPopulatorFilter.java:30)
 ... 10 filtered
 at com.atlassian.web.servlet.plugin.request.RedirectInterceptingFilter.doFilter(RedirectInterceptingFilter.java:21)
 ... 4 filtered
 at com.atlassian.web.servlet.plugin.LocationCleanerFilter.doFilter(LocationCleanerFilter.java:36)
 ... 26 filtered
 at com.atlassian.jira.servermetrics.MetricsCollectorFilter.doFilter(MetricsCollectorFilter.java:25)
 ... 25 filtered
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
 at java.lang.Thread.run(Thread.java:748)
Caused by: com.atlassian.activeobjects.internal.ActiveObjectsSqlException: There was a SQL exception thrown by the Active Objects library:
Database:
 - name:Microsoft SQL Server
 - version:12.00.6024
 - minor version:0
 - major version:12
 - name:Microsoft JDBC Driver 7.2 for SQL Server
 - version:7.2.1.0

com.microsoft.sqlserver.jdbc.SQLServerException: Invalid column name 'LAST_POLLED'.
 at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.find(EntityManagedActiveObjects.java:142)
 at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects.find(TenantAwareActiveObjects.java:296)
 ... 2 filtered
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
 at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
 at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
 at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
 at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
 at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
 at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
 at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
 at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
 at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
 at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
 at com.sun.proxy.$Proxy2987.find(Unknown Source)
 at com.atlassian.jira.plugins.dvcs.dao.impl.OrganizationAOFacadeImpl.lambda$fetch$0(OrganizationAOFacadeImpl.java:130)
 at com.atlassian.sal.core.transaction.HostContextTransactionTemplate$1.doInTransaction(HostContextTransactionTemplate.java:21)
 at com.atlassian.jira.DefaultHostContextAccessor.doInTransaction(DefaultHostContextAccessor.java:50)
 ... 2 filtered
 at java.lang.reflect.Method.invoke(Method.java:498)
 at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26)
 at com.sun.proxy.$Proxy441.doInTransaction(Unknown Source)
 ... 2 filtered
 at java.lang.reflect.Method.invoke(Method.java:498)
 at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:131)
 at com.sun.proxy.$Proxy441.doInTransaction(Unknown Source)
 at com.atlassian.sal.core.transaction.HostContextTransactionTemplate.execute(HostContextTransactionTemplate.java:18)
 ... 2 filtered
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
 at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
 at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
 at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
 at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
 at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
 at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
 at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
 at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
 at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
 at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
 at com.sun.proxy.$Proxy1055.execute(Unknown Source)
 at com.atlassian.activeobjects.internal.SalTransactionManager.inTransaction(SalTransactionManager.java:42)
 at com.atlassian.activeobjects.internal.AbstractLoggingTransactionManager.doInTransaction(AbstractLoggingTransactionManager.java:16)
 at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.executeInTransaction(EntityManagedActiveObjects.java:204)
 at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects.executeInTransaction(TenantAwareActiveObjects.java:336)
 ... 2 filtered
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
 at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
 at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
 at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
 at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
 at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
 at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
 at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
 at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
 at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
 at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
 at com.sun.proxy.$Proxy2987.executeInTransaction(Unknown Source)
 at com.atlassian.jira.plugins.dvcs.dao.impl.OrganizationAOFacadeImpl.fetch(OrganizationAOFacadeImpl.java:129)
 at com.atlassian.cache.memory.MemoryCacheManager$1.load(MemoryCacheManager.java:120)
 at com.atlassian.cache.memory.MemoryCacheManager$1.load(MemoryCacheManager.java:96)
 at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
 at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
 at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
 at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
 at com.google.common.cache.LocalCache.get(LocalCache.java:3952)
 at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
 at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
 at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4964)
 at com.atlassian.cache.memory.DelegatingCachedReference.getUnderLock(DelegatingCachedReference.java:102)
 at com.atlassian.cache.memory.DelegatingCachedReference.get(DelegatingCachedReference.java:87)
 ... 333 more
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Invalid column name 'LAST_POLLED'.
 at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:256)
 at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1621)
 at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:592)
 at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:522)
 at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7194)
 at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:248)
 at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:223)
 at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeQuery(SQLServerPreparedStatement.java:444)
 at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83)
 at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83)
 at com.atlassian.jira.ofbiz.sql.PreparedStatementWrapper.executeQuery(PreparedStatementWrapper.java:42)
 at com.atlassian.jira.diagnostic.connection.DiagnosticPreparedStatement.lambda$executeQuery$5(DiagnosticPreparedStatement.java:59)
 at com.atlassian.diagnostics.internal.platform.monitor.db.DefaultDatabaseDiagnosticsCollector.recordExecutionTime(DefaultDatabaseDiagnosticsCollector.java:69)
 at com.atlassian.jira.diagnostic.connection.DatabaseDiagnosticsCollectorDelegate.recordExecutionTime(DatabaseDiagnosticsCollectorDelegate.java:55)
 at com.atlassian.jira.diagnostic.connection.DiagnosticPreparedStatement.executeQuery(DiagnosticPreparedStatement.java:59)
 at net.java.ao.EntityManager.find(EntityManager.java:671)
 at net.java.ao.EntityManager.find(EntityManager.java:621)
 at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.find(EntityManagedActiveObjects.java:140)
 ... 425 more
2020-02-13 07:44:43,087 automation-queue-claimer:thread-1 WARN      [c.c.j.p.automation.queue.JiraAutomationQueueExecutor] JQL search indexes are currently not available (Indexing: true, Scheduler: SHUTDOWN). No automation items were claimed from the queue for processing on this node since most automation rules depend on JQL indexes being available.