dgidx: error while loading shared libraries: libaio.so.1: cannot open shared object file: No such file or directory

Problem:

When I tried to fire up a baseline update on my CRS Endeca application, I got the following error in my Dgidx.*.log file inside <ENDECA_BASE>/apps/CRS/logs/dgidx

dgidx: error while loading shared libraries: libaio.so.1: cannot open shared object file: No such file or directory

Solution:

First as soon as I saw this I tried to install the missing library libaio.so using my package manage (YUM on Oracle Linux)

yum install -y libaio.so

That definitely installed something but the error doesnt seem to have gone yet because I should have installed libaio as a package as against just installing the library file with a .so extension, so tried the following again and the baselineUpdate was able to get past the indexing process to deploying the index to my Authoring and Live MDEXes

yum install -y libaio

WARNING: Unable to retrieve host 'ITLHost' associated with custom component

Problem:

When I was trying to deploy the CRS app that come with ATG11.1 on my linux box I got the following warning

WARNING: Unable to retrieve host 'ITLHost' associated with custom component 

Solution:

On looking at the Appconfig.xml and related configuration files, I found out that the warning occurred because the definition of ITLHost was happening in DataIngest.xml but it was used in WorkbenchConfig.xml which was imported before DataIngest.xml in Appconfig.xml, see below the CRS Default definition of Appconfig.xml 

  <spr:import resource="InitialSetup.xml" />
  <spr:import resource="DgraphDefaults.xml" />
  <spr:import resource="AuthoringDgraphCluster.xml" />
  <spr:import resource="LiveDgraphCluster.xml" />
  <spr:import resource="LiveAppServerCluster.xml" />
  <spr:import resource="UsageCollectionConfig.xml" />
  <spr:import resource="WorkbenchConfig.xml" />
  <spr:import resource="ReportGeneration.xml" />
  <spr:import resource="DataIngest.xml" />

  <spr:import resource="AssemblerConfig.xml" />

So changing the order of imports as follows, fixed that warning, when I tried to a new endeca deploy. See updated Appconfig.xml below..

  <spr:import resource="InitialSetup.xml" />
  <spr:import resource="DgraphDefaults.xml" />
  <spr:import resource="AuthoringDgraphCluster.xml" />
  <spr:import resource="LiveDgraphCluster.xml" />
  <spr:import resource="LiveAppServerCluster.xml" />
  <spr:import resource="UsageCollectionConfig.xml" />

  <spr:import resource="DataIngest.xml" />

  <spr:import resource="WorkbenchConfig.xml" />
  <spr:import resource="ReportGeneration.xml" />
  <spr:import resource="AssemblerConfig.xml" />

Error During BCC Deployment :: /atg/dynamo/service/jdbc/JTDataSource_production :: java.lang.IllegalStateException: Cannot delist resource, transaction has been rolled back :: Transaction timed out after 31 seconds

Problem:

I was trying to do a full deployment after setting up CA and got this error. I was using ATG 11.1 on Weblogic 12c connecting to a Oracle 12c Pluggable Database (with ojdbc7.jar) - used FakeXA as against WeblogicDS

**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production --- java.lang.IllegalStateException: Cannot delist resource, transaction has been rolled back. BEA1-346CAD26CDE45A7A06F2. xid=BEA1-346CAD26CDE45A7A06F2, status=Marked rollback. [Reason=weblogic.transaction.internal.TimedOutException: Transaction timed out after 31 seconds 
**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production BEA1-346CAD26CDE45A7A06F2]
**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production at weblogic.transaction.internal.TransactionImpl.throwIllegalStateException(TransactionImpl.java:1889)
**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production at weblogic.transaction.internal.ServerTransactionImpl.delistResource(ServerTransactionImpl.java:598)
**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production at atg.dtm.TransactionWrapper.delistResource(TransactionWrapper.java:183)
**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production at atg.service.jdbc.MonitoredDataSource.connectionClosed(MonitoredDataSource.java:1383)
**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production at atg.service.jdbc.PooledXAConnection.close(PooledXAConnection.java:329)
**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production at atg.service.jdbc.PooledMonitoredConnection.close(PooledMonitoredConnection.java:312)
**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production at atg.adapter.gsa.GSATransaction.closeConnection(GSATransaction.java:808)
**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production at atg.adapter.gsa.GSAItemDescriptor.close(GSAItemDescriptor.java:2764)
**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production at atg.adapter.gsa.GSAItemDescriptor.addItem(GSAItemDescriptor.java:7418)
**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production at atg.adapter.gsa.GSARepository.addItem(GSARepository.java:1054)
**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production at atg.deployment.repository.RepositoryWorkerThread.deployItem(RepositoryWorkerThread.java:1128)
**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production at atg.deployment.repository.RepositoryWorkerThread.processMarkerForAddUpdatePhase(RepositoryWorkerThread.java:263)
**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production at atg.deployment.DeploymentWorkerThread.processMarkerPhase(DeploymentWorkerThread.java:562)
**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production at atg.deployment.DeploymentWorkerThread.run(DeploymentWorkerThread.java:335)

**** Error Fri Apr 24 15:30:15 UTC 2015 1429889415294 /atg/dynamo/service/jdbc/JTDataSource_production

Solution:

Changed the JTA transaction timeout from 30 to 200 and the issue got resolved.