2015-08-18 23:30:48,138 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf".
2015-08-18 23:30:48,157 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties.
2015-08-18 23:30:48,157 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf".
2015-08-18 23:30:48,158 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf".
2015-08-18 23:30:48,158 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-java.conf".
2015-08-18 23:30:48,158 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf".
2015-08-18 23:30:48,158 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf".
2015-08-18 23:30:48,158 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf".
2015-08-18 23:30:48,158 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf".
2015-08-18 23:30:48,158 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_AJP_ENABLED" is "true".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_AJP_PORT" is "8702".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_DATABASE" is "engine".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_HOST" is "localhost".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_PASSWORD" is "***".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_PORT" is "5432".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_SECURED" is "False".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_DB_USER" is "engine".
2015-08-18 23:30:48,159 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_DEBUG_ADDRESS" is "".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_ETC" is "/etc/ovirt-engine".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE" is "/var/lib/ovirt-engine/external_truststore".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_PASSWORD" is "***".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_TYPE" is "JKS".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_FQDN" is "ycuihevm.redhat.com".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_GROUP" is "ovirt".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_HEAP_MAX" is "1024M".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_HEAP_MIN" is "1024M".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_HTTPS_ENABLED" is "false".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_HTTPS_PORT" is "None".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_HTTP_ENABLED" is "false".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_HTTP_PORT" is "None".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine".
2015-08-18 23:30:48,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_LOG_TO_CONSOLE" is "false".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_PERM_MAX" is "256m".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_PERM_MIN" is "256m".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_PROPERTIES" is " java.awt.headless=true sun.rmi.dgc.client.gcInterval=3600000 sun.rmi.dgc.server.gcInterval=3600000 jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_PROXY_ENABLED" is "true".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_PROXY_HTTP_PORT" is "80".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_REPORTS_BASE_URL" is "".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_REPORTS_DASHBOARD_URL" is "/flow.html?viewAsDashboardFrame=true".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_REPORTS_NOT_INSTALLED_URL" is "/ovirt-engine/ReportsNotInstalled.html".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_REPORTS_PROXY_URL" is "/ovirt/reports-interface".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_REPORTS_READ_TIMEOUT" is "".
2015-08-18 23:30:48,161 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_REPORTS_RIGHTCLICK_URL" is "".
2015-08-18 23:30:48,162 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_REPORTS_VERIFY_CHAIN" is "true".
2015-08-18 23:30:48,162 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_REPORTS_VERIFY_HOST" is "true".
2015-08-18 23:30:48,162 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_STOP_INTERVAL" is "1".
2015-08-18 23:30:48,162 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_STOP_TIME" is "10".
2015-08-18 23:30:48,162 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine".
2015-08-18 23:30:48,162 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up".
2015-08-18 23:30:48,162 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_URI" is "/ovirt-engine".
2015-08-18 23:30:48,162 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_USER" is "ovirt".
2015-08-18 23:30:48,162 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine".
2015-08-18 23:30:48,162 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine".
2015-08-18 23:30:48,162 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "ENGINE_VERBOSE_GC" is "false".
2015-08-18 23:30:48,162 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "JBOSS_HOME" is "/usr/share/jbossas".
2015-08-18 23:30:48,162 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime".
2015-08-18 23:30:48,162 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD,ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_PASSWORD".
2015-08-18 23:31:18,756 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-1) Start initializing Backend
2015-08-18 23:31:19,244 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ajp-/127.0.0.1:8702-1) Loading file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties
2015-08-18 23:31:20,314 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-1) Running ovirt-engine 3.5.4-1.2.el6ev
2015-08-18 23:31:20,314 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ajp-/127.0.0.1:8702-1) Start initializing dictionaries
2015-08-18 23:31:20,316 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ajp-/127.0.0.1:8702-1) Finished initializing dictionaries
2015-08-18 23:31:20,316 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Start initializing AuditLogCleanupManager
2015-08-18 23:31:20,316 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Setting audit cleanup manager to run at: 35 35 3 * * ?
2015-08-18 23:31:20,358 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Finished initializing AuditLogCleanupManager
2015-08-18 23:31:20,358 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Start initializing CommandEntityCleanupManager
2015-08-18 23:31:20,358 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Setting command entity cleanup manager to run at: 35 35 3 * * ?
2015-08-18 23:31:20,359 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Finished initializing CommandEntityCleanupManager
2015-08-18 23:31:20,360 INFO [org.ovirt.engine.core.bll.TagsDirector] (ajp-/127.0.0.1:8702-1) Start initializing TagsDirector
2015-08-18 23:31:20,365 INFO [org.ovirt.engine.core.bll.TagsDirector] (ajp-/127.0.0.1:8702-1) Tag root added to tree
2015-08-18 23:31:20,380 INFO [org.ovirt.engine.core.bll.TagsDirector] (ajp-/127.0.0.1:8702-1) Finished initializing TagsDirector
2015-08-18 23:31:20,380 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ajp-/127.0.0.1:8702-1) Start initializing IsoDomainListSyncronizer
2015-08-18 23:31:20,384 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ajp-/127.0.0.1:8702-1) Finished initializing IsoDomainListSyncronizer
2015-08-18 23:31:20,411 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-1) Completed initializing handlers
2015-08-18 23:31:20,431 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-1) Start initializing ErrorTranslatorImpl
2015-08-18 23:31:20,453 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-1) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table.
2015-08-18 23:31:20,455 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-1) Finished initializing ErrorTranslatorImpl
2015-08-18 23:31:20,455 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-1) Start initializing ErrorTranslatorImpl
2015-08-18 23:31:20,455 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-1) Finished initializing ErrorTranslatorImpl
2015-08-18 23:31:20,455 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-1) Mark incomplete jobs as UNKNOWN
2015-08-18 23:31:20,779 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ajp-/127.0.0.1:8702-1) Start initializing JobRepositoryCleanupManager
2015-08-18 23:31:20,779 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ajp-/127.0.0.1:8702-1) Finished initializing JobRepositoryCleanupManager
2015-08-18 23:31:20,779 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ajp-/127.0.0.1:8702-1) Start initializing AutoRecoveryManager
2015-08-18 23:31:20,780 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ajp-/127.0.0.1:8702-1) Finished initializing AutoRecoveryManager
2015-08-18 23:31:20,781 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ajp-/127.0.0.1:8702-1) Start initializing ExecutionMessageDirector
2015-08-18 23:31:20,782 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ajp-/127.0.0.1:8702-1) Finished initializing ExecutionMessageDirector
2015-08-18 23:31:20,831 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Loading extension 'builtin-authn-internal'
2015-08-18 23:31:20,833 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Extension 'builtin-authn-internal' loaded
2015-08-18 23:31:20,834 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Loading extension 'internal'
2015-08-18 23:31:20,835 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Extension 'internal' loaded
2015-08-18 23:31:20,843 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Initializing extension 'builtin-authn-internal'
2015-08-18 23:31:20,843 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Extension 'builtin-authn-internal' initialized
2015-08-18 23:31:20,844 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Initializing extension 'internal'
2015-08-18 23:31:20,844 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Extension 'internal' initialized
2015-08-18 23:31:20,844 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Start of enabled extensions list
2015-08-18 23:31:20,844 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true'
2015-08-18 23:31:20,844 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true'
2015-08-18 23:31:20,844 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) End of enabled extensions list
2015-08-18 23:31:20,949 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 58) Initialization of AsyncTaskManager completed successfully.
2015-08-18 23:31:20,950 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 58) Start initializing ResourceManager
2015-08-18 23:31:21,058 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 58) Finished initializing ResourceManager
2015-08-18 23:31:21,058 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 58) Initialization of OvfDataUpdater completed successfully.
2015-08-18 23:31:21,058 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 58) Start scheduling to enable vds load balancer
2015-08-18 23:31:21,058 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 58) Finished scheduling to enable vds load balancer
2015-08-18 23:31:21,058 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 58) Start HA Reservation check
2015-08-18 23:31:21,059 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 58) Finished HA Reservation check
2015-08-18 23:31:21,086 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Start initializing MacPoolManagerRanges
2015-08-18 23:31:21,108 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Finished initializing. Available MACs in pool: 256
2015-08-18 23:31:21,111 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 58) Init VM custom properties utilities
2015-08-18 23:31:21,112 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 58) Init device custom properties utilities
2015-08-18 23:31:21,114 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 58) Initializing Scheduling manager
2015-08-18 23:31:21,166 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 58) External scheduler disabled, discovery skipped
2015-08-18 23:31:21,166 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 58) Initialized Scheduling manager
2015-08-18 23:31:21,167 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 58) Initializing DWH Heart Beat
2015-08-18 23:31:21,167 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 58) DWH Heart Beat initialized
2015-08-18 23:31:30,495 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false.
2015-08-18 23:31:30,506 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: UNASSIGNED not exist in string table
2015-08-18 23:31:30,506 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table
2015-08-18 23:31:30,506 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table
2015-08-18 23:31:30,506 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table
2015-08-18 23:31:30,506 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table
2015-08-18 23:31:30,506 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table
2015-08-18 23:31:30,506 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table
2015-08-18 23:31:30,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table
2015-08-18 23:31:30,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table
2015-08-18 23:31:30,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table
2015-08-18 23:31:30,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table
2015-08-18 23:31:30,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table
2015-08-18 23:31:30,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table
2015-08-18 23:31:30,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table
2015-08-18 23:31:30,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table
2015-08-18 23:31:30,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table
2015-08-18 23:31:30,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table
2015-08-18 23:31:30,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table
2015-08-18 23:31:30,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in.
2015-08-18 23:31:42,185 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-7) [5261e282] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN
2015-08-18 23:31:42,347 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-7) [42794a13] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 80ce2151-3dbc-4945-834b-56315ca9f823 Type: VDS
2015-08-18 23:31:42,477 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-7) [42794a13] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-00000000010d value: REGISTER_VDS
, sharedLocks= ]
2015-08-18 23:31:42,493 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [42794a13] START, RemoveVdsVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823), log id: 16093e64
2015-08-18 23:31:42,494 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [42794a13] FINISH, RemoveVdsVDSCommand, log id: 16093e64
2015-08-18 23:31:42,504 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [42794a13] START, AddVdsVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823), log id: 3f7ba7e2
2015-08-18 23:31:42,504 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [42794a13] AddVds - entered , starting logic to add VDS 80ce2151-3dbc-4945-834b-56315ca9f823
2015-08-18 23:31:42,508 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [42794a13] AddVds - VDS 80ce2151-3dbc-4945-834b-56315ca9f823 was added, will try to add it to the resource manager
2015-08-18 23:31:42,510 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-7) [42794a13] Entered VdsManager constructor
2015-08-18 23:31:42,520 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-7) [42794a13] Initialize vdsBroker (rhevhtest-1.redhat.com,54,321)
2015-08-18 23:31:42,664 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-7) [42794a13] VDS 80ce2151-3dbc-4945-834b-56315ca9f823 was added to the Resource Manager
2015-08-18 23:31:42,665 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [42794a13] FINISH, AddVdsVDSCommand, log id: 3f7ba7e2
2015-08-18 23:31:42,676 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [42794a13] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host hosted_engine_1.
2015-08-18 23:31:43,010 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [42794a13] Correlation ID: 5261e282, Job ID: f8a7055a-5837-491a-aa29-336e07c4a757, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_1 was added by admin@internal.
2015-08-18 23:31:43,124 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-3) [f1cf267] Lock Acquired to object EngineLock [exclusiveLocks= key: 80ce2151-3dbc-4945-834b-56315ca9f823 value: VDS
, sharedLocks= ]
2015-08-18 23:31:43,469 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-3) [f1cf267] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: 80ce2151-3dbc-4945-834b-56315ca9f823 Type: VDS
2015-08-18 23:31:43,472 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-3) [f1cf267] Before Installation host 80ce2151-3dbc-4945-834b-56315ca9f823, hosted_engine_1
2015-08-18 23:31:43,496 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-3) [f1cf267] START, SetVdsStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 14986a48
2015-08-18 23:31:43,708 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-3) [f1cf267] FINISH, SetVdsStatusVDSCommand, log id: 14986a48
2015-08-18 23:31:43,903 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-7-thread-3) [f1cf267] Installation rhevhtest-1.redhat.com: Connected to host rhevhtest-1.redhat.com with SSH key fingerprint: be:da:9a:50:6c:89:30:f9:23:2b:16:35:4b:b1:f1:7d
2015-08-18 23:31:44,026 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-3) [f1cf267] Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Connected to host rhevhtest-1.redhat.com with SSH key fingerprint: be:da:9a:50:6c:89:30:f9:23:2b:16:35:4b:b1:f1:7d.
2015-08-18 23:31:44,045 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-7-thread-3) [f1cf267] Installation of rhevhtest-1.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar
2015-08-18 23:31:44,055 INFO [org.ovirt.engine.core.utils.archivers.tar.CachedTar] (org.ovirt.thread.pool-7-thread-3) Tarball '/var/cache/ovirt-engine/ovirt-host-deploy.tar' is missing, creating
2015-08-18 23:31:44,354 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-7-thread-3) SSH execute root@rhevhtest-1.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True'
2015-08-18 23:31:45,051 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Stage: Initializing
2015-08-18 23:31:45,148 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Stage: Initializing.
2015-08-18 23:31:45,161 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Stage: Environment setup
2015-08-18 23:31:45,242 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Stage: Environment setup.
2015-08-18 23:31:45,243 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Stage: Environment packages setup
2015-08-18 23:31:45,342 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Stage: Environment packages setup.
2015-08-18 23:31:45,342 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Stage: Programs detection
2015-08-18 23:31:45,433 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Stage: Programs detection.
2015-08-18 23:31:45,433 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Stage: Environment customization
2015-08-18 23:31:45,533 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Stage: Environment customization.
2015-08-18 23:31:45,533 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Kdump unsupported
2015-08-18 23:31:45,633 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Kdump unsupported.
2015-08-18 23:31:45,638 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Host is hypervisor
2015-08-18 23:31:45,667 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Initializing Host: hosted_engine_1
2015-08-18 23:31:45,741 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Host is hypervisor.
2015-08-18 23:31:45,845 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20150819033146-2d4zwf.log'
2015-08-18 23:31:45,966 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Logs at host located at: '/tmp/ovirt-host-deploy-20150819033146-2d4zwf.log'.
2015-08-18 23:31:45,995 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host rhevhtest-1.redhat.com reports unique id 4C4C4544-0056-4210-8032-B9C04F463358
2015-08-18 23:31:45,996 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0056-4210-8032-B9C04F463358 to Host rhevhtest-1.redhat.com
2015-08-18 23:31:46,229 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Disabling Kdump integration
2015-08-18 23:31:46,358 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Disabling Kdump integration.
2015-08-18 23:31:46,372 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Stage: Setup validation
2015-08-18 23:31:46,483 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Stage: Setup validation.
2015-08-18 23:31:46,483 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Cannot validate host name settings, reason: resolved host does not match any of the local addresses
2015-08-18 23:31:46,666 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_1 installation in progress . Cannot validate host name settings, reason: resolved host does not match any of the local addresses.
2015-08-18 23:31:46,666 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Hardware supports virtualization
2015-08-18 23:31:46,766 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Hardware supports virtualization.
2015-08-18 23:31:46,766 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Stage: Transaction setup
2015-08-18 23:31:46,899 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Stage: Transaction setup.
2015-08-18 23:31:46,900 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Stage: Misc configuration
2015-08-18 23:31:47,024 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Stage: Misc configuration.
2015-08-18 23:31:47,024 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Stage: Package installation
2015-08-18 23:31:47,149 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Stage: Package installation.
2015-08-18 23:31:48,286 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Stage: Misc configuration
2015-08-18 23:31:48,415 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Stage: Misc configuration.
2015-08-18 23:31:48,506 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Enrolling certificate
2015-08-18 23:31:48,970 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Enrolling certificate.
2015-08-18 23:31:50,246 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Stage: Transaction commit
2015-08-18 23:31:50,330 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Stage: Transaction commit.
2015-08-18 23:31:50,330 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Stage: Closing up
2015-08-18 23:31:50,463 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Stage: Closing up.
2015-08-18 23:31:55,561 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Stopping libvirtd
2015-08-18 23:31:55,651 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Stopping libvirtd.
2015-08-18 23:31:55,727 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Starting vdsm
2015-08-18 23:31:55,818 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Starting vdsm.
2015-08-18 23:32:06,982 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Stage: Pre-termination
2015-08-18 23:32:07,084 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Stage: Pre-termination.
2015-08-18 23:32:07,112 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20150818233207-rhevhtest-1.redhat.com-f1cf267.log'
2015-08-18 23:32:07,192 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20150818233207-rhevhtest-1.redhat.com-f1cf267.log'.
2015-08-18 23:32:07,385 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-1.redhat.com: Stage: Termination
2015-08-18 23:32:07,559 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: f1cf267, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_1. Stage: Termination.
2015-08-18 23:32:07,568 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-1.redhat.com/10.66.11.167
2015-08-18 23:32:07,962 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-7-thread-3) [f1cf267] Engine managed to communicate with VDSM agent on host hosted_engine_1
2015-08-18 23:32:08,375 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-7-thread-3) [f1cf267] The management network rhevm is already configured on host hosted_engine_1
2015-08-18 23:32:08,377 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-3) [f1cf267] START, SetVdsStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, status=Initializing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 2d6d2018
2015-08-18 23:32:08,499 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-3) [f1cf267] FINISH, SetVdsStatusVDSCommand, log id: 2d6d2018
2015-08-18 23:32:08,499 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-3) [f1cf267] After Installation host hosted_engine_1, oVirtNode
2015-08-18 23:32:09,007 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-3) [f1cf267] Correlation ID: f1cf267, Job ID: f8a7055a-5837-491a-aa29-336e07c4a757, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_1 installed
2015-08-18 23:32:09,099 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-3) [f1cf267] Lock freed to object EngineLock [exclusiveLocks= key: 80ce2151-3dbc-4945-834b-56315ca9f823 value: VDS
, sharedLocks= ]
2015-08-18 23:32:09,752 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-74) START, GetHardwareInfoVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vds=Host[hosted_engine_1,80ce2151-3dbc-4945-834b-56315ca9f823]), log id: 2e0e8136
2015-08-18 23:32:09,814 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, GetHardwareInfoVDSCommand, log id: 2e0e8136
2015-08-18 23:32:10,019 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-74) [78ac6d93] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 80ce2151-3dbc-4945-834b-56315ca9f823 Type: VDS
2015-08-18 23:32:10,127 INFO [org.ovirt.engine.core.bll.UpdateVdsGroupCommand] (DefaultQuartzScheduler_Worker-74) [f78b393] Running command: UpdateVdsGroupCommand internal: true. Entities affected : ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group EDIT_CLUSTER_CONFIGURATION with role type ADMIN
2015-08-18 23:32:10,331 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-74) [f78b393] Correlation ID: f78b393, Call Stack: null, Custom Event ID: -1, Message: Host cluster Default was updated by system
2015-08-18 23:32:10,499 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-74) [f78b393] START, GetHardwareInfoVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vds=Host[hosted_engine_1,80ce2151-3dbc-4945-834b-56315ca9f823]), log id: 35b0fbed
2015-08-18 23:32:10,512 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-74) [f78b393] FINISH, GetHardwareInfoVDSCommand, log id: 35b0fbed
2015-08-18 23:32:10,631 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-74) [f78b393] START, FullListVdsCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vds=Host[hosted_engine_1,80ce2151-3dbc-4945-834b-56315ca9f823], vmIds=[c5312b39-316c-4906-bc20-df2a472bda1f]), log id: 26532ecd
2015-08-18 23:32:10,638 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-74) [f78b393] FINISH, FullListVdsCommand, return: [{guestFQDN=, emulatedMachine=rhel6.5.0, pid=19275, guestDiskMapping={QEMU_DVD-ROM_={name=/dev/sr0}, 4548c127-e9eb-4be9-b={name=/dev/vda}}, displaySecurePort=-1, cpuType=Conroe, pauseCode=NOERR, smp=1, vmType=kvm, memSize=4096, vmName=HostedEngine, username=Unknown, clientIp=, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, displayIp=0, displayPort=5900, spiceSecureChannels=smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir, nicModel=rtl8139,pv, devices=[Ljava.lang.Object;@5174c98c, status=Up, guestIPs=, display=vnc}], log id: 26532ecd
2015-08-18 23:32:10,638 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) [f78b393] Importing VM HostedEngine as HostedEngine, as it is running on the on Host, but does not exist in the engine.
2015-08-18 23:32:10,666 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) [f78b393] VDS::UpdateVmRunTimeInfo Error: found VM on a VDS that is not in the database!
2015-08-18 23:32:10,672 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-74) [2be99d6d] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000010d Type: StoragePool
2015-08-18 23:32:10,802 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-74) [2be99d6d] START, SetMOMPolicyParametersVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823), log id: 4de8c7f7
2015-08-18 23:32:10,862 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-74) [2be99d6d] FINISH, SetMOMPolicyParametersVDSCommand, log id: 4de8c7f7
2015-08-18 23:32:11,223 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-74) [2be99d6d] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host hosted_engine_1 was set to Up.
2015-08-18 23:32:11,226 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-74) [2be99d6d] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host hosted_engine_1.
2015-08-18 23:32:11,734 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (DefaultQuartzScheduler_Worker-74) [7daa1372] Lock Acquired to object EngineLock [exclusiveLocks= key: HostedEngine value: VM_NAME
, sharedLocks= ]
2015-08-18 23:32:12,064 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (DefaultQuartzScheduler_Worker-74) [7daa1372] Running command: AddVmFromScratchCommand internal: true. Entities affected : ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_VM with role type USER
2015-08-18 23:32:12,152 INFO [org.ovirt.engine.core.bll.UpdateVdsGroupCommand] (ajp-/127.0.0.1:8702-2) [2b62e07a] Running command: UpdateVdsGroupCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group EDIT_CLUSTER_CONFIGURATION with role type ADMIN
2015-08-18 23:32:12,688 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [2b62e07a] Correlation ID: 2b62e07a, Call Stack: null, Custom Event ID: -1, Message: Host cluster Default was updated by admin@internal
2015-08-18 23:32:12,801 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-74) [7daa1372] Correlation ID: 7daa1372, Call Stack: null, Custom Event ID: -1, Message: VM HostedEngine was created by null@N/A.
2015-08-18 23:32:12,802 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (DefaultQuartzScheduler_Worker-74) [7daa1372] Lock freed to object EngineLock [exclusiveLocks= key: HostedEngine value: VM_NAME
, sharedLocks= ]
2015-08-18 23:32:12,809 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-74) [57eaf082] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 80ce2151-3dbc-4945-834b-56315ca9f823 Type: VDS
2015-08-18 23:32:12,813 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-74) [70e172ab] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 80ce2151-3dbc-4945-834b-56315ca9f823 Type: VDS
2015-08-18 23:32:15,872 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-85) START, FullListVdsCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vds=Host[hosted_engine_1,80ce2151-3dbc-4945-834b-56315ca9f823], vmIds=[c5312b39-316c-4906-bc20-df2a472bda1f]), log id: 7660820b
2015-08-18 23:32:15,890 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-85) FINISH, FullListVdsCommand, return: [{guestFQDN=, emulatedMachine=rhel6.5.0, pid=19275, guestDiskMapping={QEMU_DVD-ROM_={name=/dev/sr0}, 4548c127-e9eb-4be9-b={name=/dev/vda}}, displaySecurePort=-1, cpuType=Conroe, pauseCode=NOERR, smp=1, vmType=kvm, memSize=4096, vmName=HostedEngine, username=Unknown, clientIp=, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, displayIp=0, displayPort=5900, spiceSecureChannels=smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir, nicModel=rtl8139,pv, devices=[Ljava.lang.Object;@43151a78, status=Up, guestIPs=, display=vnc}], log id: 7660820b
2015-08-18 23:32:16,045 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Received a console Device without an address when processing VM c5312b39-316c-4906-bc20-df2a472bda1f devices, skipping device: {alias=console0, specParams={}, deviceType=console, deviceId=eee60f81-ce5e-4d1b-8fda-93529e74915b, device=console, type=console}
2015-08-18 23:32:16,053 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Received a memballoon Device without an address when processing VM c5312b39-316c-4906-bc20-df2a472bda1f devices, skipping device: {device=memballoon, specParams={model=none}, deviceType=balloon, type=balloon, alias=balloon0}
2015-08-18 23:32:16,053 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Received a vnc Device without an address when processing VM c5312b39-316c-4906-bc20-df2a472bda1f devices, skipping device: {device=vnc, specParams={spiceSecureChannels=smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir, displayIp=0}, deviceType=graphics, type=graphics, port=5900}
2015-08-18 23:32:16,054 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Empty or NULL values were passed for a VM c5312b39-316c-4906-bc20-df2a472bda1f device, Device is skipped
2015-08-18 23:35:50,817 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf".
2015-08-18 23:35:50,893 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties.
2015-08-18 23:35:50,893 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf".
2015-08-18 23:35:50,893 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf".
2015-08-18 23:35:50,893 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-java.conf".
2015-08-18 23:35:50,893 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf".
2015-08-18 23:35:50,894 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf".
2015-08-18 23:35:50,894 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf".
2015-08-18 23:35:50,894 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf".
2015-08-18 23:35:50,894 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf".
2015-08-18 23:35:50,894 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_AJP_ENABLED" is "true".
2015-08-18 23:35:50,894 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_AJP_PORT" is "8702".
2015-08-18 23:35:50,894 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear".
2015-08-18 23:35:50,894 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine".
2015-08-18 23:35:50,894 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000".
2015-08-18 23:35:50,894 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000".
2015-08-18 23:35:50,894 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_DATABASE" is "engine".
2015-08-18 23:35:50,894 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver".
2015-08-18 23:35:50,894 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_HOST" is "localhost".
2015-08-18 23:35:50,894 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_PASSWORD" is "***".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_PORT" is "5432".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_SECURED" is "False".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_DB_USER" is "engine".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_DEBUG_ADDRESS" is "".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_ETC" is "/etc/ovirt-engine".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE" is "/var/lib/ovirt-engine/external_truststore".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_PASSWORD" is "***".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_TYPE" is "JKS".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_FQDN" is "ycuihevm.redhat.com".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_GROUP" is "ovirt".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_HEAP_MAX" is "1024M".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_HEAP_MIN" is "1024M".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_HTTPS_ENABLED" is "false".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_HTTPS_PORT" is "None".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_HTTP_ENABLED" is "false".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_HTTP_PORT" is "None".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine".
2015-08-18 23:35:50,895 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_LOG_TO_CONSOLE" is "false".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_PERM_MAX" is "256m".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_PERM_MIN" is "256m".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_PROPERTIES" is " java.awt.headless=true sun.rmi.dgc.client.gcInterval=3600000 sun.rmi.dgc.server.gcInterval=3600000 jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_PROXY_ENABLED" is "true".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_PROXY_HTTP_PORT" is "80".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_REPORTS_BASE_URL" is "".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_REPORTS_DASHBOARD_URL" is "/flow.html?viewAsDashboardFrame=true".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_REPORTS_NOT_INSTALLED_URL" is "/ovirt-engine/ReportsNotInstalled.html".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_REPORTS_PROXY_URL" is "/ovirt/reports-interface".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_REPORTS_READ_TIMEOUT" is "".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_REPORTS_RIGHTCLICK_URL" is "".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_REPORTS_VERIFY_CHAIN" is "true".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_REPORTS_VERIFY_HOST" is "true".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_STOP_INTERVAL" is "1".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_STOP_TIME" is "10".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine".
2015-08-18 23:35:50,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up".
2015-08-18 23:35:50,897 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_URI" is "/ovirt-engine".
2015-08-18 23:35:50,897 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_USER" is "ovirt".
2015-08-18 23:35:50,897 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine".
2015-08-18 23:35:50,897 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine".
2015-08-18 23:35:50,897 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "ENGINE_VERBOSE_GC" is "false".
2015-08-18 23:35:50,897 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "JBOSS_HOME" is "/usr/share/jbossas".
2015-08-18 23:35:50,897 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime".
2015-08-18 23:35:50,897 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 43) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD,ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_PASSWORD".
2015-08-18 23:35:57,788 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-3) Start initializing Backend
2015-08-18 23:35:58,184 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ajp-/127.0.0.1:8702-3) Loading file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties
2015-08-18 23:35:59,012 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-3) Running ovirt-engine 3.5.4-1.2.el6ev
2015-08-18 23:35:59,013 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ajp-/127.0.0.1:8702-3) Start initializing dictionaries
2015-08-18 23:35:59,014 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ajp-/127.0.0.1:8702-3) Finished initializing dictionaries
2015-08-18 23:35:59,014 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-3) Start initializing AuditLogCleanupManager
2015-08-18 23:35:59,014 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-3) Setting audit cleanup manager to run at: 35 35 3 * * ?
2015-08-18 23:35:59,053 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-3) Finished initializing AuditLogCleanupManager
2015-08-18 23:35:59,053 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-3) Start initializing CommandEntityCleanupManager
2015-08-18 23:35:59,053 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-3) Setting command entity cleanup manager to run at: 35 35 3 * * ?
2015-08-18 23:35:59,054 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-3) Finished initializing CommandEntityCleanupManager
2015-08-18 23:35:59,055 INFO [org.ovirt.engine.core.bll.TagsDirector] (ajp-/127.0.0.1:8702-3) Start initializing TagsDirector
2015-08-18 23:35:59,060 INFO [org.ovirt.engine.core.bll.TagsDirector] (ajp-/127.0.0.1:8702-3) Tag root added to tree
2015-08-18 23:35:59,099 INFO [org.ovirt.engine.core.bll.TagsDirector] (ajp-/127.0.0.1:8702-3) Finished initializing TagsDirector
2015-08-18 23:35:59,099 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ajp-/127.0.0.1:8702-3) Start initializing IsoDomainListSyncronizer
2015-08-18 23:35:59,102 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ajp-/127.0.0.1:8702-3) Finished initializing IsoDomainListSyncronizer
2015-08-18 23:35:59,137 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-3) Completed initializing handlers
2015-08-18 23:35:59,177 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-3) Start initializing ErrorTranslatorImpl
2015-08-18 23:35:59,220 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-3) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table.
2015-08-18 23:35:59,221 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-3) Finished initializing ErrorTranslatorImpl
2015-08-18 23:35:59,221 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-3) Start initializing ErrorTranslatorImpl
2015-08-18 23:35:59,221 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-3) Finished initializing ErrorTranslatorImpl
2015-08-18 23:35:59,221 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-3) Mark incomplete jobs as UNKNOWN
2015-08-18 23:35:59,304 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ajp-/127.0.0.1:8702-3) Start initializing JobRepositoryCleanupManager
2015-08-18 23:35:59,305 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ajp-/127.0.0.1:8702-3) Finished initializing JobRepositoryCleanupManager
2015-08-18 23:35:59,305 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ajp-/127.0.0.1:8702-3) Start initializing AutoRecoveryManager
2015-08-18 23:35:59,305 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ajp-/127.0.0.1:8702-3) Finished initializing AutoRecoveryManager
2015-08-18 23:35:59,306 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ajp-/127.0.0.1:8702-3) Start initializing ExecutionMessageDirector
2015-08-18 23:35:59,307 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ajp-/127.0.0.1:8702-3) Finished initializing ExecutionMessageDirector
2015-08-18 23:35:59,466 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-3) Loading extension 'builtin-authn-internal'
2015-08-18 23:35:59,469 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-3) Extension 'builtin-authn-internal' loaded
2015-08-18 23:35:59,470 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-3) Loading extension 'internal'
2015-08-18 23:35:59,471 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-3) Extension 'internal' loaded
2015-08-18 23:35:59,477 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-3) Initializing extension 'builtin-authn-internal'
2015-08-18 23:35:59,477 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-3) Extension 'builtin-authn-internal' initialized
2015-08-18 23:35:59,478 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-3) Initializing extension 'internal'
2015-08-18 23:35:59,478 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-3) Extension 'internal' initialized
2015-08-18 23:35:59,478 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-3) Start of enabled extensions list
2015-08-18 23:35:59,478 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-3) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true'
2015-08-18 23:35:59,478 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-3) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true'
2015-08-18 23:35:59,478 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-3) End of enabled extensions list
2015-08-18 23:35:59,637 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 48) Initialization of AsyncTaskManager completed successfully.
2015-08-18 23:35:59,638 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 48) Start initializing ResourceManager
2015-08-18 23:35:59,745 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 48) Entered VdsManager constructor
2015-08-18 23:35:59,815 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 48) Initialize vdsBroker (rhevhtest-1.redhat.com,54,321)
2015-08-18 23:36:00,000 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 48) VDS 80ce2151-3dbc-4945-834b-56315ca9f823 was added to the Resource Manager
2015-08-18 23:36:00,029 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 48) Finished initializing ResourceManager
2015-08-18 23:36:00,029 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 48) Initialization of OvfDataUpdater completed successfully.
2015-08-18 23:36:00,029 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Start scheduling to enable vds load balancer
2015-08-18 23:36:00,029 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Finished scheduling to enable vds load balancer
2015-08-18 23:36:00,029 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Start HA Reservation check
2015-08-18 23:36:00,030 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Finished HA Reservation check
2015-08-18 23:36:00,041 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 48) Init VM custom properties utilities
2015-08-18 23:36:00,041 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 48) Init device custom properties utilities
2015-08-18 23:36:00,043 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Initializing Scheduling manager
2015-08-18 23:36:00,045 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Start initializing MacPoolManagerRanges
2015-08-18 23:36:00,071 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Finished initializing. Available MACs in pool: 256
2015-08-18 23:36:00,090 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) External scheduler disabled, discovery skipped
2015-08-18 23:36:00,094 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Initialized Scheduling manager
2015-08-18 23:36:00,094 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 48) Initializing DWH Heart Beat
2015-08-18 23:36:00,095 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 48) DWH Heart Beat initialized
2015-08-18 23:36:03,059 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-1.redhat.com/10.66.11.167
2015-08-18 23:36:03,688 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-5) START, GetHardwareInfoVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vds=Host[hosted_engine_1,80ce2151-3dbc-4945-834b-56315ca9f823]), log id: 1ce15afd
2015-08-18 23:36:03,698 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-5) FINISH, GetHardwareInfoVDSCommand, log id: 1ce15afd
2015-08-18 23:36:03,940 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-5) START, FullListVdsCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vds=Host[hosted_engine_1,80ce2151-3dbc-4945-834b-56315ca9f823], vmIds=[c5312b39-316c-4906-bc20-df2a472bda1f]), log id: 2d6b957e
2015-08-18 23:36:03,946 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-5) FINISH, FullListVdsCommand, return: [{status=Up, nicModel=rtl8139,pv, emulatedMachine=rhel6.5.0, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, pid=22592, clientIp=, devices=[Ljava.lang.Object;@58b4f7a, smp=1, vmType=kvm, displayIp=0, displaySecurePort=-1, memSize=4096, displayPort=5900, cpuType=Conroe, spiceSecureChannels=smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir, guestDiskMapping={QEMU_DVD-ROM_={name=/dev/sr0}, 4548c127-e9eb-4be9-b={name=/dev/vda}}, vmName=HostedEngine, display=vnc, pauseCode=NOERR}], log id: 2d6b957e
2015-08-18 23:36:03,955 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Received a console Device without an address when processing VM c5312b39-316c-4906-bc20-df2a472bda1f devices, skipping device: {alias=console0, specParams={}, deviceType=console, deviceId=eee60f81-ce5e-4d1b-8fda-93529e74915b, device=console, type=console}
2015-08-18 23:36:03,955 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Received a memballoon Device without an address when processing VM c5312b39-316c-4906-bc20-df2a472bda1f devices, skipping device: {device=memballoon, specParams={model=none}, deviceType=balloon, type=balloon, alias=balloon0}
2015-08-18 23:36:03,955 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Received a vnc Device without an address when processing VM c5312b39-316c-4906-bc20-df2a472bda1f devices, skipping device: {device=vnc, specParams={spiceSecureChannels=smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir, displayIp=0}, deviceType=graphics, type=graphics, port=5900}
2015-08-18 23:36:03,955 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Empty or NULL values were passed for a VM c5312b39-316c-4906-bc20-df2a472bda1f device, Device is skipped
2015-08-18 23:36:05,194 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Initializing Host: hosted_engine_1
2015-08-18 23:36:05,346 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-5) [356f0b95] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 80ce2151-3dbc-4945-834b-56315ca9f823 Type: VDS
2015-08-18 23:37:17,358 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-8) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json]
2015-08-18 23:37:17,362 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-8) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json]
2015-08-18 23:37:26,674 INFO [org.ovirt.engine.docs.utils.servlet.ContextSensitiveHelpMappingServlet] (ajp-/127.0.0.1:8702-8) Successfully read CSH mapping file '/usr/share/doc/rhevm-doc/manual/en-US/csh.conf.d/webadmin/10-webadmin-en-US.json'
2015-08-18 23:37:36,862 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginAdminUserCommand internal: false.
2015-08-18 23:37:36,870 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: UNASSIGNED not exist in string table
2015-08-18 23:37:36,870 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table
2015-08-18 23:37:36,870 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table
2015-08-18 23:37:36,870 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table
2015-08-18 23:37:36,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table
2015-08-18 23:37:36,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table
2015-08-18 23:37:36,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table
2015-08-18 23:37:36,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table
2015-08-18 23:37:36,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table
2015-08-18 23:37:36,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table
2015-08-18 23:37:36,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table
2015-08-18 23:37:36,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table
2015-08-18 23:37:36,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table
2015-08-18 23:37:36,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table
2015-08-18 23:37:36,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table
2015-08-18 23:37:36,872 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table
2015-08-18 23:37:36,872 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table
2015-08-18 23:37:36,872 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table
2015-08-18 23:37:37,450 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in.
2015-08-18 23:44:48,110 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginUserCommand internal: false.
2015-08-18 23:44:48,207 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in.
2015-08-18 23:44:51,379 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-7) [90f5353] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN
2015-08-18 23:44:51,523 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-7) [38e2bc2b] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: bed31fc7-9abc-426f-859c-3a2df7d40ee1 Type: VDS
2015-08-18 23:44:52,012 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-7) [38e2bc2b] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-00000000010d value: REGISTER_VDS
, sharedLocks= ]
2015-08-18 23:44:52,015 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [38e2bc2b] START, RemoveVdsVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1), log id: 3a04f460
2015-08-18 23:44:52,015 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [38e2bc2b] FINISH, RemoveVdsVDSCommand, log id: 3a04f460
2015-08-18 23:44:52,018 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [38e2bc2b] START, AddVdsVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1), log id: fef5c3c
2015-08-18 23:44:52,018 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [38e2bc2b] AddVds - entered , starting logic to add VDS bed31fc7-9abc-426f-859c-3a2df7d40ee1
2015-08-18 23:44:52,019 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [38e2bc2b] AddVds - VDS bed31fc7-9abc-426f-859c-3a2df7d40ee1 was added, will try to add it to the resource manager
2015-08-18 23:44:52,019 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-7) [38e2bc2b] Entered VdsManager constructor
2015-08-18 23:44:52,026 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-7) [38e2bc2b] Initialize vdsBroker (rhevhtest-2.redhat.com,54,321)
2015-08-18 23:44:52,044 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-7) [38e2bc2b] VDS bed31fc7-9abc-426f-859c-3a2df7d40ee1 was added to the Resource Manager
2015-08-18 23:44:52,044 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [38e2bc2b] FINISH, AddVdsVDSCommand, log id: fef5c3c
2015-08-18 23:44:52,054 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [38e2bc2b] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host hosted_engine_2.
2015-08-18 23:44:52,364 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-20) [759d5d33] Lock Acquired to object EngineLock [exclusiveLocks= key: bed31fc7-9abc-426f-859c-3a2df7d40ee1 value: VDS
, sharedLocks= ]
2015-08-18 23:44:52,462 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [38e2bc2b] Correlation ID: 90f5353, Job ID: 10f12b6b-23da-44ac-a246-ba2f3025933b, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_2 was added by admin@internal.
2015-08-18 23:44:52,689 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-20) [759d5d33] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: bed31fc7-9abc-426f-859c-3a2df7d40ee1 Type: VDS
2015-08-18 23:44:52,692 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-20) [759d5d33] Before Installation host bed31fc7-9abc-426f-859c-3a2df7d40ee1, hosted_engine_2
2015-08-18 23:44:52,716 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-20) [759d5d33] START, SetVdsStatusVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 4333e4a7
2015-08-18 23:44:52,883 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-20) [759d5d33] FINISH, SetVdsStatusVDSCommand, log id: 4333e4a7
2015-08-18 23:44:53,001 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-7-thread-20) [759d5d33] Installation rhevhtest-2.redhat.com: Connected to host rhevhtest-2.redhat.com with SSH key fingerprint: d3:87:4c:7c:d9:d1:fd:21:13:a9:c9:8d:5d:f9:b0:2a
2015-08-18 23:44:53,121 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-20) [759d5d33] Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Connected to host rhevhtest-2.redhat.com with SSH key fingerprint: d3:87:4c:7c:d9:d1:fd:21:13:a9:c9:8d:5d:f9:b0:2a.
2015-08-18 23:44:53,140 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-7-thread-20) [759d5d33] Installation of rhevhtest-2.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar
2015-08-18 23:44:53,438 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-7-thread-20) SSH execute root@rhevhtest-2.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True'
2015-08-18 23:44:54,133 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Stage: Initializing
2015-08-18 23:44:54,252 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Initializing.
2015-08-18 23:44:54,263 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Stage: Environment setup
2015-08-18 23:44:54,360 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Environment setup.
2015-08-18 23:44:54,360 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Stage: Environment packages setup
2015-08-18 23:44:54,460 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Environment packages setup.
2015-08-18 23:44:54,460 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Stage: Programs detection
2015-08-18 23:44:54,560 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Programs detection.
2015-08-18 23:44:54,560 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Stage: Environment customization
2015-08-18 23:44:54,660 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Environment customization.
2015-08-18 23:44:54,660 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Kdump unsupported
2015-08-18 23:44:54,769 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Kdump unsupported.
2015-08-18 23:44:54,774 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Host is hypervisor
2015-08-18 23:44:54,893 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Host is hypervisor.
2015-08-18 23:44:55,044 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Initializing Host: hosted_engine_2
2015-08-18 23:44:55,114 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20150819034454-h0pppq.log'
2015-08-18 23:44:55,302 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Logs at host located at: '/tmp/ovirt-host-deploy-20150819034454-h0pppq.log'.
2015-08-18 23:44:55,353 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host rhevhtest-2.redhat.com reports unique id 4C4C4544-0056-4210-8032-C2C04F463358
2015-08-18 23:44:55,363 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0056-4210-8032-C2C04F463358 to Host rhevhtest-2.redhat.com
2015-08-18 23:44:55,560 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Disabling Kdump integration
2015-08-18 23:44:55,762 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Disabling Kdump integration.
2015-08-18 23:44:55,774 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Stage: Setup validation
2015-08-18 23:44:55,943 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Setup validation.
2015-08-18 23:44:55,944 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Cannot validate host name settings, reason: resolved host does not match any of the local addresses
2015-08-18 23:44:56,035 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_2 installation in progress . Cannot validate host name settings, reason: resolved host does not match any of the local addresses.
2015-08-18 23:44:56,035 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Hardware supports virtualization
2015-08-18 23:44:56,143 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Hardware supports virtualization.
2015-08-18 23:44:56,143 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Stage: Transaction setup
2015-08-18 23:44:56,252 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Transaction setup.
2015-08-18 23:44:56,252 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Stage: Misc configuration
2015-08-18 23:44:56,352 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Misc configuration.
2015-08-18 23:44:56,352 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Stage: Package installation
2015-08-18 23:44:56,452 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Package installation.
2015-08-18 23:44:57,090 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Stage: Misc configuration
2015-08-18 23:44:57,293 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Misc configuration.
2015-08-18 23:44:57,391 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Enrolling certificate
2015-08-18 23:44:57,498 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Enrolling certificate.
2015-08-18 23:44:58,789 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Stage: Transaction commit
2015-08-18 23:44:58,908 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Transaction commit.
2015-08-18 23:44:58,908 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Stage: Closing up
2015-08-18 23:44:59,000 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Closing up.
2015-08-18 23:45:04,230 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Stopping libvirtd
2015-08-18 23:45:04,328 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stopping libvirtd.
2015-08-18 23:45:04,328 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Starting vdsm
2015-08-18 23:45:04,428 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Starting vdsm.
2015-08-18 23:45:15,025 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Stage: Pre-termination
2015-08-18 23:45:15,136 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Pre-termination.
2015-08-18 23:45:15,143 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20150818234515-rhevhtest-2.redhat.com-759d5d33.log'
2015-08-18 23:45:15,256 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20150818234515-rhevhtest-2.redhat.com-759d5d33.log'.
2015-08-18 23:45:15,454 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-2.redhat.com: Stage: Termination
2015-08-18 23:45:15,561 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 759d5d33, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Termination.
2015-08-18 23:45:15,564 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-2.redhat.com/10.66.11.107
2015-08-18 23:45:15,710 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-7-thread-20) [759d5d33] Engine managed to communicate with VDSM agent on host hosted_engine_2
2015-08-18 23:45:16,193 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-7-thread-20) [2b62fb1a] Running command: SetupNetworksCommand internal: true. Entities affected : ID: bed31fc7-9abc-426f-859c-3a2df7d40ee1 Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN
2015-08-18 23:45:16,198 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-7-thread-20) [2b62fb1a] START, SetupNetworksVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, force=false, checkConnectivity=true, conectivityTimeout=120,
networks=[rhevm {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-00000000010d, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}],
bonds=[],
interfaces=[em1 {id=06472982-5b8f-44c4-b566-805633a0959f, vdsId=bed31fc7-9abc-426f-859c-3a2df7d40ee1, macAddress=18:03:73:39:a6:9c, networkName=rhevm, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.66.11.107, subnet=255.255.252.0, gateway=null, mtu=1500, bridged=false, speed=1000, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}],
removedNetworks=[],
removedBonds=[]), log id: 1bf7c8ac
2015-08-18 23:45:16,199 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-7-thread-20) [2b62fb1a] FINISH, SetupNetworksVDSCommand, log id: 1bf7c8ac
2015-08-18 23:45:16,203 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-2.redhat.com/10.66.11.107
2015-08-18 23:45:16,219 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-2.redhat.com/10.66.11.107
2015-08-18 23:45:25,580 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-2.redhat.com/10.66.11.107
2015-08-18 23:45:26,148 INFO [org.ovirt.engine.core.bll.network.host.CommitNetworkChangesCommand] (org.ovirt.thread.pool-7-thread-20) [7e88c227] Running command: CommitNetworkChangesCommand internal: true. Entities affected : ID: bed31fc7-9abc-426f-859c-3a2df7d40ee1 Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN
2015-08-18 23:45:26,149 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-7-thread-20) [7e88c227] START, SetSafeNetworkConfigVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1), log id: 3ba6a640
2015-08-18 23:45:28,751 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-7-thread-20) [7e88c227] FINISH, SetSafeNetworkConfigVDSCommand, log id: 3ba6a640
2015-08-18 23:45:29,071 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-20) [7e88c227] Correlation ID: 7e88c227, Call Stack: null, Custom Event ID: -1, Message: Network changes were saved on host hosted_engine_2
2015-08-18 23:45:29,072 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-20) [7e88c227] START, SetVdsStatusVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, status=Initializing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 547370e5
2015-08-18 23:45:29,264 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-20) [7e88c227] FINISH, SetVdsStatusVDSCommand, log id: 547370e5
2015-08-18 23:45:29,264 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-20) [7e88c227] After Installation host hosted_engine_2, oVirtNode
2015-08-18 23:45:29,462 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-20) [7e88c227] Correlation ID: 759d5d33, Job ID: 10f12b6b-23da-44ac-a246-ba2f3025933b, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_2 installed
2015-08-18 23:45:29,646 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-20) [7e88c227] Lock freed to object EngineLock [exclusiveLocks= key: bed31fc7-9abc-426f-859c-3a2df7d40ee1 value: VDS
, sharedLocks= ]
2015-08-18 23:45:31,084 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-46) START, GetHardwareInfoVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, vds=Host[hosted_engine_2,bed31fc7-9abc-426f-859c-3a2df7d40ee1]), log id: 3271cff4
2015-08-18 23:45:31,109 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, GetHardwareInfoVDSCommand, log id: 3271cff4
2015-08-18 23:45:31,509 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-46) [9b50c4a] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: bed31fc7-9abc-426f-859c-3a2df7d40ee1 Type: VDS
2015-08-18 23:45:31,534 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-46) [9b50c4a] START, GetHardwareInfoVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, vds=Host[hosted_engine_2,bed31fc7-9abc-426f-859c-3a2df7d40ee1]), log id: 142d995c
2015-08-18 23:45:31,546 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-46) [9b50c4a] FINISH, GetHardwareInfoVDSCommand, log id: 142d995c
2015-08-18 23:45:31,888 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-46) [4f66f756] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000010d Type: StoragePool
2015-08-18 23:45:32,068 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-46) [4f66f756] START, SetMOMPolicyParametersVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1), log id: 40fce0ea
2015-08-18 23:45:32,131 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-46) [4f66f756] FINISH, SetMOMPolicyParametersVDSCommand, log id: 40fce0ea
2015-08-18 23:45:32,268 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-46) [4f66f756] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host hosted_engine_2 was set to Up.
2015-08-18 23:45:32,270 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-46) [4f66f756] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host hosted_engine_2.
2015-08-18 23:45:32,668 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-46) [369f1543] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: bed31fc7-9abc-426f-859c-3a2df7d40ee1 Type: VDS
2015-08-18 23:45:32,670 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-46) [61285494] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: bed31fc7-9abc-426f-859c-3a2df7d40ee1 Type: VDS
2015-08-18 23:45:33,526 INFO [org.ovirt.engine.core.bll.UpdateVdsGroupCommand] (ajp-/127.0.0.1:8702-1) [20ad3d42] Running command: UpdateVdsGroupCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group EDIT_CLUSTER_CONFIGURATION with role type ADMIN
2015-08-18 23:45:33,912 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [20ad3d42] Correlation ID: 20ad3d42, Call Stack: null, Custom Event ID: -1, Message: Host cluster Default was updated by admin@internal
2015-08-18 23:52:35,689 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-7-thread-27) [51ef3790] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: bed31fc7-9abc-426f-859c-3a2df7d40ee1 Type: VDSAction group MANIPULATE_HOST with role type ADMIN
2015-08-18 23:52:35,690 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-27) [51ef3790] START, SetVdsStatusVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 3d8fc0e4
2015-08-18 23:52:35,790 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-27) [51ef3790] FINISH, SetVdsStatusVDSCommand, log id: 3d8fc0e4
2015-08-18 23:52:36,345 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-7-thread-27) [51ef3790] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: bed31fc7-9abc-426f-859c-3a2df7d40ee1 Type: VDS
2015-08-18 23:52:36,348 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetHaMaintenanceModeVDSCommand] (org.ovirt.thread.pool-7-thread-27) [51ef3790] START, SetHaMaintenanceModeVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1), log id: 12fc68c1
2015-08-18 23:52:36,351 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetHaMaintenanceModeVDSCommand] (org.ovirt.thread.pool-7-thread-27) [51ef3790] FINISH, SetHaMaintenanceModeVDSCommand, log id: 12fc68c1
2015-08-18 23:52:36,448 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-27) [51ef3790] Correlation ID: 51ef3790, Job ID: 0d662137-0104-413c-89d3-ab7dbf76d9f2, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_2 was switched to Maintenance mode by admin@internal.
2015-08-18 23:52:37,776 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = bed31fc7-9abc-426f-859c-3a2df7d40ee1 : hosted_engine_2
2015-08-18 23:52:38,428 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-29) Clearing cache of pool: 00000002-0002-0002-0002-00000000010d for problematic entities of VDS: hosted_engine_2.
2015-08-18 23:52:38,428 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-29) Removing vds [bed31fc7-9abc-426f-859c-3a2df7d40ee1] from the domain in maintenance cache
2015-08-18 23:52:38,428 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-29) Removing host(s) [bed31fc7-9abc-426f-859c-3a2df7d40ee1] from hosts unseen domain report cache
2015-08-18 23:57:08,898 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-5) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic
2015-08-18 23:57:08,898 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-5) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic
2015-08-18 23:57:08,899 INFO [org.ovirt.engine.core.bll.UpgradeOvirtNodeCommand] (ajp-/127.0.0.1:8702-5) [232b6d49] Running command: UpgradeOvirtNodeCommand internal: false. Entities affected : ID: bed31fc7-9abc-426f-859c-3a2df7d40ee1 Type: VDSAction group EDIT_HOST_CONFIGURATION with role type ADMIN
2015-08-18 23:57:09,007 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-5) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic
2015-08-18 23:57:09,007 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-5) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic
2015-08-18 23:57:09,010 INFO [org.ovirt.engine.core.bll.UpgradeOvirtNodeInternalCommand] (ajp-/127.0.0.1:8702-5) [232b6d49] Lock Acquired to object EngineLock [exclusiveLocks= key: bed31fc7-9abc-426f-859c-3a2df7d40ee1 value: VDS
, sharedLocks= ]
2015-08-18 23:57:09,011 INFO [org.ovirt.engine.core.bll.UpgradeOvirtNodeInternalCommand] (org.ovirt.thread.pool-7-thread-9) [232b6d49] Running command: UpgradeOvirtNodeInternalCommand internal: true. Entities affected : ID: bed31fc7-9abc-426f-859c-3a2df7d40ee1 Type: VDS
2015-08-18 23:57:09,012 INFO [org.ovirt.engine.core.bll.UpgradeOvirtNodeInternalCommand] (org.ovirt.thread.pool-7-thread-9) [232b6d49] Execute upgrade host bed31fc7-9abc-426f-859c-3a2df7d40ee1, hosted_engine_2
2015-08-18 23:57:09,013 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-9) [232b6d49] START, SetVdsStatusVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 2b24e76e
2015-08-18 23:57:09,017 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [232b6d49] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host hosted_engine_2.
2015-08-18 23:57:09,124 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-9) [232b6d49] FINISH, SetVdsStatusVDSCommand, log id: 2b24e76e
2015-08-18 23:57:09,247 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-7-thread-9) [232b6d49] Installation rhevhtest-2.redhat.com: Connected to host rhevhtest-2.redhat.com with SSH key fingerprint: d3:87:4c:7c:d9:d1:fd:21:13:a9:c9:8d:5d:f9:b0:2a
2015-08-18 23:57:09,316 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [232b6d49] Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_2 configuration was updated by admin@internal.
2015-08-18 23:57:09,424 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-9) [232b6d49] Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Connected to host rhevhtest-2.redhat.com with SSH key fingerprint: d3:87:4c:7c:d9:d1:fd:21:13:a9:c9:8d:5d:f9:b0:2a.
2015-08-18 23:57:09,437 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-7-thread-9) [232b6d49] Installation rhevhtest-2.redhat.com: Sending file /usr/share/rhev-hypervisor/rhevh-7.1-20150813.0.el7ev.iso to /data/updates/ovirt-node-image.iso
2015-08-18 23:57:09,549 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-9) [232b6d49] Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Sending file /usr/share/rhev-hypervisor/rhevh-7.1-20150813.0.el7ev.iso to /data/updates/ovirt-node-image.iso.
2015-08-18 23:57:09,549 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-7-thread-9) SSH execute root@rhevhtest-2.redhat.com 'mkdir -p '/data/updates''
2015-08-18 23:57:27,460 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-7-thread-9) [232b6d49] Installation rhevhtest-2.redhat.com: Executing /usr/share/vdsm-reg/vdsm-upgrade
2015-08-18 23:57:49,493 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-9) [232b6d49] Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Executing /usr/share/vdsm-reg/vdsm-upgrade.
2015-08-18 23:57:49,493 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-7-thread-9) SSH execute root@rhevhtest-2.redhat.com '/usr/share/vdsm-reg/vdsm-upgrade'
2015-08-18 23:57:49,625 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:57:49,628 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Temporary Directory is: /data/tmpAJq92f
2015-08-18 23:57:54,843 WARN [org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker] (ResponseWorker) Exception thrown during message processing
2015-08-18 23:58:13,995 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Temporary Directory is: /data/tmpAJq92f
.
2015-08-18 23:58:14,342 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:58:14,343 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.LockFile: INFO Acquiring Lock
2015-08-18 23:58:22,201 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.LockFile: INFO Acquiring Lock
.
2015-08-18 23:58:22,202 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:58:22,245 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running pre-upgrade hooks
2015-08-18 23:58:22,435 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running pre-upgrade hooks
.
2015-08-18 23:58:22,435 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:58:22,436 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-vdsm
2015-08-18 23:58:22,634 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-vdsm
.
2015-08-18 23:58:22,634 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:58:22,635 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO pre-upgrade hooks completed
2015-08-18 23:58:22,842 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO pre-upgrade hooks completed
.
2015-08-18 23:58:22,843 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:58:22,843 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running pre-upgrade hooks
2015-08-18 23:58:23,245 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running pre-upgrade hooks
.
2015-08-18 23:58:23,245 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:58:23,246 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-vdsm
2015-08-18 23:58:23,467 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-vdsm
.
2015-08-18 23:58:23,467 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:58:23,468 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO pre-upgrade hooks completed
2015-08-18 23:58:23,676 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO pre-upgrade hooks completed
.
2015-08-18 23:58:23,676 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:58:23,677 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO hooks: False
2015-08-18 23:58:23,776 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO hooks: False
.
2015-08-18 23:58:23,776 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:58:23,777 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Installing Bootloader
2015-08-18 23:58:23,876 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Installing Bootloader
.
2015-08-18 23:58:23,876 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:58:23,877 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running post-upgrade hooks
2015-08-18 23:58:23,984 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running post-upgrade hooks
.
2015-08-18 23:58:23,984 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:58:23,985 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-sanlock-check
2015-08-18 23:58:24,084 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-sanlock-check
.
2015-08-18 23:59:49,470 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:59:49,471 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO post-upgrade hooks completed
2015-08-18 23:59:49,601 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO post-upgrade hooks completed
.
2015-08-18 23:59:49,601 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:59:49,602 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running post-upgrade hooks
2015-08-18 23:59:49,784 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running post-upgrade hooks
.
2015-08-18 23:59:49,785 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:59:49,785 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-sanlock-check
2015-08-18 23:59:49,976 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-sanlock-check
.
2015-08-18 23:59:51,595 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:59:51,596 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO post-upgrade hooks completed
2015-08-18 23:59:51,724 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO post-upgrade hooks completed
.
2015-08-18 23:59:51,724 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:59:51,725 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Scheduling Reboot
2015-08-18 23:59:51,824 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Scheduling Reboot
.
2015-08-18 23:59:51,824 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:59:51,825 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Reboot Scheduled
2015-08-18 23:59:51,933 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Reboot Scheduled
.
2015-08-18 23:59:51,933 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:59:51,934 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Upgrade Completed
2015-08-18 23:59:52,032 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Upgrade Completed
.
2015-08-18 23:59:52,033 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:59:52,033 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Cleaning up temporary directory
2015-08-18 23:59:52,133 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Cleaning up temporary directory
.
2015-08-18 23:59:52,133 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:59:52,133 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: ovirt-node-upgrade; Details: Upgraded Succeeded. Rebooting
2015-08-18 23:59:52,233 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: ovirt-node-upgrade; Details: Upgraded Succeeded. Rebooting .
2015-08-18 23:59:52,233 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-2.redhat.com:
2015-08-18 23:59:52,234 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-2.redhat.com: Step: RHEV_INSTALL
2015-08-18 23:59:52,333 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Step: RHEV_INSTALL.
2015-08-18 23:59:52,334 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-9) [232b6d49] START, SetVdsStatusVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, status=Reboot, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 33848135
2015-08-18 23:59:52,433 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-9) [232b6d49] FINISH, SetVdsStatusVDSCommand, log id: 33848135
2015-08-18 23:59:52,433 INFO [org.ovirt.engine.core.bll.UpgradeOvirtNodeInternalCommand] (org.ovirt.thread.pool-7-thread-9) [232b6d49] After upgrade host bed31fc7-9abc-426f-859c-3a2df7d40ee1, hosted_engine_2: success
2015-08-18 23:59:52,433 INFO [org.ovirt.engine.core.bll.UpgradeOvirtNodeInternalCommand] (org.ovirt.thread.pool-7-thread-43) [232b6d49] Waiting 300 seconds, for server to finish reboot process.
2015-08-18 23:59:52,757 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-9) [232b6d49] Correlation ID: 232b6d49, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_2 installed
2015-08-18 23:59:52,757 INFO [org.ovirt.engine.core.bll.UpgradeOvirtNodeInternalCommand] (org.ovirt.thread.pool-7-thread-9) [232b6d49] Lock freed to object EngineLock [exclusiveLocks= key: bed31fc7-9abc-426f-859c-3a2df7d40ee1 value: VDS
, sharedLocks= ]
2015-08-19 00:04:52,642 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-43) [232b6d49] START, SetVdsStatusVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, status=NonResponsive, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 7599eb08
2015-08-19 00:04:52,767 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-43) [232b6d49] FINISH, SetVdsStatusVDSCommand, log id: 7599eb08
2015-08-19 00:04:55,417 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-2.redhat.com/10.66.11.107
2015-08-19 00:04:55,579 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GetHardwareInfoVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, vds=Host[hosted_engine_2,bed31fc7-9abc-426f-859c-3a2df7d40ee1]), log id: 4119b6a2
2015-08-19 00:04:55,630 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GetHardwareInfoVDSCommand, log id: 4119b6a2
2015-08-19 00:04:55,832 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-77) [656cea7b] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: bed31fc7-9abc-426f-859c-3a2df7d40ee1 Type: VDS
2015-08-19 00:04:55,842 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-77) [68492391] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000010d Type: StoragePool
2015-08-19 00:04:55,942 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-77) [68492391] START, SetMOMPolicyParametersVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1), log id: 5c8a83ae
2015-08-19 00:04:56,002 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-77) [68492391] FINISH, SetMOMPolicyParametersVDSCommand, log id: 5c8a83ae
2015-08-19 00:04:56,105 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-77) [68492391] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host hosted_engine_2 was set to Up.
2015-08-19 00:04:56,118 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-77) [68492391] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host hosted_engine_2.
2015-08-19 00:04:56,439 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-77) [12a8ff55] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: bed31fc7-9abc-426f-859c-3a2df7d40ee1 Type: VDS
2015-08-19 00:06:31,945 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp-/127.0.0.1:8702-19) [17935375] Lock Acquired to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 00:06:32,108 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-19) [17935375] Candidate host hosted_engine_1 (80ce2151-3dbc-4945-834b-56315ca9f823) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory
2015-08-19 00:06:32,575 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-2) [17935375] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group MIGRATE_VM with role type USER, ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_VM with role type USER
2015-08-19 00:06:33,228 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-2) [17935375] START, MigrateVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=bed31fc7-9abc-426f-859c-3a2df7d40ee1, dstHost=rhevhtest-2.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 38dedddf
2015-08-19 00:06:33,230 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-2) [17935375] START, MigrateBrokerVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=bed31fc7-9abc-426f-859c-3a2df7d40ee1, dstHost=rhevhtest-2.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 45166e7c
2015-08-19 00:06:33,242 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-2) [17935375] FINISH, MigrateBrokerVDSCommand, log id: 45166e7c
2015-08-19 00:06:33,372 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-2) [17935375] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 38dedddf
2015-08-19 00:06:33,506 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-2) [17935375] Correlation ID: 17935375, Job ID: 90b66d93-710a-467b-b752-0083add02eb7, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_2, User: admin@internal).
2015-08-19 00:06:35,142 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) VM HostedEngine c5312b39-316c-4906-bc20-df2a472bda1f moved from MigratingFrom --> Up
2015-08-19 00:06:35,142 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Adding VM c5312b39-316c-4906-bc20-df2a472bda1f to re-run list
2015-08-19 00:06:35,720 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Rerun vm c5312b39-316c-4906-bc20-df2a472bda1f. Called from vds hosted_engine_1
2015-08-19 00:06:35,743 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) START, MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f), log id: 33183aff
2015-08-19 00:06:35,748 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) Failed in MigrateStatusVDS method
2015-08-19 00:06:35,749 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value
StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=10, mMessage=Could not connect to peer VDS]]
2015-08-19 00:06:35,749 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) HostName = hosted_engine_1
2015-08-19 00:06:35,752 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) Command MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Could not connect to peer VDS, code = 10
2015-08-19 00:06:35,752 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) FINISH, MigrateStatusVDSCommand, log id: 33183aff
2015-08-19 00:06:36,287 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-5) Correlation ID: 17935375, Job ID: 90b66d93-710a-467b-b752-0083add02eb7, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Could not connect to peer host (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_2).
2015-08-19 00:06:36,403 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-5) Lock freed to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 00:06:57,001 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-26) [1bd01c10] Running command: SetVmTicketCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group CONNECT_TO_VM with role type USER
2015-08-19 00:06:57,008 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-26) [1bd01c10] START, SetVmTicketVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, ticket=cs/CUrUKypel, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 92db18a
2015-08-19 00:06:57,020 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-26) [1bd01c10] FINISH, SetVmTicketVDSCommand, log id: 92db18a
2015-08-19 00:06:57,220 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-26) [1bd01c10] Correlation ID: 1bd01c10, Call Stack: null, Custom Event ID: -1, Message: user admin@internal initiated console session for VM HostedEngine
2015-08-19 00:07:12,325 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-21) [37093b6e] Running command: SetVmTicketCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group CONNECT_TO_VM with role type USER
2015-08-19 00:07:12,327 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-21) [37093b6e] START, SetVmTicketVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, ticket=+ws+B6d4qwGR, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 42041b88
2015-08-19 00:07:12,339 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-21) [37093b6e] FINISH, SetVmTicketVDSCommand, log id: 42041b88
2015-08-19 00:07:12,553 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-21) [37093b6e] Correlation ID: 37093b6e, Call Stack: null, Custom Event ID: -1, Message: user admin@internal initiated console session for VM HostedEngine
2015-08-19 00:07:36,764 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-4) [7acf215e] Running command: SetVmTicketCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group CONNECT_TO_VM with role type USER
2015-08-19 00:07:36,768 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-4) [7acf215e] START, SetVmTicketVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, ticket=Wq9rDClASLV+, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 5876aafe
2015-08-19 00:07:36,779 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-4) [7acf215e] FINISH, SetVmTicketVDSCommand, log id: 5876aafe
2015-08-19 00:07:37,086 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [7acf215e] Correlation ID: 7acf215e, Call Stack: null, Custom Event ID: -1, Message: user admin@internal initiated console session for VM HostedEngine
2015-08-19 00:07:59,988 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-7) [32f70f5] Running command: SetVmTicketCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group CONNECT_TO_VM with role type USER
2015-08-19 00:07:59,990 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-7) [32f70f5] START, SetVmTicketVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, ticket=XeJiCaNnz7+B, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 7306fc49
2015-08-19 00:08:00,001 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-7) [32f70f5] FINISH, SetVmTicketVDSCommand, log id: 7306fc49
2015-08-19 00:08:00,186 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [32f70f5] Correlation ID: 32f70f5, Call Stack: null, Custom Event ID: -1, Message: user admin@internal initiated console session for VM HostedEngine
2015-08-19 00:08:11,473 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp-/127.0.0.1:8702-9) [49b01d94] Lock Acquired to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 00:08:11,531 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-9) [49b01d94] Candidate host hosted_engine_1 (80ce2151-3dbc-4945-834b-56315ca9f823) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory
2015-08-19 00:08:12,320 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-17) [49b01d94] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group MIGRATE_VM with role type USER, ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_VM with role type USER
2015-08-19 00:08:12,439 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-17) [49b01d94] START, MigrateVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=bed31fc7-9abc-426f-859c-3a2df7d40ee1, dstHost=rhevhtest-2.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 143a0e3b
2015-08-19 00:08:12,440 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-17) [49b01d94] START, MigrateBrokerVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=bed31fc7-9abc-426f-859c-3a2df7d40ee1, dstHost=rhevhtest-2.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 12f0425
2015-08-19 00:08:12,445 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-17) [49b01d94] FINISH, MigrateBrokerVDSCommand, log id: 12f0425
2015-08-19 00:08:12,562 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-17) [49b01d94] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 143a0e3b
2015-08-19 00:08:12,662 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-17) [49b01d94] Correlation ID: 49b01d94, Job ID: 24201405-112a-40d1-9a05-174748dbd93e, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_2, User: admin@internal).
2015-08-19 00:08:15,397 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) VM HostedEngine c5312b39-316c-4906-bc20-df2a472bda1f moved from MigratingFrom --> Up
2015-08-19 00:08:15,397 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Adding VM c5312b39-316c-4906-bc20-df2a472bda1f to re-run list
2015-08-19 00:08:15,693 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Rerun vm c5312b39-316c-4906-bc20-df2a472bda1f. Called from vds hosted_engine_1
2015-08-19 00:08:15,696 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-22) START, MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f), log id: 70e7d841
2015-08-19 00:08:15,699 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-22) Failed in MigrateStatusVDS method
2015-08-19 00:08:15,699 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-22) Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value
StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=10, mMessage=Could not connect to peer VDS]]
2015-08-19 00:08:15,699 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-22) HostName = hosted_engine_1
2015-08-19 00:08:15,699 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-22) Command MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Could not connect to peer VDS, code = 10
2015-08-19 00:08:15,699 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-22) FINISH, MigrateStatusVDSCommand, log id: 70e7d841
2015-08-19 00:08:15,984 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-22) Correlation ID: 49b01d94, Job ID: 24201405-112a-40d1-9a05-174748dbd93e, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Could not connect to peer host (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_2).
2015-08-19 00:08:16,084 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-22) Lock freed to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 00:09:58,443 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp-/127.0.0.1:8702-15) [61ad2160] Lock Acquired to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 00:09:58,475 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-15) [61ad2160] Candidate host hosted_engine_1 (80ce2151-3dbc-4945-834b-56315ca9f823) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory
2015-08-19 00:09:58,963 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-37) [61ad2160] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group MIGRATE_VM with role type USER, ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_VM with role type USER
2015-08-19 00:09:59,188 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-37) [61ad2160] START, MigrateVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=bed31fc7-9abc-426f-859c-3a2df7d40ee1, dstHost=rhevhtest-2.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 735ed2e1
2015-08-19 00:09:59,188 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-37) [61ad2160] START, MigrateBrokerVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=bed31fc7-9abc-426f-859c-3a2df7d40ee1, dstHost=rhevhtest-2.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 3d6e1cfd
2015-08-19 00:09:59,194 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-37) [61ad2160] FINISH, MigrateBrokerVDSCommand, log id: 3d6e1cfd
2015-08-19 00:09:59,287 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-37) [61ad2160] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 735ed2e1
2015-08-19 00:09:59,396 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-37) [61ad2160] Correlation ID: 61ad2160, Job ID: 3c053eab-0d7c-4037-ae6d-8f29ac1baeed, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_2, User: admin@internal).
2015-08-19 00:10:00,141 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) VM HostedEngine c5312b39-316c-4906-bc20-df2a472bda1f moved from MigratingFrom --> Up
2015-08-19 00:10:00,141 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Adding VM c5312b39-316c-4906-bc20-df2a472bda1f to re-run list
2015-08-19 00:10:00,445 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Rerun vm c5312b39-316c-4906-bc20-df2a472bda1f. Called from vds hosted_engine_1
2015-08-19 00:10:00,465 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-38) START, MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f), log id: 430a02bf
2015-08-19 00:10:00,468 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-38) Failed in MigrateStatusVDS method
2015-08-19 00:10:00,468 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-38) Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value
StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=10, mMessage=Could not connect to peer VDS]]
2015-08-19 00:10:00,468 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-38) HostName = hosted_engine_1
2015-08-19 00:10:00,468 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-38) Command MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Could not connect to peer VDS, code = 10
2015-08-19 00:10:00,468 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-38) FINISH, MigrateStatusVDSCommand, log id: 430a02bf
2015-08-19 00:10:00,787 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-38) Correlation ID: 61ad2160, Job ID: 3c053eab-0d7c-4037-ae6d-8f29ac1baeed, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Could not connect to peer host (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_2).
2015-08-19 00:10:00,905 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-38) Lock freed to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 00:15:36,716 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-7-thread-49) [2b2de125] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 80ce2151-3dbc-4945-834b-56315ca9f823 Type: VDSAction group MANIPULATE_HOST with role type ADMIN
2015-08-19 00:15:36,718 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-49) [2b2de125] START, SetVdsStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 5a2d35d6
2015-08-19 00:15:37,788 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-49) [2b2de125] FINISH, SetVdsStatusVDSCommand, log id: 5a2d35d6
2015-08-19 00:15:38,228 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-7-thread-49) [2b2de125] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 80ce2151-3dbc-4945-834b-56315ca9f823 Type: VDS
2015-08-19 00:15:38,229 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetHaMaintenanceModeVDSCommand] (org.ovirt.thread.pool-7-thread-49) [2b2de125] START, SetHaMaintenanceModeVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823), log id: 1f37c090
2015-08-19 00:15:38,233 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetHaMaintenanceModeVDSCommand] (org.ovirt.thread.pool-7-thread-49) [2b2de125] FINISH, SetHaMaintenanceModeVDSCommand, log id: 1f37c090
2015-08-19 00:15:38,444 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-49) [2b2de125] Correlation ID: 2b2de125, Job ID: a4a05451-8fa1-4570-be77-ea9e1f1c7433, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_1 was switched to Maintenance mode by admin@internal.
2015-08-19 00:19:30,987 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-13) [15c9ab1f] Lock Acquired to object EngineLock [exclusiveLocks= key: 80ce2151-3dbc-4945-834b-56315ca9f823 value: VDS
, sharedLocks= ]
2015-08-19 00:19:31,393 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-43) [15c9ab1f] Running command: ActivateVdsCommand internal: false. Entities affected : ID: 80ce2151-3dbc-4945-834b-56315ca9f823 Type: VDSAction group MANIPULATE_HOST with role type ADMIN
2015-08-19 00:19:31,394 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-43) [15c9ab1f] Before acquiring lock in order to prevent monitoring for host hosted_engine_1 from data-center Default
2015-08-19 00:19:31,394 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-43) [15c9ab1f] Lock acquired, from now a monitoring of host will be skipped for host hosted_engine_1 from data-center Default
2015-08-19 00:19:31,498 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-43) [15c9ab1f] START, SetVdsStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 4d8ebf7d
2015-08-19 00:19:31,593 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-43) [15c9ab1f] FINISH, SetVdsStatusVDSCommand, log id: 4d8ebf7d
2015-08-19 00:19:31,711 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetHaMaintenanceModeVDSCommand] (org.ovirt.thread.pool-7-thread-43) START, SetHaMaintenanceModeVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823), log id: 70c5a46c
2015-08-19 00:19:31,714 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetHaMaintenanceModeVDSCommand] (org.ovirt.thread.pool-7-thread-43) FINISH, SetHaMaintenanceModeVDSCommand, log id: 70c5a46c
2015-08-19 00:19:31,715 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-43) Activate finished. Lock released. Monitoring can run now for host hosted_engine_1 from data-center Default
2015-08-19 00:19:31,884 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-43) Correlation ID: 15c9ab1f, Job ID: d247fd81-6186-4f0e-b07e-c8c0eeb1537f, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_1 was activated by admin@internal.
2015-08-19 00:19:32,076 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-43) Lock freed to object EngineLock [exclusiveLocks= key: 80ce2151-3dbc-4945-834b-56315ca9f823 value: VDS
, sharedLocks= ]
2015-08-19 00:19:33,206 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-48) START, GetHardwareInfoVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vds=Host[hosted_engine_1,80ce2151-3dbc-4945-834b-56315ca9f823]), log id: 3292eee1
2015-08-19 00:19:33,209 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, GetHardwareInfoVDSCommand, log id: 3292eee1
2015-08-19 00:19:33,476 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-48) [6fb921f4] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 80ce2151-3dbc-4945-834b-56315ca9f823 Type: VDS
2015-08-19 00:19:33,484 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-48) [703d7b91] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000010d Type: StoragePool
2015-08-19 00:19:33,678 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-48) [703d7b91] START, SetMOMPolicyParametersVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823), log id: 116caf09
2015-08-19 00:19:33,739 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-48) [703d7b91] FINISH, SetMOMPolicyParametersVDSCommand, log id: 116caf09
2015-08-19 00:19:33,893 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-48) [703d7b91] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host hosted_engine_1 was set to Up.
2015-08-19 00:19:33,895 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-48) [703d7b91] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host hosted_engine_1.
2015-08-19 00:19:34,080 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-48) [7b162587] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 80ce2151-3dbc-4945-834b-56315ca9f823 Type: VDS
2015-08-19 00:28:10,815 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false.
2015-08-19 00:28:11,013 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in.
2015-08-19 00:28:13,255 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-16) [4a84185d] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN
2015-08-19 00:28:13,439 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-16) [764eab57] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c Type: VDS
2015-08-19 00:28:13,538 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-16) [764eab57] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-00000000010d value: REGISTER_VDS
, sharedLocks= ]
2015-08-19 00:28:13,539 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-16) [764eab57] START, RemoveVdsVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c), log id: 6ef710e6
2015-08-19 00:28:13,539 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-16) [764eab57] FINISH, RemoveVdsVDSCommand, log id: 6ef710e6
2015-08-19 00:28:13,540 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-16) [764eab57] START, AddVdsVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c), log id: ad6754e
2015-08-19 00:28:13,540 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-16) [764eab57] AddVds - entered , starting logic to add VDS 6ebcfbc5-eb17-43e4-bd87-c08354402d1c
2015-08-19 00:28:13,540 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-16) [764eab57] AddVds - VDS 6ebcfbc5-eb17-43e4-bd87-c08354402d1c was added, will try to add it to the resource manager
2015-08-19 00:28:13,540 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-16) [764eab57] Entered VdsManager constructor
2015-08-19 00:28:13,545 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-16) [764eab57] Initialize vdsBroker (rhevhtest-3.redhat.com,54,321)
2015-08-19 00:28:13,546 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-16) [764eab57] VDS 6ebcfbc5-eb17-43e4-bd87-c08354402d1c was added to the Resource Manager
2015-08-19 00:28:13,546 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-16) [764eab57] FINISH, AddVdsVDSCommand, log id: ad6754e
2015-08-19 00:28:13,553 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-16) [764eab57] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host hosted_engine_3.
2015-08-19 00:28:13,838 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-18) [35004e41] Lock Acquired to object EngineLock [exclusiveLocks= key: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c value: VDS
, sharedLocks= ]
2015-08-19 00:28:13,930 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-16) [764eab57] Correlation ID: 4a84185d, Job ID: 48570963-9565-4c7f-b033-95c4e61123d3, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_3 was added by admin@internal.
2015-08-19 00:28:14,113 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-18) [35004e41] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c Type: VDS
2015-08-19 00:28:14,113 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-18) [35004e41] Before Installation host 6ebcfbc5-eb17-43e4-bd87-c08354402d1c, hosted_engine_3
2015-08-19 00:28:14,131 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-18) [35004e41] START, SetVdsStatusVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 3371f6b
2015-08-19 00:28:14,221 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-18) [35004e41] FINISH, SetVdsStatusVDSCommand, log id: 3371f6b
2015-08-19 00:28:14,332 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-7-thread-18) [35004e41] Installation rhevhtest-3.redhat.com: Connected to host rhevhtest-3.redhat.com with SSH key fingerprint: 04:a9:c9:29:dd:ca:c8:c0:e8:22:b9:0c:2e:20:a8:c4
2015-08-19 00:28:14,463 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-18) [35004e41] Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Connected to host rhevhtest-3.redhat.com with SSH key fingerprint: 04:a9:c9:29:dd:ca:c8:c0:e8:22:b9:0c:2e:20:a8:c4.
2015-08-19 00:28:14,477 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-7-thread-18) [35004e41] Installation of rhevhtest-3.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar
2015-08-19 00:28:14,482 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-7-thread-18) SSH execute root@rhevhtest-3.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True'
2015-08-19 00:28:15,126 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Stage: Initializing
2015-08-19 00:28:15,229 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Stage: Initializing.
2015-08-19 00:28:15,239 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Stage: Environment setup
2015-08-19 00:28:15,337 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Stage: Environment setup.
2015-08-19 00:28:15,337 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Stage: Environment packages setup
2015-08-19 00:28:15,429 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Stage: Environment packages setup.
2015-08-19 00:28:15,429 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Stage: Programs detection
2015-08-19 00:28:15,521 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Stage: Programs detection.
2015-08-19 00:28:15,521 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Stage: Environment customization
2015-08-19 00:28:15,621 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Stage: Environment customization.
2015-08-19 00:28:15,621 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Kdump unsupported
2015-08-19 00:28:15,721 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Kdump unsupported.
2015-08-19 00:28:15,728 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Host is hypervisor
2015-08-19 00:28:15,829 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Host is hypervisor.
2015-08-19 00:28:16,053 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20150819042815-94w8iu.log'
2015-08-19 00:28:16,254 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Logs at host located at: '/tmp/ovirt-host-deploy-20150819042815-94w8iu.log'.
2015-08-19 00:28:16,283 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host rhevhtest-3.redhat.com reports unique id 6EC9B800-7ECB-11E1-0000-E839355062A4
2015-08-19 00:28:16,292 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 6EC9B800-7ECB-11E1-0000-E839355062A4 to Host rhevhtest-3.redhat.com
2015-08-19 00:28:16,444 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Disabling Kdump integration
2015-08-19 00:28:16,537 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Disabling Kdump integration.
2015-08-19 00:28:16,547 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Initializing Host: hosted_engine_3
2015-08-19 00:28:16,550 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Stage: Setup validation
2015-08-19 00:28:16,645 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Stage: Setup validation.
2015-08-19 00:28:16,645 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Cannot validate host name settings, reason: resolved host does not match any of the local addresses
2015-08-19 00:28:16,795 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_3 installation in progress . Cannot validate host name settings, reason: resolved host does not match any of the local addresses.
2015-08-19 00:28:16,795 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Hardware supports virtualization
2015-08-19 00:28:16,895 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Hardware supports virtualization.
2015-08-19 00:28:16,895 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Stage: Transaction setup
2015-08-19 00:28:16,995 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Stage: Transaction setup.
2015-08-19 00:28:16,995 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Stage: Misc configuration
2015-08-19 00:28:17,095 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Stage: Misc configuration.
2015-08-19 00:28:17,095 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Stage: Package installation
2015-08-19 00:28:17,195 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Stage: Package installation.
2015-08-19 00:28:19,735 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Stage: Misc configuration
2015-08-19 00:28:19,843 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Stage: Misc configuration.
2015-08-19 00:28:19,843 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Setting time
2015-08-19 00:28:19,935 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Setting time.
2015-08-19 00:28:20,592 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Enrolling certificate
2015-08-19 00:28:20,692 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Enrolling certificate.
2015-08-19 00:28:21,876 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Stage: Transaction commit
2015-08-19 00:28:21,966 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Stage: Transaction commit.
2015-08-19 00:28:21,966 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Stage: Closing up
2015-08-19 00:28:22,066 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Stage: Closing up.
2015-08-19 00:28:26,842 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Stopping libvirtd
2015-08-19 00:28:27,059 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Stopping libvirtd.
2015-08-19 00:28:27,059 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Starting vdsm
2015-08-19 00:28:27,254 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Starting vdsm.
2015-08-19 00:28:36,841 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Stage: Pre-termination
2015-08-19 00:28:36,932 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Stage: Pre-termination.
2015-08-19 00:28:36,946 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20150819002836-rhevhtest-3.redhat.com-35004e41.log'
2015-08-19 00:28:37,149 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20150819002836-rhevhtest-3.redhat.com-35004e41.log'.
2015-08-19 00:28:37,304 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation rhevhtest-3.redhat.com: Stage: Termination
2015-08-19 00:28:37,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 35004e41, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Stage: Termination.
2015-08-19 00:28:37,508 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-3.redhat.com/10.66.65.29
2015-08-19 00:28:37,621 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-7-thread-18) [35004e41] Engine managed to communicate with VDSM agent on host hosted_engine_3
2015-08-19 00:28:37,963 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-7-thread-18) [30c419c0] Running command: SetupNetworksCommand internal: true. Entities affected : ID: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN
2015-08-19 00:28:37,966 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-7-thread-18) [30c419c0] START, SetupNetworksVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c, force=false, checkConnectivity=true, conectivityTimeout=120,
networks=[rhevm {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-00000000010d, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}],
bonds=[],
interfaces=[eno1 {id=4360a6ea-cf30-4694-aaec-5a2b17f767e4, vdsId=6ebcfbc5-eb17-43e4-bd87-c08354402d1c, macAddress=e8:39:35:50:62:a4, networkName=rhevm, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.66.65.29, subnet=255.255.254.0, gateway=null, mtu=1500, bridged=false, speed=100, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}],
removedNetworks=[],
removedBonds=[]), log id: 4214c288
2015-08-19 00:28:37,967 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-7-thread-18) [30c419c0] FINISH, SetupNetworksVDSCommand, log id: 4214c288
2015-08-19 00:28:37,967 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-3.redhat.com/10.66.65.29
2015-08-19 00:28:48,138 WARN [org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker] (ResponseWorker) Exception thrown during message processing
2015-08-19 00:28:48,138 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-3.redhat.com/10.66.65.29
2015-08-19 00:28:48,500 INFO [org.ovirt.engine.core.bll.network.host.CommitNetworkChangesCommand] (org.ovirt.thread.pool-7-thread-18) [5289045d] Running command: CommitNetworkChangesCommand internal: true. Entities affected : ID: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN
2015-08-19 00:28:48,501 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-7-thread-18) [5289045d] START, SetSafeNetworkConfigVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c), log id: 444ae6d4
2015-08-19 00:28:51,110 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-7-thread-18) [5289045d] FINISH, SetSafeNetworkConfigVDSCommand, log id: 444ae6d4
2015-08-19 00:28:51,305 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-18) [5289045d] Correlation ID: 5289045d, Call Stack: null, Custom Event ID: -1, Message: Network changes were saved on host hosted_engine_3
2015-08-19 00:28:51,306 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-18) [5289045d] START, SetVdsStatusVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c, status=Initializing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: ee90950
2015-08-19 00:28:51,413 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-18) [5289045d] FINISH, SetVdsStatusVDSCommand, log id: ee90950
2015-08-19 00:28:51,413 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-18) [5289045d] After Installation host hosted_engine_3, oVirtNode
2015-08-19 00:28:51,505 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-18) [5289045d] Correlation ID: 35004e41, Job ID: 48570963-9565-4c7f-b033-95c4e61123d3, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_3 installed
2015-08-19 00:28:51,596 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-18) [5289045d] Lock freed to object EngineLock [exclusiveLocks= key: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c value: VDS
, sharedLocks= ]
2015-08-19 00:28:52,587 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-45) START, GetHardwareInfoVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c, vds=Host[hosted_engine_3,6ebcfbc5-eb17-43e4-bd87-c08354402d1c]), log id: 3921a239
2015-08-19 00:28:52,804 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-45) FINISH, GetHardwareInfoVDSCommand, log id: 3921a239
2015-08-19 00:28:53,072 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-45) [64599a85] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c Type: VDS
2015-08-19 00:28:53,098 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-45) [64599a85] START, GetHardwareInfoVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c, vds=Host[hosted_engine_3,6ebcfbc5-eb17-43e4-bd87-c08354402d1c]), log id: 5ef1c730
2015-08-19 00:28:53,102 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-45) [64599a85] FINISH, GetHardwareInfoVDSCommand, log id: 5ef1c730
2015-08-19 00:28:53,211 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-45) [114f9d78] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000010d Type: StoragePool
2015-08-19 00:28:53,314 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-45) [114f9d78] START, SetMOMPolicyParametersVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c), log id: 1ac30dcd
2015-08-19 00:28:53,375 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-45) [114f9d78] FINISH, SetMOMPolicyParametersVDSCommand, log id: 1ac30dcd
2015-08-19 00:28:53,479 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) [114f9d78] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host hosted_engine_3 was set to Up.
2015-08-19 00:28:53,483 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) [114f9d78] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host hosted_engine_3.
2015-08-19 00:28:53,789 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-45) [2d62a309] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c Type: VDS
2015-08-19 00:28:53,791 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-45) [7c67092e] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c Type: VDS
2015-08-19 00:28:54,619 INFO [org.ovirt.engine.core.bll.UpdateVdsGroupCommand] (ajp-/127.0.0.1:8702-28) [db72637] Running command: UpdateVdsGroupCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group EDIT_CLUSTER_CONFIGURATION with role type ADMIN
2015-08-19 00:28:54,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-28) [db72637] Correlation ID: db72637, Call Stack: null, Custom Event ID: -1, Message: Host cluster Default was updated by admin@internal
2015-08-19 00:29:59,025 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp-/127.0.0.1:8702-7) [8e2d817] Lock Acquired to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 00:29:59,062 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-7) [8e2d817] Candidate host hosted_engine_1 (80ce2151-3dbc-4945-834b-56315ca9f823) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory
2015-08-19 00:29:59,229 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-7) [8e2d817] Candidate host hosted_engine_3 (6ebcfbc5-eb17-43e4-bd87-c08354402d1c) was filtered out by VAR__FILTERTYPE__INTERNAL filter HA
2015-08-19 00:29:59,703 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-15) [8e2d817] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group MIGRATE_VM with role type USER, ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_VM with role type USER
2015-08-19 00:29:59,709 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (org.ovirt.thread.pool-7-thread-15) [8e2d817] Candidate host hosted_engine_3 (6ebcfbc5-eb17-43e4-bd87-c08354402d1c) was filtered out by VAR__FILTERTYPE__INTERNAL filter HA (correlation id: 8e2d817)
2015-08-19 00:29:59,819 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-15) [8e2d817] Correlation ID: 8e2d817, Job ID: 48038c52-9761-4fdc-b0be-29e4f7e7647d, Call Stack: null, Custom Event ID: -1, Message: Migration failed, No available host found (VM: HostedEngine, Source: hosted_engine_1).
2015-08-19 00:29:59,911 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-15) [8e2d817] Lock freed to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 00:30:31,888 INFO [org.ovirt.engine.docs.utils.servlet.ContextSensitiveHelpMappingServlet] (ajp-/127.0.0.1:8702-18) Successfully read CSH mapping file '/usr/share/doc/rhevm-doc/manual/en-US/csh.conf.d/webadmin/10-webadmin-en-US.json'
2015-08-19 00:30:58,839 INFO [org.ovirt.engine.docs.utils.servlet.ContextSensitiveHelpMappingServlet] (ajp-/127.0.0.1:8702-25) Successfully read CSH mapping file '/usr/share/doc/rhevm-doc/manual/en-US/csh.conf.d/webadmin/10-webadmin-en-US.json'
2015-08-19 00:31:52,574 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp-/127.0.0.1:8702-29) [7cade60b] Lock Acquired to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 00:31:52,605 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-29) [7cade60b] Candidate host hosted_engine_1 (80ce2151-3dbc-4945-834b-56315ca9f823) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory
2015-08-19 00:31:53,135 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-25) [7cade60b] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group MIGRATE_VM with role type USER, ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_VM with role type USER
2015-08-19 00:31:53,271 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-25) [7cade60b] START, MigrateVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=6ebcfbc5-eb17-43e4-bd87-c08354402d1c, dstHost=rhevhtest-3.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 6246f9b9
2015-08-19 00:31:53,271 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-25) [7cade60b] START, MigrateBrokerVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=6ebcfbc5-eb17-43e4-bd87-c08354402d1c, dstHost=rhevhtest-3.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 1a39cf36
2015-08-19 00:31:53,277 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-25) [7cade60b] FINISH, MigrateBrokerVDSCommand, log id: 1a39cf36
2015-08-19 00:31:53,393 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-25) [7cade60b] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 6246f9b9
2015-08-19 00:31:53,501 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-25) [7cade60b] Correlation ID: 7cade60b, Job ID: 99ed1f75-447a-40aa-addf-998048bf7789, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_3, User: admin@internal).
2015-08-19 00:31:54,217 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) VM HostedEngine c5312b39-316c-4906-bc20-df2a472bda1f moved from MigratingFrom --> Up
2015-08-19 00:31:54,217 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Adding VM c5312b39-316c-4906-bc20-df2a472bda1f to re-run list
2015-08-19 00:31:54,510 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Rerun vm c5312b39-316c-4906-bc20-df2a472bda1f. Called from vds hosted_engine_1
2015-08-19 00:31:54,531 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-26) START, MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f), log id: d1f8d36
2015-08-19 00:31:54,533 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-26) Failed in MigrateStatusVDS method
2015-08-19 00:31:54,533 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-26) Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value
StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=10, mMessage=Could not connect to peer VDS]]
2015-08-19 00:31:54,533 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-26) HostName = hosted_engine_1
2015-08-19 00:31:54,533 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-26) Command MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Could not connect to peer VDS, code = 10
2015-08-19 00:31:54,533 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-26) FINISH, MigrateStatusVDSCommand, log id: d1f8d36
2015-08-19 00:31:54,893 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-26) Correlation ID: 7cade60b, Job ID: 99ed1f75-447a-40aa-addf-998048bf7789, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Could not connect to peer host (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_3).
2015-08-19 00:31:55,235 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-26) Lock freed to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 00:36:00,031 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-94) Attempting to update VMs/Templates Ovf.
2015-08-19 00:49:25,324 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp-/127.0.0.1:8702-26) [216f331d] Lock Acquired to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 00:49:25,352 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-26) [216f331d] Candidate host hosted_engine_1 (80ce2151-3dbc-4945-834b-56315ca9f823) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory
2015-08-19 00:49:25,759 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-14) [216f331d] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group MIGRATE_VM with role type USER, ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_VM with role type USER
2015-08-19 00:49:25,961 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-14) [216f331d] START, MigrateVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=6ebcfbc5-eb17-43e4-bd87-c08354402d1c, dstHost=rhevhtest-3.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 4a06420e
2015-08-19 00:49:25,961 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-14) [216f331d] START, MigrateBrokerVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=6ebcfbc5-eb17-43e4-bd87-c08354402d1c, dstHost=rhevhtest-3.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 71f9081c
2015-08-19 00:49:25,967 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-14) [216f331d] FINISH, MigrateBrokerVDSCommand, log id: 71f9081c
2015-08-19 00:49:26,133 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-14) [216f331d] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 4a06420e
2015-08-19 00:49:26,325 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-14) [216f331d] Correlation ID: 216f331d, Job ID: b3524d9e-0fd1-4b69-a4f5-6f545b263dae, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_3, User: admin@internal).
2015-08-19 00:49:28,384 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) VM HostedEngine c5312b39-316c-4906-bc20-df2a472bda1f moved from MigratingFrom --> Up
2015-08-19 00:49:28,385 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Adding VM c5312b39-316c-4906-bc20-df2a472bda1f to re-run list
2015-08-19 00:49:28,782 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Rerun vm c5312b39-316c-4906-bc20-df2a472bda1f. Called from vds hosted_engine_1
2015-08-19 00:49:28,802 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-17) START, MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f), log id: 1f848d51
2015-08-19 00:49:28,805 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-17) Failed in MigrateStatusVDS method
2015-08-19 00:49:28,805 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-17) Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value
StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=10, mMessage=Could not connect to peer VDS]]
2015-08-19 00:49:28,805 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-17) HostName = hosted_engine_1
2015-08-19 00:49:28,805 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-17) Command MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Could not connect to peer VDS, code = 10
2015-08-19 00:49:28,805 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-17) FINISH, MigrateStatusVDSCommand, log id: 1f848d51
2015-08-19 00:49:29,369 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-17) Correlation ID: 216f331d, Job ID: b3524d9e-0fd1-4b69-a4f5-6f545b263dae, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Could not connect to peer host (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_3).
2015-08-19 00:49:29,565 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-17) Lock freed to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 01:34:10,478 INFO [org.ovirt.engine.core.services.RegisterServlet] (ajp-/127.0.0.1:8702-16) Registration request: source='10.66.9.134', secured='true', address='10.66.9.134', vdsPort=54321, name='test.redhat.com', uniqueId='44454C4C-3200-104D-8035-C4C04F4E3258_00:23:ae:8f:f3:ea'
2015-08-19 01:34:10,595 INFO [org.ovirt.engine.core.bll.RegisterVdsQuery] (ajp-/127.0.0.1:8702-16) Running Command: RegisterVds
2015-08-19 01:34:10,680 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-16) [4e73e8da] Running command: AddVdsCommand internal: true. Entities affected : ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN
2015-08-19 01:34:10,819 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-16) [176f365e] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 5be4d9f4-cb4e-4148-aec1-f697527515ef Type: VDS
2015-08-19 01:34:10,926 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-16) [176f365e] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-00000000010d value: REGISTER_VDS
, sharedLocks= ]
2015-08-19 01:34:10,927 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-16) [176f365e] START, RemoveVdsVDSCommand(HostName = test.redhat.com, HostId = 5be4d9f4-cb4e-4148-aec1-f697527515ef), log id: 6ef34367
2015-08-19 01:34:10,927 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-16) [176f365e] FINISH, RemoveVdsVDSCommand, log id: 6ef34367
2015-08-19 01:34:10,927 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-16) [176f365e] START, AddVdsVDSCommand(HostName = test.redhat.com, HostId = 5be4d9f4-cb4e-4148-aec1-f697527515ef), log id: 4bc80bf9
2015-08-19 01:34:10,927 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-16) [176f365e] AddVds - entered , starting logic to add VDS 5be4d9f4-cb4e-4148-aec1-f697527515ef
2015-08-19 01:34:10,927 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-16) [176f365e] AddVds - VDS 5be4d9f4-cb4e-4148-aec1-f697527515ef was added, will try to add it to the resource manager
2015-08-19 01:34:10,928 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-16) [176f365e] Entered VdsManager constructor
2015-08-19 01:34:10,933 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-16) [176f365e] Initialize vdsBroker (10.66.9.134,54,321)
2015-08-19 01:34:10,933 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-16) [176f365e] VDS 5be4d9f4-cb4e-4148-aec1-f697527515ef was added to the Resource Manager
2015-08-19 01:34:10,933 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-16) [176f365e] FINISH, AddVdsVDSCommand, log id: 4bc80bf9
2015-08-19 01:34:10,946 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-16) [176f365e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host test.redhat.com.
2015-08-19 01:34:11,159 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-16) [176f365e] Correlation ID: 4e73e8da, Call Stack: null, Custom Event ID: -1, Message: Host test.redhat.com was added by null@N/A.
2015-08-19 01:34:11,159 INFO [org.ovirt.engine.core.bll.RegisterVdsQuery] (ajp-/127.0.0.1:8702-16) [176f365e] RegisterVdsQuery::Register - Registered a new VDS pending approval - Name: test.redhat.com, Hostname: 10.66.9.134, UniqueID: 44454C4C-3200-104D-8035-C4C04F4E3258
2015-08-19 01:34:11,367 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-16) [176f365e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host test.redhat.com registered.
2015-08-19 01:34:39,504 INFO [org.ovirt.engine.docs.utils.servlet.ContextSensitiveHelpMappingServlet] (ajp-/127.0.0.1:8702-23) Successfully read CSH mapping file '/usr/share/doc/rhevm-doc/manual/en-US/csh.conf.d/webadmin/10-webadmin-en-US.json'
2015-08-19 01:35:01,286 INFO [org.ovirt.engine.docs.utils.servlet.ContextSensitiveHelpMappingServlet] (ajp-/127.0.0.1:8702-24) Successfully read CSH mapping file '/usr/share/doc/rhevm-doc/manual/en-US/csh.conf.d/webadmin/10-webadmin-en-US.json'
2015-08-19 01:35:28,097 INFO [org.ovirt.engine.core.bll.storage.AddEmptyStoragePoolCommand] (ajp-/127.0.0.1:8702-7) [1b86b9b8] Running command: AddEmptyStoragePoolCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_POOL with role type ADMIN
2015-08-19 01:35:28,453 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [1b86b9b8] Correlation ID: 1b86b9b8, Call Stack: null, Custom Event ID: -1, Message: Data Center test_abc, Compatibility Version 3.5 and Quota Type DISABLED was added by admin@internal
2015-08-19 01:35:56,421 INFO [org.ovirt.engine.core.bll.AddVdsGroupCommand] (ajp-/127.0.0.1:8702-19) [100c75bc] Running command: AddVdsGroupCommand internal: false. Entities affected : ID: d609f34a-1231-4023-a5d3-521e6bba4bde Type: StoragePoolAction group CREATE_CLUSTER with role type ADMIN
2015-08-19 01:35:56,658 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-19) [100c75bc] Correlation ID: 100c75bc, Call Stack: null, Custom Event ID: -1, Message: Cluster test_abc was added by admin@internal
2015-08-19 01:36:00,032 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-64) Attempting to update VMs/Templates Ovf.
2015-08-19 01:36:09,549 INFO [org.ovirt.engine.core.bll.ChangeVDSClusterCommand] (ajp-/127.0.0.1:8702-27) [81b83d5] Lock Acquired to object EngineLock [exclusiveLocks= key: 5be4d9f4-cb4e-4148-aec1-f697527515ef value: VDS
, sharedLocks= ]
2015-08-19 01:36:09,550 INFO [org.ovirt.engine.core.bll.ChangeVDSClusterCommand] (ajp-/127.0.0.1:8702-27) [81b83d5] Running command: ChangeVDSClusterCommand internal: false. Entities affected : ID: 34a23dd2-561f-417d-bb89-90bf89fbf966 Type: VdsGroupsAction group EDIT_HOST_CONFIGURATION with role type ADMIN
2015-08-19 01:36:09,554 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-27) [81b83d5] START, RemoveVdsVDSCommand(HostName = test.redhat.com, HostId = 5be4d9f4-cb4e-4148-aec1-f697527515ef), log id: 152682c9
2015-08-19 01:36:09,554 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-27) [81b83d5] vdsManager::disposing
2015-08-19 01:36:09,557 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-27) [81b83d5] FINISH, RemoveVdsVDSCommand, log id: 152682c9
2015-08-19 01:36:09,561 WARN [org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker] (ResponseWorker) Exception thrown during message processing
2015-08-19 01:36:09,561 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-27) [81b83d5] START, AddVdsVDSCommand(HostName = test.redhat.com, HostId = 5be4d9f4-cb4e-4148-aec1-f697527515ef), log id: f3aca23
2015-08-19 01:36:09,562 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-27) [81b83d5] AddVds - entered , starting logic to add VDS 5be4d9f4-cb4e-4148-aec1-f697527515ef
2015-08-19 01:36:09,562 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-27) [81b83d5] AddVds - VDS 5be4d9f4-cb4e-4148-aec1-f697527515ef was added, will try to add it to the resource manager
2015-08-19 01:36:09,562 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-27) [81b83d5] Entered VdsManager constructor
2015-08-19 01:36:09,567 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-27) [81b83d5] Initialize vdsBroker (10.66.9.134,54,321)
2015-08-19 01:36:09,567 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-27) [81b83d5] VDS 5be4d9f4-cb4e-4148-aec1-f697527515ef was added to the Resource Manager
2015-08-19 01:36:09,570 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-27) [81b83d5] FINISH, AddVdsVDSCommand, log id: f3aca23
2015-08-19 01:36:09,672 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-27) [6fbbc374] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 5be4d9f4-cb4e-4148-aec1-f697527515ef Type: VDS
2015-08-19 01:36:09,766 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-27) [6fbbc374] Lock freed to object EngineLock [exclusiveLocks= key: d609f34a-1231-4023-a5d3-521e6bba4bde value: REGISTER_VDS
, sharedLocks= ]
2015-08-19 01:36:09,881 INFO [org.ovirt.engine.core.bll.network.host.PersistentSetupNetworksCommand] (org.ovirt.thread.pool-7-thread-5) [56a8e8ab] Running command: PersistentSetupNetworksCommand internal: true. Entities affected : ID: 5be4d9f4-cb4e-4148-aec1-f697527515ef Type: VDS
2015-08-19 01:36:09,883 WARN [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-7-thread-5) [56a8e8ab] CanDoAction of action SetupNetworks failed for user admin@internal. Reasons: VAR__ACTION__SETUP,VAR__TYPE__NETWORKS,VAR__HOST_STATUS__UP_MAINTENANCE_OR_NON_OPERATIONAL,ACTION_TYPE_FAILED_VDS_STATUS_ILLEGAL
2015-08-19 01:36:10,078 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-27) [6fbbc374] Correlation ID: 81b83d5, Call Stack: null, Custom Event ID: -1, Message: Host test.redhat.com configuration was updated by admin@internal.
2015-08-19 01:36:10,078 INFO [org.ovirt.engine.core.bll.ChangeVDSClusterCommand] (ajp-/127.0.0.1:8702-27) [6fbbc374] Lock freed to object EngineLock [exclusiveLocks= key: 5be4d9f4-cb4e-4148-aec1-f697527515ef value: VDS
, sharedLocks= ]
2015-08-19 01:36:10,139 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-9) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic
2015-08-19 01:36:10,139 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-9) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic
2015-08-19 01:36:10,139 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-9) [7c0aff85] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 5be4d9f4-cb4e-4148-aec1-f697527515ef Type: VDSAction group EDIT_HOST_CONFIGURATION with role type ADMIN
2015-08-19 01:36:10,316 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-9) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic
2015-08-19 01:36:10,316 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-9) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic
2015-08-19 01:36:10,318 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [7c0aff85] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host test.redhat.com.
2015-08-19 01:36:10,516 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [7c0aff85] Correlation ID: 7c0aff85, Call Stack: null, Custom Event ID: -1, Message: Host test.redhat.com configuration was updated by admin@internal.
2015-08-19 01:36:11,124 INFO [org.ovirt.engine.core.bll.ApproveVdsCommand] (org.ovirt.thread.pool-7-thread-10) [4f5b5db7] Running command: ApproveVdsCommand internal: false. Entities affected : ID: 5be4d9f4-cb4e-4148-aec1-f697527515ef Type: VDSAction group CREATE_HOST with role type ADMIN
2015-08-19 01:36:11,124 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-10) [4f5b5db7] Before Installation host 5be4d9f4-cb4e-4148-aec1-f697527515ef, test.redhat.com
2015-08-19 01:36:11,140 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-10) [4f5b5db7] START, SetVdsStatusVDSCommand(HostName = test.redhat.com, HostId = 5be4d9f4-cb4e-4148-aec1-f697527515ef, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 63dc089
2015-08-19 01:36:11,240 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-10) [4f5b5db7] FINISH, SetVdsStatusVDSCommand, log id: 63dc089
2015-08-19 01:36:11,523 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-7-thread-10) [4f5b5db7] Installation 10.66.9.134: Connected to host 10.66.9.134 with SSH key fingerprint: db:5d:2d:10:86:dc:7b:2e:34:22:d0:76:41:90:9a:58
2015-08-19 01:36:11,640 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-10) [4f5b5db7] Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Connected to host 10.66.9.134 with SSH key fingerprint: db:5d:2d:10:86:dc:7b:2e:34:22:d0:76:41:90:9a:58.
2015-08-19 01:36:11,659 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-7-thread-10) [4f5b5db7] Installation of 10.66.9.134. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar
2015-08-19 01:36:11,663 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-7-thread-10) SSH execute root@10.66.9.134 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True'
2015-08-19 01:36:12,568 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-21) Initializing Host: test.redhat.com
2015-08-19 01:36:12,734 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Stage: Initializing
2015-08-19 01:36:12,856 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Stage: Initializing.
2015-08-19 01:36:12,871 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Stage: Environment setup
2015-08-19 01:36:12,981 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Stage: Environment setup.
2015-08-19 01:36:12,981 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Stage: Environment packages setup
2015-08-19 01:36:13,098 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Stage: Environment packages setup.
2015-08-19 01:36:13,098 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Stage: Programs detection
2015-08-19 01:36:13,208 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Stage: Programs detection.
2015-08-19 01:36:13,208 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Stage: Environment customization
2015-08-19 01:36:13,388 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Stage: Environment customization.
2015-08-19 01:36:13,388 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Kdump unsupported
2015-08-19 01:36:13,497 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Kdump unsupported.
2015-08-19 01:36:13,512 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Host is hypervisor
2015-08-19 01:36:13,622 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Host is hypervisor.
2015-08-19 01:36:13,733 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Logs at host located at: '/tmp/ovirt-host-deploy-20150819053611-cib030.log'
2015-08-19 01:36:13,906 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Logs at host located at: '/tmp/ovirt-host-deploy-20150819053611-cib030.log'.
2015-08-19 01:36:13,926 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host 10.66.9.134 reports unique id 44454C4C-3200-104D-8035-C4C04F4E3258
2015-08-19 01:36:13,935 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 44454C4C-3200-104D-8035-C4C04F4E3258 to Host 10.66.9.134
2015-08-19 01:36:14,141 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Disabling Kdump integration
2015-08-19 01:36:14,231 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Disabling Kdump integration.
2015-08-19 01:36:14,242 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Stage: Setup validation
2015-08-19 01:36:14,431 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Stage: Setup validation.
2015-08-19 01:36:14,431 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Cannot validate host name settings, reason: resolved host does not match any of the local addresses
2015-08-19 01:36:14,522 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Host test.redhat.com installation in progress . Cannot validate host name settings, reason: resolved host does not match any of the local addresses.
2015-08-19 01:36:14,522 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Hardware supports virtualization
2015-08-19 01:36:14,614 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Hardware supports virtualization.
2015-08-19 01:36:14,614 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Stage: Transaction setup
2015-08-19 01:36:14,714 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Stage: Transaction setup.
2015-08-19 01:36:14,714 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Stage: Misc configuration
2015-08-19 01:36:14,822 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Stage: Misc configuration.
2015-08-19 01:36:14,822 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Stage: Package installation
2015-08-19 01:36:14,955 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Stage: Package installation.
2015-08-19 01:36:14,955 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Stage: Misc configuration
2015-08-19 01:36:15,122 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Stage: Misc configuration.
2015-08-19 01:36:15,122 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Setting time
2015-08-19 01:36:15,230 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Setting time.
2015-08-19 01:36:16,164 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Enrolling certificate
2015-08-19 01:36:16,271 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Enrolling certificate.
2015-08-19 01:36:17,468 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Stage: Transaction commit
2015-08-19 01:36:17,679 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Stage: Transaction commit.
2015-08-19 01:36:17,679 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Stage: Closing up
2015-08-19 01:36:17,788 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Stage: Closing up.
2015-08-19 01:36:22,287 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Stopping libvirtd
2015-08-19 01:36:22,392 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Stopping libvirtd.
2015-08-19 01:36:22,454 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Starting vdsm
2015-08-19 01:36:22,785 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Starting vdsm.
2015-08-19 01:36:41,520 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Stage: Pre-termination
2015-08-19 01:36:41,620 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Stage: Pre-termination.
2015-08-19 01:36:41,627 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20150819013641-10.66.9.134-4f5b5db7.log'
2015-08-19 01:36:41,720 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20150819013641-10.66.9.134-4f5b5db7.log'.
2015-08-19 01:36:41,924 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.66.9.134: Stage: Termination
2015-08-19 01:36:42,020 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 4f5b5db7, Call Stack: null, Custom Event ID: -1, Message: Installing Host test.redhat.com. Stage: Termination.
2015-08-19 01:36:42,020 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.66.9.134
2015-08-19 01:36:42,148 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-7-thread-10) [4f5b5db7] Engine managed to communicate with VDSM agent on host test.redhat.com
2015-08-19 01:36:42,645 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-7-thread-10) [4f5b5db7] The management network rhevm is already configured on host test.redhat.com
2015-08-19 01:36:42,646 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-10) [4f5b5db7] START, SetVdsStatusVDSCommand(HostName = test.redhat.com, HostId = 5be4d9f4-cb4e-4148-aec1-f697527515ef, status=Initializing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 5bf1f723
2015-08-19 01:36:42,745 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-10) [4f5b5db7] FINISH, SetVdsStatusVDSCommand, log id: 5bf1f723
2015-08-19 01:36:42,745 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-10) [4f5b5db7] After Installation host test.redhat.com, oVirtNode
2015-08-19 01:36:42,750 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-10) [4f5b5db7] START, SetVdsStatusVDSCommand(HostName = test.redhat.com, HostId = 5be4d9f4-cb4e-4148-aec1-f697527515ef, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 77317cd4
2015-08-19 01:36:42,944 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-10) [4f5b5db7] FINISH, SetVdsStatusVDSCommand, log id: 77317cd4
2015-08-19 01:36:43,053 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-10) [4f5b5db7] Correlation ID: 4f5b5db7, Job ID: 045b37af-a4c1-4445-b60e-84001c14c5f2, Call Stack: null, Custom Event ID: -1, Message: Host test.redhat.com was successfully approved by user admin@internal.
2015-08-19 01:36:45,618 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-100) START, GetHardwareInfoVDSCommand(HostName = test.redhat.com, HostId = 5be4d9f4-cb4e-4148-aec1-f697527515ef, vds=Host[test.redhat.com,5be4d9f4-cb4e-4148-aec1-f697527515ef]), log id: 1d24f357
2015-08-19 01:36:45,709 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, GetHardwareInfoVDSCommand, log id: 1d24f357
2015-08-19 01:36:45,994 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-100) [78c075f4] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 5be4d9f4-cb4e-4148-aec1-f697527515ef Type: VDS
2015-08-19 01:36:46,131 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-100) [78c075f4] START, GetHardwareInfoVDSCommand(HostName = test.redhat.com, HostId = 5be4d9f4-cb4e-4148-aec1-f697527515ef, vds=Host[test.redhat.com,5be4d9f4-cb4e-4148-aec1-f697527515ef]), log id: 4707dea9
2015-08-19 01:36:46,137 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-100) [78c075f4] FINISH, GetHardwareInfoVDSCommand, log id: 4707dea9
2015-08-19 01:36:46,243 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-100) [23eeeb55] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: d609f34a-1231-4023-a5d3-521e6bba4bde Type: StoragePool
2015-08-19 01:36:46,348 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-100) [23eeeb55] START, SetMOMPolicyParametersVDSCommand(HostName = test.redhat.com, HostId = 5be4d9f4-cb4e-4148-aec1-f697527515ef), log id: 1d337bee
2015-08-19 01:36:46,452 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-100) [23eeeb55] FINISH, SetMOMPolicyParametersVDSCommand, log id: 1d337bee
2015-08-19 01:36:46,551 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-100) [23eeeb55] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host test.redhat.com was set to Up.
2015-08-19 01:36:46,553 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-100) [23eeeb55] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host test.redhat.com.
2015-08-19 01:36:46,755 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-100) [4670be02] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 5be4d9f4-cb4e-4148-aec1-f697527515ef Type: VDS
2015-08-19 01:36:46,756 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-100) [5c004211] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 5be4d9f4-cb4e-4148-aec1-f697527515ef Type: VDS
2015-08-19 02:05:06,811 INFO [org.ovirt.engine.docs.utils.servlet.ContextSensitiveHelpMappingServlet] (ajp-/127.0.0.1:8702-19) Successfully read CSH mapping file '/usr/share/doc/rhevm-doc/manual/en-US/csh.conf.d/webadmin/10-webadmin-en-US.json'
2015-08-19 02:05:20,917 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp-/127.0.0.1:8702-26) [4880be4c] Lock Acquired to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 02:05:20,964 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-26) [4880be4c] Candidate host hosted_engine_1 (80ce2151-3dbc-4945-834b-56315ca9f823) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory
2015-08-19 02:05:21,742 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-1) [4880be4c] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group MIGRATE_VM with role type USER, ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_VM with role type USER
2015-08-19 02:05:22,028 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-1) [4880be4c] START, MigrateVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=bed31fc7-9abc-426f-859c-3a2df7d40ee1, dstHost=rhevhtest-2.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 1cf0680e
2015-08-19 02:05:22,029 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-1) [4880be4c] START, MigrateBrokerVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=bed31fc7-9abc-426f-859c-3a2df7d40ee1, dstHost=rhevhtest-2.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 4f7d9825
2015-08-19 02:05:22,034 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-1) [4880be4c] FINISH, MigrateBrokerVDSCommand, log id: 4f7d9825
2015-08-19 02:05:22,228 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-1) [4880be4c] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 1cf0680e
2015-08-19 02:05:22,345 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-1) [4880be4c] Correlation ID: 4880be4c, Job ID: 837a9426-19bd-4592-9a2f-62cc0b8c889c, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_2, User: admin@internal).
2015-08-19 02:05:23,292 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) VM HostedEngine c5312b39-316c-4906-bc20-df2a472bda1f moved from MigratingFrom --> Up
2015-08-19 02:05:23,292 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) Adding VM c5312b39-316c-4906-bc20-df2a472bda1f to re-run list
2015-08-19 02:05:23,773 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) Rerun vm c5312b39-316c-4906-bc20-df2a472bda1f. Called from vds hosted_engine_1
2015-08-19 02:05:23,792 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-6) START, MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f), log id: 69ccadee
2015-08-19 02:05:23,795 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-6) Failed in MigrateStatusVDS method
2015-08-19 02:05:23,795 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-6) Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value
StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=10, mMessage=Could not connect to peer VDS]]
2015-08-19 02:05:23,795 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-6) HostName = hosted_engine_1
2015-08-19 02:05:23,795 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-6) Command MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Could not connect to peer VDS, code = 10
2015-08-19 02:05:23,795 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-6) FINISH, MigrateStatusVDSCommand, log id: 69ccadee
2015-08-19 02:05:24,087 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-6) Correlation ID: 4880be4c, Job ID: 837a9426-19bd-4592-9a2f-62cc0b8c889c, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Could not connect to peer host (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_2).
2015-08-19 02:05:24,186 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-6) Lock freed to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 02:14:56,849 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp-/127.0.0.1:8702-3) [505ebae] Lock Acquired to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 02:14:56,879 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-3) [505ebae] Candidate host hosted_engine_1 (80ce2151-3dbc-4945-834b-56315ca9f823) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory
2015-08-19 02:14:57,446 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-35) [505ebae] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group MIGRATE_VM with role type USER, ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_VM with role type USER
2015-08-19 02:14:57,578 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-35) [505ebae] START, MigrateVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=bed31fc7-9abc-426f-859c-3a2df7d40ee1, dstHost=rhevhtest-2.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 3b150007
2015-08-19 02:14:57,579 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-35) [505ebae] START, MigrateBrokerVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=bed31fc7-9abc-426f-859c-3a2df7d40ee1, dstHost=rhevhtest-2.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 637e6a78
2015-08-19 02:14:57,589 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-35) [505ebae] FINISH, MigrateBrokerVDSCommand, log id: 637e6a78
2015-08-19 02:14:57,693 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-35) [505ebae] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 3b150007
2015-08-19 02:14:57,725 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) VM HostedEngine c5312b39-316c-4906-bc20-df2a472bda1f moved from MigratingFrom --> Up
2015-08-19 02:14:57,725 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) Adding VM c5312b39-316c-4906-bc20-df2a472bda1f to re-run list
2015-08-19 02:14:57,821 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-35) [505ebae] Correlation ID: 505ebae, Job ID: 511b7070-42e9-49f4-8c1a-7d91a85e3456, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_2, User: admin@internal).
2015-08-19 02:14:58,156 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) Rerun vm c5312b39-316c-4906-bc20-df2a472bda1f. Called from vds hosted_engine_1
2015-08-19 02:14:58,158 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-20) START, MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f), log id: 554dc534
2015-08-19 02:14:58,161 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-20) Failed in MigrateStatusVDS method
2015-08-19 02:14:58,161 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-20) Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value
StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=10, mMessage=Could not connect to peer VDS]]
2015-08-19 02:14:58,161 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-20) HostName = hosted_engine_1
2015-08-19 02:14:58,161 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-20) Command MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Could not connect to peer VDS, code = 10
2015-08-19 02:14:58,161 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-20) FINISH, MigrateStatusVDSCommand, log id: 554dc534
2015-08-19 02:14:58,621 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-20) Correlation ID: 505ebae, Job ID: 511b7070-42e9-49f4-8c1a-7d91a85e3456, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Could not connect to peer host (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_2).
2015-08-19 02:14:58,738 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-20) Lock freed to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 02:36:00,033 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf.
2015-08-19 03:23:03,140 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-7-thread-39) [19610c8] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c Type: VDSAction group MANIPULATE_HOST with role type ADMIN
2015-08-19 03:23:03,142 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-39) [19610c8] START, SetVdsStatusVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 464768f7
2015-08-19 03:23:03,382 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-39) [19610c8] FINISH, SetVdsStatusVDSCommand, log id: 464768f7
2015-08-19 03:23:03,840 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c : hosted_engine_3
2015-08-19 03:23:04,249 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-7-thread-39) [19610c8] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c Type: VDS
2015-08-19 03:23:04,253 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetHaMaintenanceModeVDSCommand] (org.ovirt.thread.pool-7-thread-39) [19610c8] START, SetHaMaintenanceModeVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c), log id: 176336d7
2015-08-19 03:23:04,256 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetHaMaintenanceModeVDSCommand] (org.ovirt.thread.pool-7-thread-39) [19610c8] FINISH, SetHaMaintenanceModeVDSCommand, log id: 176336d7
2015-08-19 03:23:04,449 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-39) [19610c8] Correlation ID: 19610c8, Job ID: a2471fb2-b83d-4374-a87d-20a6a2658025, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_3 was switched to Maintenance mode by admin@internal.
2015-08-19 03:23:04,734 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-42) Clearing cache of pool: 00000002-0002-0002-0002-00000000010d for problematic entities of VDS: hosted_engine_3.
2015-08-19 03:23:04,734 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-42) Removing vds [6ebcfbc5-eb17-43e4-bd87-c08354402d1c] from the domain in maintenance cache
2015-08-19 03:23:04,734 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-42) Removing host(s) [6ebcfbc5-eb17-43e4-bd87-c08354402d1c] from hosts unseen domain report cache
2015-08-19 03:23:30,903 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-27) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic
2015-08-19 03:23:30,903 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-27) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic
2015-08-19 03:23:30,903 INFO [org.ovirt.engine.core.bll.UpgradeOvirtNodeCommand] (ajp-/127.0.0.1:8702-27) [594d6e96] Running command: UpgradeOvirtNodeCommand internal: false. Entities affected : ID: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c Type: VDSAction group EDIT_HOST_CONFIGURATION with role type ADMIN
2015-08-19 03:23:31,071 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-27) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic
2015-08-19 03:23:31,071 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-27) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic
2015-08-19 03:23:31,072 INFO [org.ovirt.engine.core.bll.UpgradeOvirtNodeInternalCommand] (ajp-/127.0.0.1:8702-27) [594d6e96] Lock Acquired to object EngineLock [exclusiveLocks= key: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c value: VDS
, sharedLocks= ]
2015-08-19 03:23:31,073 INFO [org.ovirt.engine.core.bll.UpgradeOvirtNodeInternalCommand] (org.ovirt.thread.pool-7-thread-17) [594d6e96] Running command: UpgradeOvirtNodeInternalCommand internal: true. Entities affected : ID: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c Type: VDS
2015-08-19 03:23:31,074 INFO [org.ovirt.engine.core.bll.UpgradeOvirtNodeInternalCommand] (org.ovirt.thread.pool-7-thread-17) [594d6e96] Execute upgrade host 6ebcfbc5-eb17-43e4-bd87-c08354402d1c, hosted_engine_3
2015-08-19 03:23:31,074 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-17) [594d6e96] START, SetVdsStatusVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 2aca3996
2015-08-19 03:23:31,077 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-27) [594d6e96] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host hosted_engine_3.
2015-08-19 03:23:31,170 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-17) [594d6e96] FINISH, SetVdsStatusVDSCommand, log id: 2aca3996
2015-08-19 03:23:31,289 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-7-thread-17) [594d6e96] Installation rhevhtest-3.redhat.com: Connected to host rhevhtest-3.redhat.com with SSH key fingerprint: 04:a9:c9:29:dd:ca:c8:c0:e8:22:b9:0c:2e:20:a8:c4
2015-08-19 03:23:31,370 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-27) [594d6e96] Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_3 configuration was updated by admin@internal.
2015-08-19 03:23:31,482 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-17) [594d6e96] Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Connected to host rhevhtest-3.redhat.com with SSH key fingerprint: 04:a9:c9:29:dd:ca:c8:c0:e8:22:b9:0c:2e:20:a8:c4.
2015-08-19 03:23:31,495 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-7-thread-17) [594d6e96] Installation rhevhtest-3.redhat.com: Sending file /usr/share/rhev-hypervisor/rhevh-7.1-20150813.0.el7ev.iso to /data/updates/ovirt-node-image.iso
2015-08-19 03:23:31,601 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-17) [594d6e96] Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Sending file /usr/share/rhev-hypervisor/rhevh-7.1-20150813.0.el7ev.iso to /data/updates/ovirt-node-image.iso.
2015-08-19 03:23:31,601 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-7-thread-17) SSH execute root@rhevhtest-3.redhat.com 'mkdir -p '/data/updates''
2015-08-19 03:23:53,836 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-7-thread-17) [594d6e96] Installation rhevhtest-3.redhat.com: Executing /usr/share/vdsm-reg/vdsm-upgrade
2015-08-19 03:23:54,144 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-17) [594d6e96] Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Executing /usr/share/vdsm-reg/vdsm-upgrade.
2015-08-19 03:23:54,144 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-7-thread-17) SSH execute root@rhevhtest-3.redhat.com '/usr/share/vdsm-reg/vdsm-upgrade'
2015-08-19 03:23:54,282 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:23:54,284 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Temporary Directory is: /data/tmpEvgoQw
2015-08-19 03:23:54,377 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Temporary Directory is: /data/tmpEvgoQw
.
2015-08-19 03:23:54,377 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:23:54,378 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.LockFile: INFO Acquiring Lock
2015-08-19 03:23:54,477 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.LockFile: INFO Acquiring Lock
.
2015-08-19 03:23:54,477 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:23:54,478 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running pre-upgrade hooks
2015-08-19 03:23:54,585 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running pre-upgrade hooks
.
2015-08-19 03:23:54,585 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:23:54,586 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-vdsm
2015-08-19 03:23:54,685 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-vdsm
.
2015-08-19 03:23:57,549 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages
2015-08-19 03:23:57,550 WARN [org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker] (ResponseWorker) Exception thrown during message processing
2015-08-19 03:23:57,686 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:23:57,687 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO pre-upgrade hooks completed
2015-08-19 03:23:57,784 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO pre-upgrade hooks completed
.
2015-08-19 03:23:57,784 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:23:57,785 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running pre-upgrade hooks
2015-08-19 03:23:57,884 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running pre-upgrade hooks
.
2015-08-19 03:23:57,884 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:23:57,885 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-vdsm
2015-08-19 03:23:57,976 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-vdsm
.
2015-08-19 03:23:58,875 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:23:58,876 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO pre-upgrade hooks completed
2015-08-19 03:23:59,175 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO pre-upgrade hooks completed
.
2015-08-19 03:23:59,175 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:23:59,176 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO hooks: False
2015-08-19 03:23:59,291 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO hooks: False
.
2015-08-19 03:24:00,253 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:24:00,254 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Installing Bootloader
2015-08-19 03:24:00,365 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Installing Bootloader
.
2015-08-19 03:24:14,855 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:24:14,856 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running post-upgrade hooks
2015-08-19 03:24:14,945 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running post-upgrade hooks
.
2015-08-19 03:24:14,946 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:24:14,946 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-sanlock-check
2015-08-19 03:24:15,045 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-sanlock-check
.
2015-08-19 03:24:18,819 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-7-thread-16) [41cde333] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 5be4d9f4-cb4e-4148-aec1-f697527515ef Type: VDSAction group MANIPULATE_HOST with role type ADMIN
2015-08-19 03:24:18,820 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-16) [41cde333] START, SetVdsStatusVDSCommand(HostName = test.redhat.com, HostId = 5be4d9f4-cb4e-4148-aec1-f697527515ef, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 1960ee83
2015-08-19 03:24:18,926 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-16) [41cde333] FINISH, SetVdsStatusVDSCommand, log id: 1960ee83
2015-08-19 03:24:19,426 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-7-thread-16) [41cde333] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 5be4d9f4-cb4e-4148-aec1-f697527515ef Type: VDS
2015-08-19 03:24:19,608 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 5be4d9f4-cb4e-4148-aec1-f697527515ef : test.redhat.com
2015-08-19 03:24:19,617 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-16) [41cde333] Correlation ID: 41cde333, Job ID: 372357c2-3850-4be1-a95a-cf2b914d525c, Call Stack: null, Custom Event ID: -1, Message: Host test.redhat.com was switched to Maintenance mode by admin@internal.
2015-08-19 03:24:26,647 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-24) [6ebf718e] Lock Acquired to object EngineLock [exclusiveLocks= key: 5be4d9f4-cb4e-4148-aec1-f697527515ef value: VDS
, sharedLocks= ]
2015-08-19 03:24:26,734 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-7-thread-29) [6ebf718e] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 5be4d9f4-cb4e-4148-aec1-f697527515ef Type: VDSAction group DELETE_HOST with role type ADMIN
2015-08-19 03:24:26,889 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-7-thread-29) [6ebf718e] START, RemoveVdsVDSCommand( HostId = 5be4d9f4-cb4e-4148-aec1-f697527515ef), log id: 249d2259
2015-08-19 03:24:26,889 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-7-thread-29) [6ebf718e] vdsManager::disposing
2015-08-19 03:24:26,889 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-7-thread-29) [6ebf718e] FINISH, RemoveVdsVDSCommand, log id: 249d2259
2015-08-19 03:24:27,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-29) [6ebf718e] Correlation ID: 6ebf718e, Call Stack: null, Custom Event ID: -1, Message: Host test.redhat.com was removed by admin@internal.
2015-08-19 03:24:27,009 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-7-thread-29) [6ebf718e] Lock freed to object EngineLock [exclusiveLocks= key: 5be4d9f4-cb4e-4148-aec1-f697527515ef value: VDS
, sharedLocks= ]
2015-08-19 03:25:55,049 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:25:55,050 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO post-upgrade hooks completed
2015-08-19 03:25:55,153 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO post-upgrade hooks completed
.
2015-08-19 03:25:55,153 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:25:55,154 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running post-upgrade hooks
2015-08-19 03:25:55,253 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running post-upgrade hooks
.
2015-08-19 03:25:55,253 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:25:55,254 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-sanlock-check
2015-08-19 03:25:55,355 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Running: 01-sanlock-check
.
2015-08-19 03:25:57,174 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:25:57,175 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO post-upgrade hooks completed
2015-08-19 03:25:57,286 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO post-upgrade hooks completed
.
2015-08-19 03:25:57,286 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:25:57,287 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Scheduling Reboot
2015-08-19 03:25:57,386 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Scheduling Reboot
.
2015-08-19 03:25:57,386 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:25:57,387 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Reboot Scheduled
2015-08-19 03:25:57,486 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Reboot Scheduled
.
2015-08-19 03:25:57,486 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:25:57,487 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Upgrade Completed
2015-08-19 03:25:57,594 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Upgrade Completed
.
2015-08-19 03:25:57,594 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:25:57,595 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Cleaning up temporary directory
2015-08-19 03:25:57,686 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: ovirt-node-upgrade.UpgradeTool: INFO Cleaning up temporary directory
.
2015-08-19 03:25:57,686 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:25:57,687 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: ovirt-node-upgrade; Details: Upgraded Succeeded. Rebooting
2015-08-19 03:25:57,803 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: ovirt-node-upgrade; Details: Upgraded Succeeded. Rebooting .
2015-08-19 03:25:57,803 INFO [org.ovirt.engine.core.bll.OVirtNodeUpgrade] (OVirtNodeUpgrade) update from host rhevhtest-3.redhat.com:
2015-08-19 03:25:57,803 INFO [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation rhevhtest-3.redhat.com: Step: RHEV_INSTALL
2015-08-19 03:25:58,135 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (OVirtNodeUpgrade) Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_3. Step: RHEV_INSTALL.
2015-08-19 03:25:58,136 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-17) [594d6e96] START, SetVdsStatusVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c, status=Reboot, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 47cfb256
2015-08-19 03:25:58,227 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-17) [594d6e96] FINISH, SetVdsStatusVDSCommand, log id: 47cfb256
2015-08-19 03:25:58,227 INFO [org.ovirt.engine.core.bll.UpgradeOvirtNodeInternalCommand] (org.ovirt.thread.pool-7-thread-17) [594d6e96] After upgrade host 6ebcfbc5-eb17-43e4-bd87-c08354402d1c, hosted_engine_3: success
2015-08-19 03:25:58,228 INFO [org.ovirt.engine.core.bll.UpgradeOvirtNodeInternalCommand] (org.ovirt.thread.pool-7-thread-37) [594d6e96] Waiting 300 seconds, for server to finish reboot process.
2015-08-19 03:25:58,319 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-17) [594d6e96] Correlation ID: 594d6e96, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_3 installed
2015-08-19 03:25:58,319 INFO [org.ovirt.engine.core.bll.UpgradeOvirtNodeInternalCommand] (org.ovirt.thread.pool-7-thread-17) [594d6e96] Lock freed to object EngineLock [exclusiveLocks= key: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c value: VDS
, sharedLocks= ]
2015-08-19 03:30:20,475 INFO [org.ovirt.engine.docs.utils.servlet.ContextSensitiveHelpMappingServlet] (ajp-/127.0.0.1:8702-23) Successfully read CSH mapping file '/usr/share/doc/rhevm-doc/manual/en-US/csh.conf.d/webadmin/10-webadmin-en-US.json'
2015-08-19 03:30:58,229 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-37) [594d6e96] START, SetVdsStatusVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c, status=NonResponsive, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 51bac981
2015-08-19 03:30:58,381 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-37) [594d6e96] FINISH, SetVdsStatusVDSCommand, log id: 51bac981
2015-08-19 03:30:59,429 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-3.redhat.com/10.66.65.29
2015-08-19 03:30:59,572 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-79) START, GetHardwareInfoVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c, vds=Host[hosted_engine_3,6ebcfbc5-eb17-43e4-bd87-c08354402d1c]), log id: 606c0cb7
2015-08-19 03:30:59,636 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-79) FINISH, GetHardwareInfoVDSCommand, log id: 606c0cb7
2015-08-19 03:30:59,846 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-79) [3a5686b5] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c Type: VDS
2015-08-19 03:30:59,853 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-79) [351a1238] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000010d Type: StoragePool
2015-08-19 03:30:59,963 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-79) [351a1238] START, SetMOMPolicyParametersVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c), log id: 9dfa73d
2015-08-19 03:31:00,022 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-79) [351a1238] FINISH, SetMOMPolicyParametersVDSCommand, log id: 9dfa73d
2015-08-19 03:31:00,213 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-79) [351a1238] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host hosted_engine_3 was set to Up.
2015-08-19 03:31:00,218 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-79) [351a1238] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host hosted_engine_3.
2015-08-19 03:31:00,919 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-79) [52b9971b] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c Type: VDS
2015-08-19 03:31:43,275 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp-/127.0.0.1:8702-16) [5231392f] Lock Acquired to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 03:31:43,294 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-16) [5231392f] Candidate host hosted_engine_1 (80ce2151-3dbc-4945-834b-56315ca9f823) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory
2015-08-19 03:31:43,295 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-16) [5231392f] Candidate host hosted_engine_3 (6ebcfbc5-eb17-43e4-bd87-c08354402d1c) was filtered out by VAR__FILTERTYPE__INTERNAL filter HA
2015-08-19 03:31:43,720 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-25) [5231392f] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group MIGRATE_VM with role type USER, ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_VM with role type USER
2015-08-19 03:31:43,834 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-25) [5231392f] START, MigrateVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=bed31fc7-9abc-426f-859c-3a2df7d40ee1, dstHost=rhevhtest-2.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 213b3abd
2015-08-19 03:31:43,835 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-25) [5231392f] START, MigrateBrokerVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=bed31fc7-9abc-426f-859c-3a2df7d40ee1, dstHost=rhevhtest-2.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 23575689
2015-08-19 03:31:43,840 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-25) [5231392f] FINISH, MigrateBrokerVDSCommand, log id: 23575689
2015-08-19 03:31:43,936 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-25) [5231392f] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 213b3abd
2015-08-19 03:31:44,028 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-25) [5231392f] Correlation ID: 5231392f, Job ID: 978c8543-b837-49cc-98b8-e79463237ed9, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_2, User: admin@internal).
2015-08-19 03:31:45,631 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) VM HostedEngine c5312b39-316c-4906-bc20-df2a472bda1f moved from MigratingFrom --> Up
2015-08-19 03:31:45,631 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) Adding VM c5312b39-316c-4906-bc20-df2a472bda1f to re-run list
2015-08-19 03:31:46,043 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) Rerun vm c5312b39-316c-4906-bc20-df2a472bda1f. Called from vds hosted_engine_1
2015-08-19 03:31:46,047 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-28) START, MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f), log id: 69051d40
2015-08-19 03:31:46,050 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-28) Failed in MigrateStatusVDS method
2015-08-19 03:31:46,050 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-28) Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value
StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=10, mMessage=Could not connect to peer VDS]]
2015-08-19 03:31:46,050 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-28) HostName = hosted_engine_1
2015-08-19 03:31:46,050 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-28) Command MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Could not connect to peer VDS, code = 10
2015-08-19 03:31:46,050 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-28) FINISH, MigrateStatusVDSCommand, log id: 69051d40
2015-08-19 03:31:46,351 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-28) Correlation ID: 5231392f, Job ID: 978c8543-b837-49cc-98b8-e79463237ed9, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Could not connect to peer host (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_2).
2015-08-19 03:31:46,451 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-28) Lock freed to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 03:34:39,973 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf".
2015-08-19 03:34:40,077 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties.
2015-08-19 03:34:40,077 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf".
2015-08-19 03:34:40,078 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf".
2015-08-19 03:34:40,078 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-java.conf".
2015-08-19 03:34:40,078 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf".
2015-08-19 03:34:40,078 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf".
2015-08-19 03:34:40,078 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf".
2015-08-19 03:34:40,078 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf".
2015-08-19 03:34:40,078 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_AJP_ENABLED" is "true".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_AJP_PORT" is "8702".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_DB_DATABASE" is "engine".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_DB_HOST" is "localhost".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_DB_PASSWORD" is "***".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_DB_PORT" is "5432".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_DB_SECURED" is "False".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_DB_USER" is "engine".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_DEBUG_ADDRESS" is "".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_ETC" is "/etc/ovirt-engine".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE" is "/var/lib/ovirt-engine/external_truststore".
2015-08-19 03:34:40,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_PASSWORD" is "***".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_TYPE" is "JKS".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_FQDN" is "ycuihevm.redhat.com".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_GROUP" is "ovirt".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_HEAP_MAX" is "1024M".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_HEAP_MIN" is "1024M".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_HTTPS_ENABLED" is "false".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_HTTPS_PORT" is "None".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_HTTP_ENABLED" is "false".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_HTTP_PORT" is "None".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_LOG_TO_CONSOLE" is "false".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_PERM_MAX" is "256m".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_PERM_MIN" is "256m".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore".
2015-08-19 03:34:40,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_PROPERTIES" is " java.awt.headless=true sun.rmi.dgc.client.gcInterval=3600000 sun.rmi.dgc.server.gcInterval=3600000 jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_PROXY_ENABLED" is "true".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_PROXY_HTTP_PORT" is "80".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_REPORTS_BASE_URL" is "".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_REPORTS_DASHBOARD_URL" is "/flow.html?viewAsDashboardFrame=true".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_REPORTS_NOT_INSTALLED_URL" is "/ovirt-engine/ReportsNotInstalled.html".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_REPORTS_PROXY_URL" is "/ovirt/reports-interface".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_REPORTS_READ_TIMEOUT" is "".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_REPORTS_RIGHTCLICK_URL" is "".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_REPORTS_VERIFY_CHAIN" is "true".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_REPORTS_VERIFY_HOST" is "true".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_STOP_INTERVAL" is "1".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_STOP_TIME" is "10".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_URI" is "/ovirt-engine".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_USER" is "ovirt".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "ENGINE_VERBOSE_GC" is "false".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "JBOSS_HOME" is "/usr/share/jbossas".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime".
2015-08-19 03:34:40,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 42) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD,ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_PASSWORD".
2015-08-19 03:34:48,191 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 52) Start initializing Backend
2015-08-19 03:34:48,912 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 52) Loading file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties
2015-08-19 03:34:49,793 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 52) Running ovirt-engine 3.5.4-1.2.el6ev
2015-08-19 03:34:49,793 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 52) Start initializing dictionaries
2015-08-19 03:34:49,794 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 52) Finished initializing dictionaries
2015-08-19 03:34:49,795 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Start initializing AuditLogCleanupManager
2015-08-19 03:34:49,795 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Setting audit cleanup manager to run at: 35 35 3 * * ?
2015-08-19 03:34:49,841 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Finished initializing AuditLogCleanupManager
2015-08-19 03:34:49,841 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Start initializing CommandEntityCleanupManager
2015-08-19 03:34:49,841 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Setting command entity cleanup manager to run at: 35 35 3 * * ?
2015-08-19 03:34:49,842 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Finished initializing CommandEntityCleanupManager
2015-08-19 03:34:49,843 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 52) Start initializing TagsDirector
2015-08-19 03:34:49,848 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 52) Tag root added to tree
2015-08-19 03:34:49,894 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 52) Finished initializing TagsDirector
2015-08-19 03:34:49,895 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 52) Start initializing IsoDomainListSyncronizer
2015-08-19 03:34:49,898 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 52) Finished initializing IsoDomainListSyncronizer
2015-08-19 03:34:49,928 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 52) Completed initializing handlers
2015-08-19 03:34:49,967 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 52) Start initializing ErrorTranslatorImpl
2015-08-19 03:34:50,010 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 52) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table.
2015-08-19 03:34:50,010 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 52) Finished initializing ErrorTranslatorImpl
2015-08-19 03:34:50,010 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 52) Start initializing ErrorTranslatorImpl
2015-08-19 03:34:50,011 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 52) Finished initializing ErrorTranslatorImpl
2015-08-19 03:34:50,011 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 52) Mark incomplete jobs as UNKNOWN
2015-08-19 03:34:50,086 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 52) Start initializing JobRepositoryCleanupManager
2015-08-19 03:34:50,086 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 52) Finished initializing JobRepositoryCleanupManager
2015-08-19 03:34:50,086 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 52) Start initializing AutoRecoveryManager
2015-08-19 03:34:50,087 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 52) Finished initializing AutoRecoveryManager
2015-08-19 03:34:50,088 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 52) Start initializing ExecutionMessageDirector
2015-08-19 03:34:50,089 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 52) Finished initializing ExecutionMessageDirector
2015-08-19 03:34:50,227 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Loading extension 'builtin-authn-internal'
2015-08-19 03:34:50,231 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'builtin-authn-internal' loaded
2015-08-19 03:34:50,232 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Loading extension 'internal'
2015-08-19 03:34:50,232 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'internal' loaded
2015-08-19 03:34:50,236 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Initializing extension 'builtin-authn-internal'
2015-08-19 03:34:50,236 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'builtin-authn-internal' initialized
2015-08-19 03:34:50,237 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Initializing extension 'internal'
2015-08-19 03:34:50,237 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'internal' initialized
2015-08-19 03:34:50,237 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Start of enabled extensions list
2015-08-19 03:34:50,237 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true'
2015-08-19 03:34:50,237 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true'
2015-08-19 03:34:50,237 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) End of enabled extensions list
2015-08-19 03:34:50,356 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 52) Initialization of AsyncTaskManager completed successfully.
2015-08-19 03:34:50,357 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 52) Start initializing ResourceManager
2015-08-19 03:34:50,435 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Entered VdsManager constructor
2015-08-19 03:34:50,475 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Initialize vdsBroker (rhevhtest-3.redhat.com,54,321)
2015-08-19 03:34:50,797 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 52) VDS 6ebcfbc5-eb17-43e4-bd87-c08354402d1c was added to the Resource Manager
2015-08-19 03:34:50,797 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Entered VdsManager constructor
2015-08-19 03:34:50,804 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Initialize vdsBroker (rhevhtest-1.redhat.com,54,321)
2015-08-19 03:34:50,822 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 52) VDS 80ce2151-3dbc-4945-834b-56315ca9f823 was added to the Resource Manager
2015-08-19 03:34:50,822 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Entered VdsManager constructor
2015-08-19 03:34:50,829 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Initialize vdsBroker (rhevhtest-2.redhat.com,54,321)
2015-08-19 03:34:50,847 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 52) VDS bed31fc7-9abc-426f-859c-3a2df7d40ee1 was added to the Resource Manager
2015-08-19 03:34:50,864 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 52) Finished initializing ResourceManager
2015-08-19 03:34:50,864 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 52) Initialization of OvfDataUpdater completed successfully.
2015-08-19 03:34:50,864 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 52) Start scheduling to enable vds load balancer
2015-08-19 03:34:50,865 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 52) Finished scheduling to enable vds load balancer
2015-08-19 03:34:50,865 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 52) Start HA Reservation check
2015-08-19 03:34:50,865 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 52) Finished HA Reservation check
2015-08-19 03:34:50,871 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 52) Init VM custom properties utilities
2015-08-19 03:34:50,871 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 52) Init device custom properties utilities
2015-08-19 03:34:50,873 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 52) Initializing Scheduling manager
2015-08-19 03:34:50,880 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Start initializing MacPoolManagerRanges
2015-08-19 03:34:50,905 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Finished initializing. Available MACs in pool: 256
2015-08-19 03:34:50,925 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 52) External scheduler disabled, discovery skipped
2015-08-19 03:34:50,925 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 52) Initialized Scheduling manager
2015-08-19 03:34:50,926 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 52) Initializing DWH Heart Beat
2015-08-19 03:34:50,926 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 52) DWH Heart Beat initialized
2015-08-19 03:34:53,898 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-1.redhat.com/10.66.11.167
2015-08-19 03:34:55,900 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed
2015-08-19 03:34:55,902 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-3.redhat.com/10.66.65.29
2015-08-19 03:34:55,904 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-2.redhat.com/10.66.11.107
2015-08-19 03:34:55,904 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-6) Exception during connection
2015-08-19 03:34:55,941 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetStatsVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vds=Host[hosted_engine_1,80ce2151-3dbc-4945-834b-56315ca9f823]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout
2015-08-19 03:34:55,976 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host hosted_engine_1 is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued.
2015-08-19 03:34:59,477 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: UNASSIGNED not exist in string table
2015-08-19 03:34:59,672 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table
2015-08-19 03:34:59,672 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table
2015-08-19 03:34:59,672 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table
2015-08-19 03:34:59,672 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table
2015-08-19 03:34:59,672 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table
2015-08-19 03:34:59,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table
2015-08-19 03:34:59,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table
2015-08-19 03:34:59,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table
2015-08-19 03:34:59,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table
2015-08-19 03:34:59,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table
2015-08-19 03:34:59,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table
2015-08-19 03:34:59,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table
2015-08-19 03:34:59,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table
2015-08-19 03:34:59,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table
2015-08-19 03:34:59,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table
2015-08-19 03:34:59,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table
2015-08-19 03:34:59,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table
2015-08-19 03:35:00,174 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GetHardwareInfoVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, vds=Host[hosted_engine_2,bed31fc7-9abc-426f-859c-3a2df7d40ee1]), log id: 45ed1eda
2015-08-19 03:35:00,208 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GetHardwareInfoVDSCommand, log id: 45ed1eda
2015-08-19 03:35:00,227 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-5) START, GetHardwareInfoVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c, vds=Host[hosted_engine_3,6ebcfbc5-eb17-43e4-bd87-c08354402d1c]), log id: 360b058d
2015-08-19 03:35:00,250 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-5) FINISH, GetHardwareInfoVDSCommand, log id: 360b058d
2015-08-19 03:35:00,441 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host hosted_engine_1 is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued.
2015-08-19 03:35:00,442 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) vds::refreshVdsStats Failed getVdsStats, vds = 80ce2151-3dbc-4945-834b-56315ca9f823 : hosted_engine_1, error = VDSNetworkException: java.rmi.ConnectException: Connection timeout
2015-08-19 03:35:00,442 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout
at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:]
at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:]
at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:]
at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:]
at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsStats(VdsUpdateRunTimeInfo.java:690) [vdsbroker.jar:]
at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:644) [vdsbroker.jar:]
at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:495) [vdsbroker.jar:]
at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_85]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_85]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_85]
at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_85]
at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:]
at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:]
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:]
Caused by: java.rmi.ConnectException: Connection timeout
at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:]
at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:]
at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_85]
at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:]
at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:60) [vdsm-jsonrpc-java-client.jar:]
2015-08-19 03:35:00,577 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Failed to refresh VDS , vds = 80ce2151-3dbc-4945-834b-56315ca9f823 : hosted_engine_1, VDS Network Error, continuing.
java.rmi.ConnectException: Connection timeout
2015-08-19 03:35:03,298 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Initializing Host: hosted_engine_3
2015-08-19 03:35:03,336 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-5) [183b936d] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 6ebcfbc5-eb17-43e4-bd87-c08354402d1c Type: VDS
2015-08-19 03:35:03,598 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-1.redhat.com/10.66.11.167
2015-08-19 03:35:03,599 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed
2015-08-19 03:35:03,599 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-23) Exception during connection
2015-08-19 03:35:03,606 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vds=Host[hosted_engine_1,80ce2151-3dbc-4945-834b-56315ca9f823]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused
2015-08-19 03:35:03,606 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host hosted_engine_1 is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued.
2015-08-19 03:35:03,622 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused
at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:]
at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:]
at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:]
at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:]
at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:]
at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:649) [vdsbroker.jar:]
at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:495) [vdsbroker.jar:]
at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_85]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_85]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_85]
at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_85]
at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:]
at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:]
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:]
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_85]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744) [rt.jar:1.7.0_85]
at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:]
at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:]
at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_85]
at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:]
at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:60) [vdsm-jsonrpc-java-client.jar:]
2015-08-19 03:35:03,635 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Failed to refresh VDS , vds = 80ce2151-3dbc-4945-834b-56315ca9f823 : hosted_engine_1, VDS Network Error, continuing.
java.net.ConnectException: Connection refused
2015-08-19 03:35:03,792 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: hosted_engine_2
2015-08-19 03:35:03,795 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-7) [2abe7dbb] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: bed31fc7-9abc-426f-859c-3a2df7d40ee1 Type: VDS
2015-08-19 03:35:06,649 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to rhevhtest-1.redhat.com/10.66.11.167
2015-08-19 03:35:06,956 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-28) START, GetHardwareInfoVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vds=Host[hosted_engine_1,80ce2151-3dbc-4945-834b-56315ca9f823]), log id: 1ef9a45d
2015-08-19 03:35:07,007 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-28) FINISH, GetHardwareInfoVDSCommand, log id: 1ef9a45d
2015-08-19 03:35:07,209 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-28) [2285fa21] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 80ce2151-3dbc-4945-834b-56315ca9f823 Type: VDS
2015-08-19 03:35:07,248 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-28) [2285fa21] START, GetHardwareInfoVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vds=Host[hosted_engine_1,80ce2151-3dbc-4945-834b-56315ca9f823]), log id: 531e518d
2015-08-19 03:35:07,253 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-28) [2285fa21] FINISH, GetHardwareInfoVDSCommand, log id: 531e518d
2015-08-19 03:35:07,369 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-28) [640d324] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000010d Type: StoragePool
2015-08-19 03:35:07,618 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-28) [640d324] START, SetMOMPolicyParametersVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823), log id: 606fa1
2015-08-19 03:35:07,682 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-28) [640d324] FINISH, SetMOMPolicyParametersVDSCommand, log id: 606fa1
2015-08-19 03:35:07,771 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-28) [640d324] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host hosted_engine_1 was set to Up.
2015-08-19 03:35:07,785 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-28) [640d324] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host hosted_engine_1.
2015-08-19 03:35:07,989 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) [640d324] Initializing Host: hosted_engine_1
2015-08-19 03:35:07,993 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-28) [2e63810f] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 80ce2151-3dbc-4945-834b-56315ca9f823 Type: VDS
2015-08-19 03:35:08,016 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-7-thread-10) [32a5b2c9] Running command: ProcessDownVmCommand internal: true.
2015-08-19 03:35:35,002 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-2) Start deleteAgedOutCommandEntities
2015-08-19 03:35:35,002 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-1) Start deleteAgedOutAuditLogs
2015-08-19 03:35:35,012 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-2) Finished deleteAgedOutCommandEntities
2015-08-19 03:35:35,033 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-1) Finished deleteAgedOutAuditLogs
2015-08-19 03:36:00,576 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-5) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json]
2015-08-19 03:36:00,580 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-5) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json]
2015-08-19 03:36:02,989 INFO [org.ovirt.engine.docs.utils.servlet.ContextSensitiveHelpMappingServlet] (ajp-/127.0.0.1:8702-4) Successfully read CSH mapping file '/usr/share/doc/rhevm-doc/manual/en-US/csh.conf.d/webadmin/10-webadmin-en-US.json'
2015-08-19 03:36:08,162 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginAdminUserCommand internal: false.
2015-08-19 03:36:08,523 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in.
2015-08-19 03:37:29,907 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp-/127.0.0.1:8702-2) [7a7431c5] Lock Acquired to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 03:37:30,058 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-2) [7a7431c5] Candidate host hosted_engine_2 (bed31fc7-9abc-426f-859c-3a2df7d40ee1) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory
2015-08-19 03:37:30,669 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-47) [7a7431c5] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group MIGRATE_VM with role type USER, ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_VM with role type USER
2015-08-19 03:37:30,879 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-47) [7a7431c5] START, MigrateVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-2.redhat.com, dstVdsId=80ce2151-3dbc-4945-834b-56315ca9f823, dstHost=rhevhtest-1.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 1e0d03d5
2015-08-19 03:37:30,883 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-47) [7a7431c5] START, MigrateBrokerVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-2.redhat.com, dstVdsId=80ce2151-3dbc-4945-834b-56315ca9f823, dstHost=rhevhtest-1.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 12bd8f0b
2015-08-19 03:37:30,896 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-47) [7a7431c5] FINISH, MigrateBrokerVDSCommand, log id: 12bd8f0b
2015-08-19 03:37:31,176 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-47) [7a7431c5] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 1e0d03d5
2015-08-19 03:37:31,319 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-47) [7a7431c5] Correlation ID: 7a7431c5, Job ID: 20a51df4-9152-4945-a399-ca1826c82120, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: HostedEngine, Source: hosted_engine_2, Destination: hosted_engine_1, User: admin@internal).
2015-08-19 03:37:33,109 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:37:36,129 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:37:39,237 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:37:43,029 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:37:45,095 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-47) START, FullListVdsCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, vds=Host[hosted_engine_2,bed31fc7-9abc-426f-859c-3a2df7d40ee1], vmIds=[c5312b39-316c-4906-bc20-df2a472bda1f]), log id: 53c7649d
2015-08-19 03:37:45,101 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-47) FINISH, FullListVdsCommand, return: [{status=Migration Source, nicModel=rtl8139,pv, emulatedMachine=rhel6.5.0, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, pid=384, clientIp=, devices=[Ljava.lang.Object;@72688bf, smp=1, vmType=kvm, displayIp=0, displaySecurePort=-1, memSize=4096, displayPort=5900, cpuType=Conroe, spiceSecureChannels=smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir, guestDiskMapping={QEMU_DVD-ROM_={name=/dev/sr0}, 4548c127-e9eb-4be9-b={name=/dev/vda}}, vmName=HostedEngine, display=vnc, pauseCode=NOERR}], log id: 53c7649d
2015-08-19 03:37:45,107 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Received a console Device without an address when processing VM c5312b39-316c-4906-bc20-df2a472bda1f devices, skipping device: {alias=console0, specParams={}, deviceType=console, deviceId=eee60f81-ce5e-4d1b-8fda-93529e74915b, device=console, type=console}
2015-08-19 03:37:45,108 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Received a memballoon Device without an address when processing VM c5312b39-316c-4906-bc20-df2a472bda1f devices, skipping device: {device=memballoon, specParams={model=none}, deviceType=balloon, type=balloon, alias=balloon0}
2015-08-19 03:37:45,108 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Received a vnc Device without an address when processing VM c5312b39-316c-4906-bc20-df2a472bda1f devices, skipping device: {device=vnc, specParams={spiceSecureChannels=smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir, displayIp=0}, deviceType=graphics, type=graphics, port=5900}
2015-08-19 03:37:45,108 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Empty or NULL values were passed for a VM c5312b39-316c-4906-bc20-df2a472bda1f device, Device is skipped
2015-08-19 03:37:46,042 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:37:49,065 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:37:52,080 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:37:55,132 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:37:58,978 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:38:01,991 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:38:05,004 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:38:08,017 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:38:11,085 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:38:15,110 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:38:18,153 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:38:21,169 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-43) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:38:24,201 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:38:27,262 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:38:31,140 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:38:34,172 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:38:37,198 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:38:40,212 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f is migrating to vds hosted_engine_1 ignoring it in the refresh until migration is done
2015-08-19 03:38:43,312 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-99) START, FullListVdsCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vds=Host[hosted_engine_1,80ce2151-3dbc-4945-834b-56315ca9f823], vmIds=[c5312b39-316c-4906-bc20-df2a472bda1f]), log id: 31251e85
2015-08-19 03:38:43,316 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-99) FINISH, FullListVdsCommand, return: [{status=Up, nicModel=rtl8139,pv, kvmEnable=true, emulatedMachine=rhel6.5.0, afterMigrationStatus=, pid=17764, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, devices=[Ljava.lang.Object;@680c1b6, guestDiskMapping={}, spiceSecureChannels=smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir, vmType=kvm, displayIp=0, displaySecurePort=-1, memSize=4096, displayPort=5900, cpuType=Conroe, clientIp=, smp=1, vmName=HostedEngine, display=vnc}], log id: 31251e85
2015-08-19 03:38:43,502 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Received a console Device without an address when processing VM c5312b39-316c-4906-bc20-df2a472bda1f devices, skipping device: {alias=console0, specParams={}, deviceType=console, deviceId=eee60f81-ce5e-4d1b-8fda-93529e74915b, device=console, type=console}
2015-08-19 03:38:43,502 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Received a memballoon Device without an address when processing VM c5312b39-316c-4906-bc20-df2a472bda1f devices, skipping device: {target=4194304, alias=balloon0, specParams={model=none}, deviceType=balloon, device=memballoon, type=balloon}
2015-08-19 03:38:43,502 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Empty or NULL values were passed for a VM c5312b39-316c-4906-bc20-df2a472bda1f device, Device is skipped
2015-08-19 03:38:43,502 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Received a vnc Device without an address when processing VM c5312b39-316c-4906-bc20-df2a472bda1f devices, skipping device: {device=vnc, specParams={spiceSecureChannels=smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir, displayIp=0}, deviceType=graphics, type=graphics, port=5900}
2015-08-19 03:38:44,403 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-3) START, DestroyVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, force=false, secondsToWait=0, gracefully=false, reason=), log id: aa315d5
2015-08-19 03:38:44,571 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-3) FINISH, DestroyVDSCommand, log id: aa315d5
2015-08-19 03:38:44,757 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Handing over VM HostedEngine c5312b39-316c-4906-bc20-df2a472bda1f to Host 80ce2151-3dbc-4945-834b-56315ca9f823. Setting VM to status MigratingTo
2015-08-19 03:38:44,757 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) VM HostedEngine (c5312b39-316c-4906-bc20-df2a472bda1f) is running in db and not running in VDS hosted_engine_2
2015-08-19 03:38:44,757 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-3) START, FullListVdsCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, vds=Host[hosted_engine_2,bed31fc7-9abc-426f-859c-3a2df7d40ee1], vmIds=[c5312b39-316c-4906-bc20-df2a472bda1f]), log id: 630b3a06
2015-08-19 03:38:44,760 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-3) FINISH, FullListVdsCommand, return: [], log id: 630b3a06
2015-08-19 03:38:45,698 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-99) Correlation ID: 7a7431c5, Job ID: 20a51df4-9152-4945-a399-ca1826c82120, Call Stack: null, Custom Event ID: -1, Message: Migration completed (VM: HostedEngine, Source: hosted_engine_2, Destination: hosted_engine_1, Duration: 1 minute 14 seconds).
2015-08-19 03:38:45,789 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (DefaultQuartzScheduler_Worker-99) Lock freed to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 03:38:48,800 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) VM HostedEngine c5312b39-316c-4906-bc20-df2a472bda1f moved from MigratingTo --> Up
2015-08-19 03:38:48,800 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-12) START, FullListVdsCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vds=Host[hosted_engine_1,80ce2151-3dbc-4945-834b-56315ca9f823], vmIds=[c5312b39-316c-4906-bc20-df2a472bda1f]), log id: 4898e73a
2015-08-19 03:38:48,804 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-12) FINISH, FullListVdsCommand, return: [{status=Up, nicModel=rtl8139,pv, kvmEnable=true, emulatedMachine=rhel6.5.0, afterMigrationStatus=, pid=17764, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, devices=[Ljava.lang.Object;@2e74b612, guestDiskMapping={QEMU_DVD-ROM_={name=/dev/sr0}, 4548c127-e9eb-4be9-b={name=/dev/vda}}, spiceSecureChannels=smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir, vmType=kvm, displayIp=0, displaySecurePort=-1, memSize=4096, displayPort=5900, cpuType=Conroe, clientIp=, smp=1, vmName=HostedEngine, display=vnc}], log id: 4898e73a
2015-08-19 03:38:49,029 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) Received a console Device without an address when processing VM c5312b39-316c-4906-bc20-df2a472bda1f devices, skipping device: {alias=console0, specParams={}, deviceType=console, deviceId=eee60f81-ce5e-4d1b-8fda-93529e74915b, device=console, type=console}
2015-08-19 03:38:49,030 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) Received a memballoon Device without an address when processing VM c5312b39-316c-4906-bc20-df2a472bda1f devices, skipping device: {target=4194304, alias=balloon0, specParams={model=none}, deviceType=balloon, device=memballoon, type=balloon}
2015-08-19 03:38:49,030 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) Empty or NULL values were passed for a VM c5312b39-316c-4906-bc20-df2a472bda1f device, Device is skipped
2015-08-19 03:38:49,030 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) Received a vnc Device without an address when processing VM c5312b39-316c-4906-bc20-df2a472bda1f devices, skipping device: {device=vnc, specParams={spiceSecureChannels=smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir, displayIp=0}, deviceType=graphics, type=graphics, port=5900}
2015-08-19 03:39:06,076 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-58) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f status = WaitForLaunch on vds hosted_engine_3 ignoring it in the refresh until migration is done
2015-08-19 03:39:09,115 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-67) START, DestroyVDSCommand(HostName = hosted_engine_3, HostId = 6ebcfbc5-eb17-43e4-bd87-c08354402d1c, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, force=false, secondsToWait=0, gracefully=false, reason=), log id: 48f6d20c
2015-08-19 03:39:09,281 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-67) FINISH, DestroyVDSCommand, log id: 48f6d20c
2015-08-19 03:39:09,283 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f status = Down on vds hosted_engine_3 ignoring it in the refresh until migration is done
2015-08-19 03:39:10,946 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f status = WaitForLaunch on vds hosted_engine_2 ignoring it in the refresh until migration is done
2015-08-19 03:39:13,956 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f status = WaitForLaunch on vds hosted_engine_2 ignoring it in the refresh until migration is done
2015-08-19 03:39:16,526 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp-/127.0.0.1:8702-7) [43beb264] Lock Acquired to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 03:39:16,580 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-7) [43beb264] Candidate host hosted_engine_1 (80ce2151-3dbc-4945-834b-56315ca9f823) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory
2015-08-19 03:39:16,973 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f status = WaitForLaunch on vds hosted_engine_2 ignoring it in the refresh until migration is done
2015-08-19 03:39:17,324 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-49) [43beb264] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group MIGRATE_VM with role type USER, ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_VM with role type USER
2015-08-19 03:39:17,462 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-49) [43beb264] START, MigrateVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=6ebcfbc5-eb17-43e4-bd87-c08354402d1c, dstHost=rhevhtest-3.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 43b244d9
2015-08-19 03:39:17,463 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-49) [43beb264] START, MigrateBrokerVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=6ebcfbc5-eb17-43e4-bd87-c08354402d1c, dstHost=rhevhtest-3.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 1e4cc6c2
2015-08-19 03:39:17,475 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-49) [43beb264] FINISH, MigrateBrokerVDSCommand, log id: 1e4cc6c2
2015-08-19 03:39:17,599 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-49) [43beb264] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 43b244d9
2015-08-19 03:39:17,774 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-49) [43beb264] Correlation ID: 43beb264, Job ID: cab22569-6201-42d1-b8e0-e1e6fc6b34ca, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_3, User: admin@internal).
2015-08-19 03:39:20,041 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f status = WaitForLaunch on vds hosted_engine_2 ignoring it in the refresh until migration is done
2015-08-19 03:39:22,058 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) VM HostedEngine c5312b39-316c-4906-bc20-df2a472bda1f moved from MigratingFrom --> Up
2015-08-19 03:39:22,195 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Adding VM c5312b39-316c-4906-bc20-df2a472bda1f to re-run list
2015-08-19 03:39:22,804 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Rerun vm c5312b39-316c-4906-bc20-df2a472bda1f. Called from vds hosted_engine_1
2015-08-19 03:39:22,808 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) START, MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f), log id: 1b535d39
2015-08-19 03:39:22,812 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) Failed in MigrateStatusVDS method
2015-08-19 03:39:22,812 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value
StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=10, mMessage=Could not connect to peer VDS]]
2015-08-19 03:39:22,812 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) HostName = hosted_engine_1
2015-08-19 03:39:22,812 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) Command MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Could not connect to peer VDS, code = 10
2015-08-19 03:39:22,812 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) FINISH, MigrateStatusVDSCommand, log id: 1b535d39
2015-08-19 03:39:23,088 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f status = WaitForLaunch on vds hosted_engine_2 ignoring it in the refresh until migration is done
2015-08-19 03:39:23,333 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-5) Correlation ID: 43beb264, Job ID: cab22569-6201-42d1-b8e0-e1e6fc6b34ca, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Could not connect to peer host (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_3).
2015-08-19 03:39:23,545 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-5) Lock freed to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 03:39:27,055 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-17) START, DestroyVDSCommand(HostName = hosted_engine_2, HostId = bed31fc7-9abc-426f-859c-3a2df7d40ee1, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, force=false, secondsToWait=0, gracefully=false, reason=), log id: 447cbed9
2015-08-19 03:39:27,231 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, DestroyVDSCommand, log id: 447cbed9
2015-08-19 03:39:27,410 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) RefreshVmList vm id c5312b39-316c-4906-bc20-df2a472bda1f status = Down on vds hosted_engine_2 ignoring it in the refresh until migration is done
2015-08-19 04:03:45,671 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp-/127.0.0.1:8702-7) [1e54d61b] Lock Acquired to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]
2015-08-19 04:03:45,702 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-7) [1e54d61b] Candidate host hosted_engine_1 (80ce2151-3dbc-4945-834b-56315ca9f823) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory
2015-08-19 04:03:46,412 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-36) [1e54d61b] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group MIGRATE_VM with role type USER, ID: c5312b39-316c-4906-bc20-df2a472bda1f Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 00000001-0001-0001-0001-0000000003b1 Type: VdsGroupsAction group CREATE_VM with role type USER
2015-08-19 04:03:46,659 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-36) [1e54d61b] START, MigrateVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=bed31fc7-9abc-426f-859c-3a2df7d40ee1, dstHost=rhevhtest-2.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: a0abea8
2015-08-19 04:03:46,659 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-36) [1e54d61b] START, MigrateBrokerVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f, srcHost=rhevhtest-1.redhat.com, dstVdsId=bed31fc7-9abc-426f-859c-3a2df7d40ee1, dstHost=rhevhtest-2.redhat.com:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 8163a1a
2015-08-19 04:03:46,665 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-7-thread-36) [1e54d61b] FINISH, MigrateBrokerVDSCommand, log id: 8163a1a
2015-08-19 04:03:46,766 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-7-thread-36) [1e54d61b] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: a0abea8
2015-08-19 04:03:46,858 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-36) [1e54d61b] Correlation ID: 1e54d61b, Job ID: 269ddf6e-d469-4e63-bb8e-5644c58fc692, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_2, User: admin@internal).
2015-08-19 04:03:50,058 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) VM HostedEngine c5312b39-316c-4906-bc20-df2a472bda1f moved from MigratingFrom --> Up
2015-08-19 04:03:50,059 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Adding VM c5312b39-316c-4906-bc20-df2a472bda1f to re-run list
2015-08-19 04:03:50,347 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Rerun vm c5312b39-316c-4906-bc20-df2a472bda1f. Called from vds hosted_engine_1
2015-08-19 04:03:50,351 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-41) START, MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f), log id: 771ed43
2015-08-19 04:03:50,355 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-41) Failed in MigrateStatusVDS method
2015-08-19 04:03:50,355 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-41) Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value
StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=10, mMessage=Could not connect to peer VDS]]
2015-08-19 04:03:50,355 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-41) HostName = hosted_engine_1
2015-08-19 04:03:50,356 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-41) Command MigrateStatusVDSCommand(HostName = hosted_engine_1, HostId = 80ce2151-3dbc-4945-834b-56315ca9f823, vmId=c5312b39-316c-4906-bc20-df2a472bda1f) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Could not connect to peer VDS, code = 10
2015-08-19 04:03:50,356 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-7-thread-41) FINISH, MigrateStatusVDSCommand, log id: 771ed43
2015-08-19 04:03:50,871 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-41) Correlation ID: 1e54d61b, Job ID: 269ddf6e-d469-4e63-bb8e-5644c58fc692, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Could not connect to peer host (VM: HostedEngine, Source: hosted_engine_1, Destination: hosted_engine_2).
2015-08-19 04:03:50,971 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (org.ovirt.thread.pool-7-thread-41) Lock freed to object EngineLock [exclusiveLocks= key: c5312b39-316c-4906-bc20-df2a472bda1f value: VM
, sharedLocks= ]