2015-02-22 16:00:02.021 21855 INFO nova.openstack.common.service [-] Starting 8 workers 2015-02-22 16:00:02.022 21855 INFO nova.openstack.common.service [-] Started child 21868 2015-02-22 16:00:02.024 21868 AUDIT nova.service [-] Starting conductor node (version 2014.2.1-12.el7ost) 2015-02-22 16:00:02.024 21855 INFO nova.openstack.common.service [-] Started child 21869 2015-02-22 16:00:02.026 21869 AUDIT nova.service [-] Starting conductor node (version 2014.2.1-12.el7ost) 2015-02-22 16:00:02.026 21855 INFO nova.openstack.common.service [-] Started child 21870 2015-02-22 16:00:02.029 21870 AUDIT nova.service [-] Starting conductor node (version 2014.2.1-12.el7ost) 2015-02-22 16:00:02.028 21855 INFO nova.openstack.common.service [-] Started child 21871 2015-02-22 16:00:02.031 21871 AUDIT nova.service [-] Starting conductor node (version 2014.2.1-12.el7ost) 2015-02-22 16:00:02.031 21855 INFO nova.openstack.common.service [-] Started child 21872 2015-02-22 16:00:02.033 21872 AUDIT nova.service [-] Starting conductor node (version 2014.2.1-12.el7ost) 2015-02-22 16:00:02.034 21855 INFO nova.openstack.common.service [-] Started child 21873 2015-02-22 16:00:02.037 21873 AUDIT nova.service [-] Starting conductor node (version 2014.2.1-12.el7ost) 2015-02-22 16:00:02.037 21855 INFO nova.openstack.common.service [-] Started child 21874 2015-02-22 16:00:02.040 21874 AUDIT nova.service [-] Starting conductor node (version 2014.2.1-12.el7ost) 2015-02-22 16:00:02.040 21855 INFO nova.openstack.common.service [-] Started child 21875 2015-02-22 16:00:02.066 21875 AUDIT nova.service [-] Starting conductor node (version 2014.2.1-12.el7ost) 2015-02-22 16:00:02.705 21868 INFO oslo.messaging._drivers.impl_rabbit [req-6f27b94c-4d14-475a-9bf3-a8cb3414176d ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:02.719 21872 INFO oslo.messaging._drivers.impl_rabbit [req-bc87dbab-9179-4f1d-9f4d-60cbc44d76f9 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:02.723 21868 INFO oslo.messaging._drivers.impl_rabbit [req-6f27b94c-4d14-475a-9bf3-a8cb3414176d ] Connected to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:02.725 21874 INFO oslo.messaging._drivers.impl_rabbit [req-ee4a0ff2-56eb-4776-b934-dc33d82004d2 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:02.725 21870 INFO oslo.messaging._drivers.impl_rabbit [req-1a3e8955-7de5-4901-81e6-f15ee9d82dcb ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:02.726 21871 INFO oslo.messaging._drivers.impl_rabbit [req-008cb521-cc6c-4da3-a0bf-6674ce830602 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:02.738 21875 INFO oslo.messaging._drivers.impl_rabbit [req-884de12b-a0a0-469c-af53-2ffde14c7754 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:02.741 21870 INFO oslo.messaging._drivers.impl_rabbit [req-1a3e8955-7de5-4901-81e6-f15ee9d82dcb ] Connected to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:02.741 21874 INFO oslo.messaging._drivers.impl_rabbit [req-ee4a0ff2-56eb-4776-b934-dc33d82004d2 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:02.744 21871 INFO oslo.messaging._drivers.impl_rabbit [req-008cb521-cc6c-4da3-a0bf-6674ce830602 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:02.750 21875 INFO oslo.messaging._drivers.impl_rabbit [req-884de12b-a0a0-469c-af53-2ffde14c7754 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:02.754 21872 INFO oslo.messaging._drivers.impl_rabbit [req-bc87dbab-9179-4f1d-9f4d-60cbc44d76f9 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:02.763 21869 INFO oslo.messaging._drivers.impl_rabbit [req-34598e13-6d58-43f7-a30e-af62ff242433 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:02.773 21869 INFO oslo.messaging._drivers.impl_rabbit [req-34598e13-6d58-43f7-a30e-af62ff242433 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:02.859 21873 INFO oslo.messaging._drivers.impl_rabbit [req-3632c1b8-0b42-4f8d-b77f-618a51bb0c0c ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:02.869 21873 INFO oslo.messaging._drivers.impl_rabbit [req-3632c1b8-0b42-4f8d-b77f-618a51bb0c0c ] Connected to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.188 21868 INFO oslo.messaging._drivers.impl_rabbit [-] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.195 21868 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.373 21870 INFO oslo.messaging._drivers.impl_rabbit [req-181b0c06-32c4-4a80-bcf9-0c87ee1ad2c9 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.382 21870 INFO oslo.messaging._drivers.impl_rabbit [req-181b0c06-32c4-4a80-bcf9-0c87ee1ad2c9 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.409 21874 INFO oslo.messaging._drivers.impl_rabbit [-] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.416 21874 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.437 21871 INFO oslo.messaging._drivers.impl_rabbit [-] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.446 21871 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.454 21875 INFO oslo.messaging._drivers.impl_rabbit [req-4aef5582-e526-483a-adfe-93e87eeb5314 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.462 21875 INFO oslo.messaging._drivers.impl_rabbit [req-4aef5582-e526-483a-adfe-93e87eeb5314 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.486 21872 INFO oslo.messaging._drivers.impl_rabbit [-] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.494 21872 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.564 21869 INFO oslo.messaging._drivers.impl_rabbit [-] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.571 21869 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.598 21873 INFO oslo.messaging._drivers.impl_rabbit [-] Connecting to AMQP server on 10.35.99.5:5672 2015-02-22 16:00:03.607 21873 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 10.35.99.5:5672 2015-02-23 09:33:13.212 21868 INFO oslo.messaging._drivers.impl_rabbit [req-22065f57-b055-4a50-9f06-d9e46a7a336f ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-23 09:33:13.220 21868 INFO oslo.messaging._drivers.impl_rabbit [req-22065f57-b055-4a50-9f06-d9e46a7a336f ] Connected to AMQP server on 10.35.99.5:5672 2015-02-23 09:33:35.941 21875 INFO oslo.messaging._drivers.impl_rabbit [req-ab93b695-e6e9-4e38-bee7-9e8e25dfcc59 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-23 09:33:35.949 21875 INFO oslo.messaging._drivers.impl_rabbit [req-ab93b695-e6e9-4e38-bee7-9e8e25dfcc59 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-23 09:48:43.825 21869 INFO oslo.messaging._drivers.impl_rabbit [req-a2bb17fe-ee2c-4b36-bcb0-8e84f9678488 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-23 09:48:43.833 21869 INFO oslo.messaging._drivers.impl_rabbit [req-a2bb17fe-ee2c-4b36-bcb0-8e84f9678488 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:20.829 21873 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting 2015-02-23 13:47:20.829 21870 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting 2015-02-23 13:47:20.829 21869 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting 2015-02-23 13:47:20.829 21871 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting 2015-02-23 13:47:20.829 21868 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting 2015-02-23 13:47:20.829 21872 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting 2015-02-23 13:47:20.830 21874 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting 2015-02-23 13:47:20.837 21875 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting 2015-02-23 13:47:20.837 21855 INFO nova.openstack.common.service [-] Caught SIGTERM, stopping children 2015-02-23 13:47:20.837 21855 INFO nova.openstack.common.service [-] Waiting on 8 children to exit 2015-02-23 13:47:20.838 21855 INFO nova.openstack.common.service [-] Child 21868 exited with status 1 2015-02-23 13:47:20.838 21855 INFO nova.openstack.common.service [-] Child 21869 exited with status 1 2015-02-23 13:47:20.838 21855 INFO nova.openstack.common.service [-] Child 21870 exited with status 1 2015-02-23 13:47:20.838 21855 INFO nova.openstack.common.service [-] Child 21871 exited with status 1 2015-02-23 13:47:20.839 21855 INFO nova.openstack.common.service [-] Child 21872 exited with status 1 2015-02-23 13:47:20.839 21855 INFO nova.openstack.common.service [-] Child 21873 exited with status 1 2015-02-23 13:47:20.839 21855 INFO nova.openstack.common.service [-] Child 21874 exited with status 1 2015-02-23 13:47:20.841 21855 INFO nova.openstack.common.service [-] Child 21875 killed by signal 15 2015-02-23 13:47:21.326 20509 INFO nova.openstack.common.service [-] Starting 8 workers 2015-02-23 13:47:21.327 20509 INFO nova.openstack.common.service [-] Started child 20533 2015-02-23 13:47:21.329 20533 AUDIT nova.service [-] Starting conductor node (version 2014.2.2-2.el7ost) 2015-02-23 13:47:21.329 20509 INFO nova.openstack.common.service [-] Started child 20534 2015-02-23 13:47:21.331 20534 AUDIT nova.service [-] Starting conductor node (version 2014.2.2-2.el7ost) 2015-02-23 13:47:21.331 20509 INFO nova.openstack.common.service [-] Started child 20535 2015-02-23 13:47:21.334 20535 AUDIT nova.service [-] Starting conductor node (version 2014.2.2-2.el7ost) 2015-02-23 13:47:21.334 20509 INFO nova.openstack.common.service [-] Started child 20536 2015-02-23 13:47:21.336 20536 AUDIT nova.service [-] Starting conductor node (version 2014.2.2-2.el7ost) 2015-02-23 13:47:21.337 20509 INFO nova.openstack.common.service [-] Started child 20537 2015-02-23 13:47:21.340 20537 AUDIT nova.service [-] Starting conductor node (version 2014.2.2-2.el7ost) 2015-02-23 13:47:21.341 20509 INFO nova.openstack.common.service [-] Started child 20538 2015-02-23 13:47:21.344 20538 AUDIT nova.service [-] Starting conductor node (version 2014.2.2-2.el7ost) 2015-02-23 13:47:21.344 20509 INFO nova.openstack.common.service [-] Started child 20539 2015-02-23 13:47:21.347 20539 AUDIT nova.service [-] Starting conductor node (version 2014.2.2-2.el7ost) 2015-02-23 13:47:21.348 20509 INFO nova.openstack.common.service [-] Started child 20540 2015-02-23 13:47:21.366 20540 AUDIT nova.service [-] Starting conductor node (version 2014.2.2-2.el7ost) 2015-02-23 13:47:22.007 20533 INFO oslo.messaging._drivers.impl_rabbit [req-d1fd29a4-9cf2-4ba3-a213-5560691a1437 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:22.007 20536 INFO oslo.messaging._drivers.impl_rabbit [req-5810c8b9-b3bc-4ae3-866c-d47b01fbc3d5 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:22.008 20535 INFO oslo.messaging._drivers.impl_rabbit [req-fe67e261-5374-4ede-ab57-33853bd9c3de ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:22.011 20534 INFO oslo.messaging._drivers.impl_rabbit [req-96d7ed79-cedf-4a00-82de-11a6d5265606 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:22.024 20538 INFO oslo.messaging._drivers.impl_rabbit [req-cd7dc24f-4949-4ec5-934e-59226d4e2366 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:22.029 20537 INFO oslo.messaging._drivers.impl_rabbit [req-d7491f82-5d9c-413c-af47-127d45b23f43 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:22.030 20536 INFO oslo.messaging._drivers.impl_rabbit [req-5810c8b9-b3bc-4ae3-866c-d47b01fbc3d5 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:22.032 20533 INFO oslo.messaging._drivers.impl_rabbit [req-d1fd29a4-9cf2-4ba3-a213-5560691a1437 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:22.040 20534 INFO oslo.messaging._drivers.impl_rabbit [req-96d7ed79-cedf-4a00-82de-11a6d5265606 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:22.043 20535 INFO oslo.messaging._drivers.impl_rabbit [req-fe67e261-5374-4ede-ab57-33853bd9c3de ] Connected to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:22.044 20538 INFO oslo.messaging._drivers.impl_rabbit [req-cd7dc24f-4949-4ec5-934e-59226d4e2366 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:22.047 20539 INFO oslo.messaging._drivers.impl_rabbit [req-fb38abaa-1d9b-47c1-8576-3646b3fddf96 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:22.048 20537 INFO oslo.messaging._drivers.impl_rabbit [req-d7491f82-5d9c-413c-af47-127d45b23f43 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:22.048 20540 INFO oslo.messaging._drivers.impl_rabbit [req-f720a991-0180-4cf0-a5e8-b537d980c335 ] Connecting to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:22.058 20539 INFO oslo.messaging._drivers.impl_rabbit [req-fb38abaa-1d9b-47c1-8576-3646b3fddf96 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-23 13:47:22.061 20540 INFO oslo.messaging._drivers.impl_rabbit [req-f720a991-0180-4cf0-a5e8-b537d980c335 ] Connected to AMQP server on 10.35.99.5:5672 2015-02-23 13:48:07.067 20533 ERROR nova.servicegroup.drivers.db [-] model server went away 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db Traceback (most recent call last): 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/servicegroup/drivers/db.py", line 99, in _report_state 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db service.service_ref, state_catalog) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/api.py", line 180, in service_update 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return self._manager.service_update(context, service, values) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/utils.py", line 949, in wrapper 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/server.py", line 139, in inner 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 320, in service_update 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db svc = self.db.service_update(context, service['id'], values) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/api.py", line 150, in service_update 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return IMPL.service_update(context, service_id, values) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 126, in wrapper 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 182, in wrapped 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 525, in service_update 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db with_compute_node=False, session=session) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 425, in _service_get 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db result = query.first() 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2341, in first 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db ret = list(self[0:1]) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2208, in __getitem__ 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return list(res) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2412, in __iter__ 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return self._execute_and_instances(context) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2425, in _execute_and_instances 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db close_with_result=True) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2416, in _connection_from_session 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db **kw) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 854, in connection 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db close_with_result=close_with_result) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 858, in _connection_for_bind 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return self.transaction._connection_for_bind(engine) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 322, in _connection_for_bind 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db conn = bind.contextual_connect() 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1801, in contextual_connect 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db **kwargs) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 85, in __init__ 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db self.dispatch.engine_connect(self, _branch) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 260, in __call__ 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db fn(*args, **kw) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py", line 342, in _connect_ping_listener 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db connection.scalar(select([1])) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 659, in scalar 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return self.execute(object, *multiparams, **params).scalar() 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 729, in execute 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return meth(self, multiparams, params) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 321, in _execute_on_connection 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return connection._execute_clauseelement(self, multiparams, params) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db compiled_sql, distilled_params 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 893, in _execute_context 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db None, None) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1169, in _handle_dbapi_exception 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db dbapi_conn_wrapper = self.connection 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 236, in connection 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return self._revalidate_connection() 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 244, in _revalidate_connection 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db self.__connection = self.engine.raw_connection() 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1848, in raw_connection 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return self.pool.unique_connection() 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 280, in unique_connection 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return _ConnectionFairy._checkout(self) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 641, in _checkout 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db fairy = _ConnectionRecord.checkout(pool) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 442, in checkout 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db dbapi_connection = rec.get_connection() 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 505, in get_connection 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db self.connection = self.__connect() 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 537, in __connect 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db connection = self.__pool._creator() 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 96, in connect 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db connection_invalidated=invalidated 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db reraise(type(exception), exception, tb=exc_tb) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 90, in connect 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return dialect.connect(*cargs, **cparams) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 377, in connect 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return self.dbapi.connect(*cargs, **cparams) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db return Connection(*args, **kwargs) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 187, in __init__ 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db super(Connection, self).__init__(*args, **kwargs2) 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db OperationalError: (OperationalError) (2003, "Can't connect to MySQL server on '10.35.99.5' (111)") None None 2015-02-23 13:48:07.067 20533 TRACE nova.servicegroup.drivers.db 2015-02-23 13:48:07.071 20536 ERROR nova.servicegroup.drivers.db [-] model server went away 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db Traceback (most recent call last): 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/servicegroup/drivers/db.py", line 99, in _report_state 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db service.service_ref, state_catalog) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/api.py", line 180, in service_update 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return self._manager.service_update(context, service, values) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/utils.py", line 949, in wrapper 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/server.py", line 139, in inner 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 320, in service_update 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db svc = self.db.service_update(context, service['id'], values) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/api.py", line 150, in service_update 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return IMPL.service_update(context, service_id, values) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 126, in wrapper 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 182, in wrapped 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 525, in service_update 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db with_compute_node=False, session=session) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 425, in _service_get 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db result = query.first() 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2341, in first 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db ret = list(self[0:1]) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2208, in __getitem__ 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return list(res) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2412, in __iter__ 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return self._execute_and_instances(context) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2425, in _execute_and_instances 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db close_with_result=True) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2416, in _connection_from_session 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db **kw) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 854, in connection 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db close_with_result=close_with_result) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 858, in _connection_for_bind 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return self.transaction._connection_for_bind(engine) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 322, in _connection_for_bind 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db conn = bind.contextual_connect() 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1801, in contextual_connect 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db **kwargs) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 85, in __init__ 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db self.dispatch.engine_connect(self, _branch) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 260, in __call__ 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db fn(*args, **kw) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py", line 342, in _connect_ping_listener 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db connection.scalar(select([1])) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 659, in scalar 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return self.execute(object, *multiparams, **params).scalar() 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 729, in execute 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return meth(self, multiparams, params) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 321, in _execute_on_connection 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return connection._execute_clauseelement(self, multiparams, params) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db compiled_sql, distilled_params 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 893, in _execute_context 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db None, None) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1169, in _handle_dbapi_exception 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db dbapi_conn_wrapper = self.connection 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 236, in connection 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return self._revalidate_connection() 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 244, in _revalidate_connection 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db self.__connection = self.engine.raw_connection() 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1848, in raw_connection 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return self.pool.unique_connection() 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 280, in unique_connection 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return _ConnectionFairy._checkout(self) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 641, in _checkout 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db fairy = _ConnectionRecord.checkout(pool) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 442, in checkout 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db dbapi_connection = rec.get_connection() 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 505, in get_connection 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db self.connection = self.__connect() 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 537, in __connect 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db connection = self.__pool._creator() 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 96, in connect 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db connection_invalidated=invalidated 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db reraise(type(exception), exception, tb=exc_tb) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 90, in connect 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return dialect.connect(*cargs, **cparams) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 377, in connect 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return self.dbapi.connect(*cargs, **cparams) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db return Connection(*args, **kwargs) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 187, in __init__ 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db super(Connection, self).__init__(*args, **kwargs2) 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db OperationalError: (OperationalError) (2003, "Can't connect to MySQL server on '10.35.99.5' (111)") None None 2015-02-23 13:48:07.071 20536 TRACE nova.servicegroup.drivers.db 2015-02-23 13:48:07.083 20535 ERROR nova.servicegroup.drivers.db [-] model server went away 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db Traceback (most recent call last): 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/servicegroup/drivers/db.py", line 99, in _report_state 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db service.service_ref, state_catalog) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/api.py", line 180, in service_update 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return self._manager.service_update(context, service, values) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/utils.py", line 949, in wrapper 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/server.py", line 139, in inner 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 320, in service_update 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db svc = self.db.service_update(context, service['id'], values) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/api.py", line 150, in service_update 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return IMPL.service_update(context, service_id, values) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 126, in wrapper 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 182, in wrapped 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 525, in service_update 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db with_compute_node=False, session=session) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 425, in _service_get 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db result = query.first() 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2341, in first 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db ret = list(self[0:1]) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2208, in __getitem__ 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return list(res) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2412, in __iter__ 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return self._execute_and_instances(context) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2425, in _execute_and_instances 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db close_with_result=True) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2416, in _connection_from_session 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db **kw) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 854, in connection 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db close_with_result=close_with_result) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 858, in _connection_for_bind 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return self.transaction._connection_for_bind(engine) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 322, in _connection_for_bind 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db conn = bind.contextual_connect() 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1801, in contextual_connect 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db **kwargs) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 85, in __init__ 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db self.dispatch.engine_connect(self, _branch) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 260, in __call__ 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db fn(*args, **kw) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py", line 342, in _connect_ping_listener 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db connection.scalar(select([1])) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 659, in scalar 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return self.execute(object, *multiparams, **params).scalar() 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 729, in execute 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return meth(self, multiparams, params) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 321, in _execute_on_connection 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return connection._execute_clauseelement(self, multiparams, params) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db compiled_sql, distilled_params 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 893, in _execute_context 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db None, None) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1169, in _handle_dbapi_exception 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db dbapi_conn_wrapper = self.connection 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 236, in connection 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return self._revalidate_connection() 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 244, in _revalidate_connection 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db self.__connection = self.engine.raw_connection() 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1848, in raw_connection 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return self.pool.unique_connection() 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 280, in unique_connection 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return _ConnectionFairy._checkout(self) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 641, in _checkout 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db fairy = _ConnectionRecord.checkout(pool) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 442, in checkout 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db dbapi_connection = rec.get_connection() 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 505, in get_connection 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db self.connection = self.__connect() 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 537, in __connect 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db connection = self.__pool._creator() 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 96, in connect 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db connection_invalidated=invalidated 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db reraise(type(exception), exception, tb=exc_tb) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 90, in connect 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return dialect.connect(*cargs, **cparams) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 377, in connect 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return self.dbapi.connect(*cargs, **cparams) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db return Connection(*args, **kwargs) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 187, in __init__ 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db super(Connection, self).__init__(*args, **kwargs2) 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db OperationalError: (OperationalError) (2003, "Can't connect to MySQL server on '10.35.99.5' (111)") None None 2015-02-23 13:48:07.083 20535 TRACE nova.servicegroup.drivers.db 2015-02-23 13:48:07.087 20539 ERROR nova.servicegroup.drivers.db [-] model server went away 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db Traceback (most recent call last): 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/servicegroup/drivers/db.py", line 99, in _report_state 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db service.service_ref, state_catalog) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/api.py", line 180, in service_update 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return self._manager.service_update(context, service, values) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/utils.py", line 949, in wrapper 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/server.py", line 139, in inner 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 320, in service_update 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db svc = self.db.service_update(context, service['id'], values) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/api.py", line 150, in service_update 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return IMPL.service_update(context, service_id, values) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 126, in wrapper 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 182, in wrapped 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 525, in service_update 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db with_compute_node=False, session=session) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 425, in _service_get 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db result = query.first() 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2341, in first 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db ret = list(self[0:1]) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2208, in __getitem__ 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return list(res) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2412, in __iter__ 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return self._execute_and_instances(context) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2425, in _execute_and_instances 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db close_with_result=True) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2416, in _connection_from_session 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db **kw) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 854, in connection 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db close_with_result=close_with_result) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 858, in _connection_for_bind 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return self.transaction._connection_for_bind(engine) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 322, in _connection_for_bind 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db conn = bind.contextual_connect() 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1801, in contextual_connect 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db **kwargs) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 85, in __init__ 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db self.dispatch.engine_connect(self, _branch) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 260, in __call__ 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db fn(*args, **kw) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py", line 342, in _connect_ping_listener 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db connection.scalar(select([1])) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 659, in scalar 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return self.execute(object, *multiparams, **params).scalar() 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 729, in execute 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return meth(self, multiparams, params) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 321, in _execute_on_connection 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return connection._execute_clauseelement(self, multiparams, params) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db compiled_sql, distilled_params 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 893, in _execute_context 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db None, None) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1169, in _handle_dbapi_exception 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db dbapi_conn_wrapper = self.connection 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 236, in connection 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return self._revalidate_connection() 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 244, in _revalidate_connection 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db self.__connection = self.engine.raw_connection() 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1848, in raw_connection 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return self.pool.unique_connection() 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 280, in unique_connection 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return _ConnectionFairy._checkout(self) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 641, in _checkout 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db fairy = _ConnectionRecord.checkout(pool) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 442, in checkout 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db dbapi_connection = rec.get_connection() 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 505, in get_connection 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db self.connection = self.__connect() 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 537, in __connect 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db connection = self.__pool._creator() 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 96, in connect 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db connection_invalidated=invalidated 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db reraise(type(exception), exception, tb=exc_tb) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 90, in connect 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return dialect.connect(*cargs, **cparams) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 377, in connect 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return self.dbapi.connect(*cargs, **cparams) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db return Connection(*args, **kwargs) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 187, in __init__ 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db super(Connection, self).__init__(*args, **kwargs2) 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db OperationalError: (OperationalError) (2003, "Can't connect to MySQL server on '10.35.99.5' (111)") None None 2015-02-23 13:48:07.087 20539 TRACE nova.servicegroup.drivers.db 2015-02-23 13:48:07.089 20538 ERROR nova.servicegroup.drivers.db [-] model server went away 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db Traceback (most recent call last): 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/servicegroup/drivers/db.py", line 99, in _report_state 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db service.service_ref, state_catalog) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/api.py", line 180, in service_update 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return self._manager.service_update(context, service, values) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/utils.py", line 949, in wrapper 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/server.py", line 139, in inner 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 320, in service_update 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db svc = self.db.service_update(context, service['id'], values) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/api.py", line 150, in service_update 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return IMPL.service_update(context, service_id, values) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 126, in wrapper 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 182, in wrapped 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 525, in service_update 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db with_compute_node=False, session=session) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 425, in _service_get 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db result = query.first() 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2341, in first 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db ret = list(self[0:1]) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2208, in __getitem__ 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return list(res) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2412, in __iter__ 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return self._execute_and_instances(context) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2425, in _execute_and_instances 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db close_with_result=True) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2416, in _connection_from_session 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db **kw) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 854, in connection 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db close_with_result=close_with_result) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 858, in _connection_for_bind 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return self.transaction._connection_for_bind(engine) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 322, in _connection_for_bind 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db conn = bind.contextual_connect() 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1801, in contextual_connect 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db **kwargs) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 85, in __init__ 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db self.dispatch.engine_connect(self, _branch) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 260, in __call__ 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db fn(*args, **kw) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py", line 342, in _connect_ping_listener 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db connection.scalar(select([1])) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 659, in scalar 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return self.execute(object, *multiparams, **params).scalar() 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 729, in execute 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return meth(self, multiparams, params) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 321, in _execute_on_connection 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return connection._execute_clauseelement(self, multiparams, params) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db compiled_sql, distilled_params 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 893, in _execute_context 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db None, None) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1169, in _handle_dbapi_exception 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db dbapi_conn_wrapper = self.connection 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 236, in connection 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return self._revalidate_connection() 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 244, in _revalidate_connection 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db self.__connection = self.engine.raw_connection() 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1848, in raw_connection 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return self.pool.unique_connection() 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 280, in unique_connection 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return _ConnectionFairy._checkout(self) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 641, in _checkout 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db fairy = _ConnectionRecord.checkout(pool) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 442, in checkout 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db dbapi_connection = rec.get_connection() 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 505, in get_connection 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db self.connection = self.__connect() 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 537, in __connect 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db connection = self.__pool._creator() 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 96, in connect 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db connection_invalidated=invalidated 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db reraise(type(exception), exception, tb=exc_tb) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 90, in connect 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return dialect.connect(*cargs, **cparams) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 377, in connect 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return self.dbapi.connect(*cargs, **cparams) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db return Connection(*args, **kwargs) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 187, in __init__ 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db super(Connection, self).__init__(*args, **kwargs2) 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db OperationalError: (OperationalError) (2003, "Can't connect to MySQL server on '10.35.99.5' (111)") None None 2015-02-23 13:48:07.089 20538 TRACE nova.servicegroup.drivers.db 2015-02-23 13:48:07.092 20537 ERROR nova.servicegroup.drivers.db [-] model server went away 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db Traceback (most recent call last): 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/servicegroup/drivers/db.py", line 99, in _report_state 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db service.service_ref, state_catalog) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/api.py", line 180, in service_update 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return self._manager.service_update(context, service, values) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/utils.py", line 949, in wrapper 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/server.py", line 139, in inner 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 320, in service_update 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db svc = self.db.service_update(context, service['id'], values) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/api.py", line 150, in service_update 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return IMPL.service_update(context, service_id, values) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 126, in wrapper 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 182, in wrapped 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 525, in service_update 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db with_compute_node=False, session=session) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 425, in _service_get 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db result = query.first() 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2341, in first 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db ret = list(self[0:1]) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2208, in __getitem__ 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return list(res) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2412, in __iter__ 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return self._execute_and_instances(context) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2425, in _execute_and_instances 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db close_with_result=True) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2416, in _connection_from_session 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db **kw) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 854, in connection 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db close_with_result=close_with_result) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 858, in _connection_for_bind 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return self.transaction._connection_for_bind(engine) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 322, in _connection_for_bind 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db conn = bind.contextual_connect() 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1801, in contextual_connect 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db **kwargs) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 85, in __init__ 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db self.dispatch.engine_connect(self, _branch) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 260, in __call__ 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db fn(*args, **kw) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py", line 342, in _connect_ping_listener 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db connection.scalar(select([1])) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 659, in scalar 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return self.execute(object, *multiparams, **params).scalar() 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 729, in execute 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return meth(self, multiparams, params) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 321, in _execute_on_connection 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return connection._execute_clauseelement(self, multiparams, params) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db compiled_sql, distilled_params 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 893, in _execute_context 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db None, None) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1169, in _handle_dbapi_exception 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db dbapi_conn_wrapper = self.connection 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 236, in connection 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return self._revalidate_connection() 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 244, in _revalidate_connection 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db self.__connection = self.engine.raw_connection() 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1848, in raw_connection 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return self.pool.unique_connection() 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 280, in unique_connection 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return _ConnectionFairy._checkout(self) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 641, in _checkout 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db fairy = _ConnectionRecord.checkout(pool) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 442, in checkout 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db dbapi_connection = rec.get_connection() 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 505, in get_connection 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db self.connection = self.__connect() 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 537, in __connect 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db connection = self.__pool._creator() 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 96, in connect 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db connection_invalidated=invalidated 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db reraise(type(exception), exception, tb=exc_tb) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 90, in connect 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return dialect.connect(*cargs, **cparams) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 377, in connect 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return self.dbapi.connect(*cargs, **cparams) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db return Connection(*args, **kwargs) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 187, in __init__ 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db super(Connection, self).__init__(*args, **kwargs2) 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db OperationalError: (OperationalError) (2003, "Can't connect to MySQL server on '10.35.99.5' (111)") None None 2015-02-23 13:48:07.092 20537 TRACE nova.servicegroup.drivers.db 2015-02-23 13:48:07.093 20534 ERROR nova.servicegroup.drivers.db [-] model server went away 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db Traceback (most recent call last): 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/servicegroup/drivers/db.py", line 99, in _report_state 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db service.service_ref, state_catalog) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/api.py", line 180, in service_update 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return self._manager.service_update(context, service, values) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/utils.py", line 949, in wrapper 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/server.py", line 139, in inner 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 320, in service_update 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db svc = self.db.service_update(context, service['id'], values) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/api.py", line 150, in service_update 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return IMPL.service_update(context, service_id, values) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 126, in wrapper 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 182, in wrapped 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 525, in service_update 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db with_compute_node=False, session=session) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 425, in _service_get 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db result = query.first() 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2341, in first 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db ret = list(self[0:1]) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2208, in __getitem__ 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return list(res) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2412, in __iter__ 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return self._execute_and_instances(context) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2425, in _execute_and_instances 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db close_with_result=True) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2416, in _connection_from_session 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db **kw) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 854, in connection 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db close_with_result=close_with_result) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 858, in _connection_for_bind 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return self.transaction._connection_for_bind(engine) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 322, in _connection_for_bind 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db conn = bind.contextual_connect() 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1801, in contextual_connect 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db **kwargs) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 85, in __init__ 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db self.dispatch.engine_connect(self, _branch) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 260, in __call__ 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db fn(*args, **kw) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py", line 342, in _connect_ping_listener 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db connection.scalar(select([1])) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 659, in scalar 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return self.execute(object, *multiparams, **params).scalar() 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 729, in execute 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return meth(self, multiparams, params) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 321, in _execute_on_connection 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return connection._execute_clauseelement(self, multiparams, params) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db compiled_sql, distilled_params 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 893, in _execute_context 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db None, None) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1169, in _handle_dbapi_exception 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db dbapi_conn_wrapper = self.connection 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 236, in connection 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return self._revalidate_connection() 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 244, in _revalidate_connection 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db self.__connection = self.engine.raw_connection() 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1848, in raw_connection 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return self.pool.unique_connection() 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 280, in unique_connection 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return _ConnectionFairy._checkout(self) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 641, in _checkout 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db fairy = _ConnectionRecord.checkout(pool) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 442, in checkout 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db dbapi_connection = rec.get_connection() 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 505, in get_connection 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db self.connection = self.__connect() 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 537, in __connect 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db connection = self.__pool._creator() 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 96, in connect 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db connection_invalidated=invalidated 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db reraise(type(exception), exception, tb=exc_tb) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 90, in connect 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return dialect.connect(*cargs, **cparams) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 377, in connect 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return self.dbapi.connect(*cargs, **cparams) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db return Connection(*args, **kwargs) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 187, in __init__ 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db super(Connection, self).__init__(*args, **kwargs2) 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db OperationalError: (OperationalError) (2003, "Can't connect to MySQL server on '10.35.99.5' (111)") None None 2015-02-23 13:48:07.093 20534 TRACE nova.servicegroup.drivers.db 2015-02-23 13:48:07.102 20540 ERROR nova.servicegroup.drivers.db [-] model server went away 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db Traceback (most recent call last): 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/servicegroup/drivers/db.py", line 99, in _report_state 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db service.service_ref, state_catalog) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/api.py", line 180, in service_update 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return self._manager.service_update(context, service, values) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/utils.py", line 949, in wrapper 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/server.py", line 139, in inner 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 320, in service_update 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db svc = self.db.service_update(context, service['id'], values) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/api.py", line 150, in service_update 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return IMPL.service_update(context, service_id, values) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 126, in wrapper 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 182, in wrapped 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 525, in service_update 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db with_compute_node=False, session=session) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 425, in _service_get 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db result = query.first() 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2341, in first 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db ret = list(self[0:1]) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2208, in __getitem__ 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return list(res) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2412, in __iter__ 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return self._execute_and_instances(context) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2425, in _execute_and_instances 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db close_with_result=True) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2416, in _connection_from_session 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db **kw) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 854, in connection 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db close_with_result=close_with_result) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 858, in _connection_for_bind 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return self.transaction._connection_for_bind(engine) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 322, in _connection_for_bind 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db conn = bind.contextual_connect() 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1801, in contextual_connect 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db **kwargs) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 85, in __init__ 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db self.dispatch.engine_connect(self, _branch) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 260, in __call__ 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db fn(*args, **kw) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py", line 342, in _connect_ping_listener 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db connection.scalar(select([1])) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 659, in scalar 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return self.execute(object, *multiparams, **params).scalar() 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 729, in execute 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return meth(self, multiparams, params) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 321, in _execute_on_connection 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return connection._execute_clauseelement(self, multiparams, params) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db compiled_sql, distilled_params 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 893, in _execute_context 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db None, None) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1169, in _handle_dbapi_exception 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db dbapi_conn_wrapper = self.connection 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 236, in connection 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return self._revalidate_connection() 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 244, in _revalidate_connection 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db self.__connection = self.engine.raw_connection() 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1848, in raw_connection 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return self.pool.unique_connection() 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 280, in unique_connection 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return _ConnectionFairy._checkout(self) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 641, in _checkout 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db fairy = _ConnectionRecord.checkout(pool) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 442, in checkout 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db dbapi_connection = rec.get_connection() 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 505, in get_connection 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db self.connection = self.__connect() 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 537, in __connect 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db connection = self.__pool._creator() 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 96, in connect 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db connection_invalidated=invalidated 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db reraise(type(exception), exception, tb=exc_tb) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 90, in connect 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return dialect.connect(*cargs, **cparams) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 377, in connect 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return self.dbapi.connect(*cargs, **cparams) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db return Connection(*args, **kwargs) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 187, in __init__ 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db super(Connection, self).__init__(*args, **kwargs2) 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db OperationalError: (OperationalError) (2003, "Can't connect to MySQL server on '10.35.99.5' (111)") None None 2015-02-23 13:48:07.102 20540 TRACE nova.servicegroup.drivers.db 2015-02-23 13:48:17.092 20533 ERROR nova.servicegroup.drivers.db [-] Recovered model server connection! 2015-02-23 13:48:17.094 20535 ERROR nova.servicegroup.drivers.db [-] Recovered model server connection! 2015-02-23 13:48:17.099 20539 ERROR nova.servicegroup.drivers.db [-] Recovered model server connection! 2015-02-23 13:48:17.099 20534 ERROR nova.servicegroup.drivers.db [-] Recovered model server connection! 2015-02-23 13:48:17.101 20538 ERROR nova.servicegroup.drivers.db [-] Recovered model server connection! 2015-02-23 13:48:17.104 20537 ERROR nova.servicegroup.drivers.db [-] Recovered model server connection! 2015-02-23 13:48:17.104 20540 ERROR nova.servicegroup.drivers.db [-] Recovered model server connection! 2015-02-23 13:48:17.108 20536 ERROR nova.servicegroup.drivers.db [-] Recovered model server connection!