-
Type:
Task
-
Resolution: Duplicate
-
Priority:
Major - P3
-
None
-
Affects Version/s: 2.9.0
-
Component/s: None
-
None
-
Environment:SO: Ubuntu Server 16.04 LTS
Provider: Azure
Mongo Version: 3.4
We are having issue in produccion aplication wich It appears several times a day.
The primary node is reported as UNKNOW.
Exception:
No primary server is available in cluster: #<Cluster topology=ReplicaSetNoPrimary[10.240.0.6:27017,mongodb02moverickprod:27017,mongodb03moverickprod:27017,name=moverick,v=3,e=7fffffff000000000000000e] servers=[#<Server address=10.240.0.6:27017 SECONDARY replica_set=moverick>,#<Server address=mongodb02moverickprod:27017 UNKNOWN>,#<Server address=mongodb03moverickprod:27017 SECONDARY replica_set=moverick>]> with timeout=30, LT=0.015. The following servers have dead monitor threads: #<Server address=10.240.0.6:27017 SECONDARY replica_set=moverick>, #<Server address=mongodb02moverickprod:27017 UNKNOWN>, #<Server address=mongodb03moverickprod:27017 SECONDARY replica_set=moverick>
Backtrace:
mongo-2.9.0/lib/mongo/server_selector/selectable.rb:184→ select_server mongo-2.9.0/lib/mongo/cluster.rb:531→ next_primary mongo-2.9.0/lib/mongo/retryable.rb:335→ legacy_write_with_retry mongo-2.9.0/lib/mongo/retryable.rb:197→ write_with_retry mongo-2.9.0/lib/mongo/collection/view/writable.rb:295→ block in update_one mongo-2.9.0/lib/mongo/cluster.rb:662→ with_session mongo-2.9.0/lib/mongo/client.rb:733→ with_session mongo-2.9.0/lib/mongo/collection/view.rb:204→ with_session mongo-2.9.0/lib/mongo/collection/view/writable.rb:294→ update_one mongoid-6.4.4/lib/mongoid/query_cache.rb:182→ update_one_with_clear_cache mongoid-6.4.4/lib/mongoid/persistable/updatable.rb:137→ block in update_document mongoid-6.4.4/lib/mongoid/persistable/updatable.rb:112→ block (2 levels) in prepare_update activesupport-5.1.6/lib/active_support/callbacks.rb:131→ run_callbacks mongoid-6.4.4/lib/mongoid/interceptable.rb:132→ run_callbacks mongoid-6.4.4/lib/mongoid/persistable/updatable.rb:111→ block in prepare_update activesupport-5.1.6/lib/active_support/callbacks.rb:131→ run_callbacks mongoid-6.4.4/lib/mongoid/interceptable.rb:132→ run_callbacks mongoid-6.4.4/lib/mongoid/persistable/updatable.rb:110→ prepare_update mongoid-6.4.4/lib/mongoid/persistable/updatable.rb:132→ update_document mongoid-6.4.4/lib/mongoid/persistable/savable.rb:25→ save rails_jwt_auth-0.21.0/app/models/concerns/rails_jwt_auth/trackable.rb:6→ update_tracked_fields! rails_jwt_auth-0.21.0/lib/rails_jwt_auth/engine.rb:20→ block (2 levels) in <class:Engine> warden-1.2.7/lib/warden/hooks.rb:15→ block in _run_callbacks warden-1.2.7/lib/warden/hooks.rb:10→ each warden-1.2.7/lib/warden/hooks.rb:10→ _run_callbacks warden-1.2.7/lib/warden/manager.rb:52→ _run_callbacks warden-1.2.7/lib/warden/proxy.rb:180→ set_user warden-1.2.7/lib/warden/proxy.rb:328→ _perform_authentication warden-1.2.7/lib/warden/proxy.rb:128→ authenticate! rails_jwt_auth-0.21.0/app/controllers/concerns/rails_jwt_auth/warden_helper.rb:16→ authenticate! activesupport-5.1.6/lib/active_support/callbacks.rb:413→ block in make_lambda activesupport-5.1.6/lib/active_support/callbacks.rb:178→ block (2 levels) in halting_and_conditional actionpack-5.1.6/lib/abstract_controller/callbacks.rb:12→ block (2 levels) in <module:Callbacks> activesupport-5.1.6/lib/active_support/callbacks.rb:179→ block in halting_and_conditional activesupport-5.1.6/lib/active_support/callbacks.rb:507→ block in invoke_before activesupport-5.1.6/lib/active_support/callbacks.rb:507→ each activesupport-5.1.6/lib/active_support/callbacks.rb:507→ invoke_before activesupport-5.1.6/lib/active_support/callbacks.rb:130→ run_callbacks actionpack-5.1.6/lib/abstract_controller/callbacks.rb:19→ process_action actionpack-5.1.6/lib/action_controller/metal/rescue.rb:20→ process_action actionpack-5.1.6/lib/action_controller/metal/instrumentation.rb:32→ block in process_action activesupport-5.1.6/lib/active_support/notifications.rb:166→ block in instrument activesupport-5.1.6/lib/active_support/notifications/instrumenter.rb:21→ instrument activesupport-5.1.6/lib/active_support/notifications.rb:166→ instrument actionpack-5.1.6/lib/action_controller/metal/instrumentation.rb:30→ process_action actionpack-5.1.6/lib/action_controller/metal/params_wrapper.rb:252→ process_action mongoid-6.4.4/lib/mongoid/railties/controller_runtime.rb:19→ process_action actionpack-5.1.6/lib/abstract_controller/base.rb:124→ process actionpack-5.1.6/lib/action_controller/metal.rb:189→ dispatch actionpack-5.1.6/lib/action_controller/metal.rb:253→ dispatch actionpack-5.1.6/lib/action_dispatch/routing/route_set.rb:49→ dispatch actionpack-5.1.6/lib/action_dispatch/routing/route_set.rb:31→ serve actionpack-5.1.6/lib/action_dispatch/journey/router.rb:50→ block in serve actionpack-5.1.6/lib/action_dispatch/journey/router.rb:33→ each actionpack-5.1.6/lib/action_dispatch/journey/router.rb:33→ serve actionpack-5.1.6/lib/action_dispatch/routing/route_set.rb:844→ call rack-2.0.5/lib/rack/deflater.rb:34→ call rack-2.0.5/lib/rack/etag.rb:25→ call rack-2.0.5/lib/rack/conditional_get.rb:38→ call rack-2.0.5/lib/rack/head.rb:12→ call warden-1.2.7/lib/warden/manager.rb:36→ block in call warden-1.2.7/lib/warden/manager.rb:35→ catch warden-1.2.7/lib/warden/manager.rb:35→ call actionpack-5.1.6/lib/action_dispatch/middleware/callbacks.rb:26→ block in call activesupport-5.1.6/lib/active_support/callbacks.rb:97→ run_callbacks actionpack-5.1.6/lib/action_dispatch/middleware/callbacks.rb:24→ call airbrake-5.8.1/lib/airbrake/rack/middleware.rb:52→ call actionpack-5.1.6/lib/action_dispatch/middleware/debug_exceptions.rb:59→ call actionpack-5.1.6/lib/action_dispatch/middleware/show_exceptions.rb:31→ call rails_semantic_logger-4.3.2/lib/rails_semantic_logger/rack/logger.rb:43→ call_app rails_semantic_logger-4.3.2/lib/rails_semantic_logger/rack/logger.rb:28→ call actionpack-5.1.6/lib/action_dispatch/middleware/remote_ip.rb:79→ call actionpack-5.1.6/lib/action_dispatch/middleware/request_id.rb:25→ call rack-2.0.5/lib/rack/runtime.rb:22→ call activesupport-5.1.6/lib/active_support/cache/strategy/local_cache_middleware.rb:27→ call actionpack-5.1.6/lib/action_dispatch/middleware/executor.rb:12→ call actionpack-5.1.6/lib/action_dispatch/middleware/static.rb:125→ call rack-2.0.5/lib/rack/sendfile.rb:111→ call rack-cors-1.0.2/lib/rack/cors.rb:97→ call railties-5.1.6/lib/rails/engine.rb:522→ call rack-2.0.5/lib/rack/urlmap.rb:68→ block in call rack-2.0.5/lib/rack/urlmap.rb:53→ each rack-2.0.5/lib/rack/urlmap.rb:53→ call rack-2.0.5/lib/rack/tempfile_reaper.rb:15→ call rack-2.0.5/lib/rack/lint.rb:49→ _call rack-2.0.5/lib/rack/lint.rb:37→ call rack-2.0.5/lib/rack/show_exceptions.rb:23→ call rack-2.0.5/lib/rack/common_logger.rb:33→ call rack-2.0.5/lib/rack/chunked.rb:54→ call rack-2.0.5/lib/rack/content_length.rb:15→ call unicorn-5.3.1/lib/unicorn/http_server.rb:606→ process_client unicorn-5.3.1/lib/unicorn/http_server.rb:702→ worker_loop unicorn-5.3.1/lib/unicorn/http_server.rb:549→ spawn_missing_workers unicorn-5.3.1/lib/unicorn/http_server.rb:142→ start unicorn-5.3.1/bin/unicorn:126→ <top (required)> [GEM_ROOT]/bin/unicorn:23→ load [GEM_ROOT]/bin/unicorn:23→ <top (required)> /usr/local/lib/ruby/site_ruby/2.4.0/bundler/cli/exec.rb:74→ load /usr/local/lib/ruby/site_ruby/2.4.0/bundler/cli/exec.rb:74→ kernel_load /usr/local/lib/ruby/site_ruby/2.4.0/bundler/cli/exec.rb:28→ run /usr/local/lib/ruby/site_ruby/2.4.0/bundler/cli.rb:463→ exec /usr/local/lib/ruby/site_ruby/2.4.0/bundler/vendor/thor/lib/thor/command.rb:27→ run /usr/local/lib/ruby/site_ruby/2.4.0/bundler/vendor/thor/lib/thor/invocation.rb:126→ invoke_command /usr/local/lib/ruby/site_ruby/2.4.0/bundler/vendor/thor/lib/thor.rb:387→ dispatch /usr/local/lib/ruby/site_ruby/2.4.0/bundler/cli.rb:27→ dispatch /usr/local/lib/ruby/site_ruby/2.4.0/bundler/vendor/thor/lib/thor/base.rb:466→ start /usr/local/lib/ruby/site_ruby/2.4.0/bundler/cli.rb:18→ start /usr/local/bin/bundle:30→ block in <main> /usr/local/lib/ruby/site_ruby/2.4.0/bundler/friendly_errors.rb:124→ with_friendly_errors /usr/local/bin/bundle:22→ <main>
The error appears randomly. I attach an image where you can see that in first line a request is received and a minute later the error is raised (second line). Then the error is repeated again (line 3 and 4) and then one request is processed correctly (line 5 and 6). How you can see, there are working requests interspersed by error requests.
Betweend the interval of request is received and error apears (1 minute), other requests in diferent workers are working without any error so It does not seem mongo error (monitor does not show any fail or hight consumption of mongodb).
- duplicates
-
-
- Closed
-