Skip to content

DragonFly service got crashed with an assertion failure error #6168

@saisreenivasb

Description

@saisreenivasb

Describe the bug
A clear and concise description of what the bug is.

We got below error in our error logs:

F20251206 06:29:14.784060 10714 db_slice.cc:820] Check failed: db.expire.Insert(main_it->first.AsRef(), ExpirePeriod(delta)).second *** Check failure stack trace: *** @ 0x5b2e60d77923 google::LogMessage::SendToLog() @ 0x5b2e60d700e7 google::LogMessage::Flush() @ 0x5b2e60d71a6f google::LogMessageFatal::~LogMessageFatal() @ 0x5b2e605b9338 dfly::DbSlice::AddExpire() @ 0x5b2e603da631 dfly::(anonymous namespace)::SetCmd::Set() @ 0x5b2e603da7cf _ZN4absl12lts_2024011619functional_internal12InvokeObjectIZN4dfly12_GLOBAL__N_110SetGenericERKNS4_6SetCmd9SetParamsESt17basic_string_viewIcSt11char_traitsIcEESC_bPNS3_11TransactionEEUlSE_PNS3_11EngineShardEE_NSD_14RunnableResultEJSE_SG_EEET0_NS1_7VoidPtrEDpNS1_8ForwardTIT1_E4typeE @ 0x5b2e605efc19 dfly::Transaction::RunCallback() @ 0x5b2e605f07a9 dfly::Transaction::ScheduleInShard() @ 0x5b2e605f1762 dfly::Transaction::ScheduleInternal() @ 0x5b2e605f23f1 dfly::Transaction::Execute() @ 0x5b2e605f250a dfly::Transaction::ScheduleSingleHop() @ 0x5b2e603d33ae dfly::StringFamily::Set() @ 0x5b2e605d2faf dfly::CommandId::Invoke() @ 0x5b2e602d4a73 dfly::Service::InvokeCmd() @ 0x5b2e602d9041 dfly::Service::DispatchCommand() @ 0x5b2e6059594a dfly::DflyShardReplica::ExecuteTx() @ 0x5b2e60595f42 dfly::DflyShardReplica::StableSyncDflyReadFb() @ 0x5b2e605a2861 _ZN5boost7context6detail11fiber_entryINS1_12fiber_recordINS0_5fiberEN4util3fb219FixedStackAllocatorEZNS6_6detail15WorkerFiberImplIMN4dfly16DflyShardReplicaEFvPNSA_7ContextEEJPSB_RSD_EEC4IS7_EESt17basic_string_viewIcSt11char_traitsIcEERKNS0_12preallocatedEOT_OSF_OSG_SH_EUlOS4_E_EEEEvNS1_10transfer_tE @ 0x5b2e60b7d00f make_fcontext *** SIGABRT received at time=1765002554 on cpu 34 *** PC: @ 0x791dbdc99a1b (unknown) pthread_kill


To Reproduce
Steps to reproduce the behavior:

  1. Insert records using command
  2. Query records using command
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

DF shouldn't crash on assertion failure


Screenshots
If applicable, add screenshots to help explain your problem.


Environment (please complete the following information):

  • OS: "Ubuntu 23.10"

  • Kernel: # Command: uname -a
    Linux mcache-dsci-mlp-orion-mp-v3-6v3-prd-ase1 6.5.0-1018-gcp INFO CPU does not return anything #18-Ubuntu SMP Fri Apr 5 03:10:24 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

  • Containerized?: Bare Metal

  • Dragonfly Version: dragonfly v1.26.3-e713b8c69298f704e4424141b27c283e45b1468b


Reproducible Code Snippet

# Minimal code snippet to reproduce this bug

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions