Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 469.530665ms trace: https://jaeger.dev.storj.io/trace/70994c85f165c3a0 FAIL (2025-11-16 18:20:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ce... Nov 16 18:11:56.176141393 (128h27m41.53480005s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176505694 (128h27m41.535164346s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176694869 (128h27m41.535353535s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176906560 (128h27m41.535565213s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178010924 (128h27m41.536669583s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178162029 (128h27m41.53682069s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178624747 (128h27m41.537283407s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.179822806 (128h27m41.538481511s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.181092396 (128h27m41.539751055s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.182267056 (128h27m41.540925728s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.298224599 (128h27m41.656883262s) [upload=113070]: Next returned nil segment Nov 16 18:11:56.298240977 (128h27m41.656899643s) [upload=113070]: Waiting for error group managing segments... Nov 16 18:11:56.322992573 (128h27m41.681651228s) [noise=true , address=81.171.14.70:4031 , piece_id=WMGGCRBCTQTVREBKXGD37YVKTKSK2P5SFJV5LH2MGFI3RJXONTKQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.326677846 (128h27m41.6853365s) [noise=true , address=81.171.14.70:4033 , piece_id=UR3UBNCMPQSDNMI44MKZ34MRQ4MYPCLHGWGWNDEPVNJ3GGFDULDQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.327086439 (128h27m41.685745102s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330596159 (128h27m41.689254822s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330894325 (128h27m41.689552985s) [noise=true , address=185.167.204.205:4016 , piece_id=XTDJJJHRNTXLU4D3HJ4OCP2UQR6RF6KHCU6OWSQAPI2Q6SONPKYA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331200302 (128h27m41.689858955s) [noise=true , address=185.167.204.205:4019 , piece_id=LJCJSOQLF4FSLVI7G753ZYDJW6VJJIQCOESIN4KVUHQ36NPZZOHQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331708487 (128h27m41.69036714s) [noise=true , address=185.167.204.206:4061 , piece_id=VWC7K7VZIUF3B47V3XQKWG4WRWFIZSCI4QOW24JI5RC77KZ2VKNQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331819138 (128h27m41.690477798s) [noise=true , address=185.167.204.205:4002 , piece_id=NL2FGXMOTSJRIS7O4ZBTIE2636T4UBL4TBWO3YMA7LCKNTSDGYSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334312534 (128h27m41.692971206s) [noise=true , address=185.167.204.205:4031 , piece_id=J2H4MRWYBVR3FYDVOB7BHQV3XJ2PBXPBKTGIS5SK6N76DDQI3VJA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334376547 (128h27m41.693035205s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334837583 (128h27m41.693496245s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.335363984 (128h27m41.694022646s) [noise=true , address=185.167.204.205:4005 , piece_id=VFVSPAC4QXC5RDOZWH5HS5GLIXCOSRJN67KX65IXBZD2WGULIAJQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.338489129 (128h27m41.697147785s) [noise=true , address=185.167.204.206:4039 , piece_id=S24WYTQCI7XAYTNW5CO6APLU3RRGU4AEJIANW3I5KT5CUHTP5WFQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.340260145 (128h27m41.698918799s) [noise=true , address=81.171.14.70:4040 , piece_id=4YLPSBJNFFRY4KJQRXR4FYNLV4DFOEO6SHIZGTJMSAUHRB4DTBWA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.342507930 (128h27m41.701166584s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.343055650 (128h27m41.701714293s) [noise=true , address=185.167.204.205:4024 , piece_id=EAG5CMHBVOSIFI2WFCQIXQFNGYU4ATNHUUIBQKYKM6B6XETO63YQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.346735021 (128h27m41.705393678s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.348599199 (128h27m41.707257859s) [noise=true , address=185.167.204.206:4051 , piece_id=3OFN2SAZSTHRRM4SNHQCU2IEVOCXMA3BZOLS2YNULUJ5IHFQQSPA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.349247455 (128h27m41.707906114s) [noise=true , address=89.149.201.147:4023 , piece_id=O533J4NWGNLHIAD4SA7VWCHP4WF7GGB33AG4EPFIUMZIVSMCZLSQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.355006125 (128h27m41.713664782s) [noise=true , address=185.167.204.205:4033 , piece_id=SWXEHUBOBIUDILZLZ2NC5G7MGHFPOWU2ZPODJAJV6NFUAYBUYZXQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.360471308 (128h27m41.719129963s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.374029787 (128h27m41.73268843s) [noise=true , address=89.149.201.147:4009 , piece_id=SEU3XXEIWPXZAES7EPYAZQTCKXCOEE7S6CLFL3SZZA3MTBNSMG2A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.789361201 (128h27m42.14801986s) [noise=true , address=81.171.14.70:4030 , piece_id=EPDZQGCWBKEEGW2EMHFP4FVUH4ZCOWFRRVJHH63TX7DF6BUELUIQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.804954891 (128h27m42.163613548s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.805830432 (128h27m42.164489092s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813392799 (128h27m42.172051464s) [noise=true , address=89.149.201.147:4015 , piece_id=W7BFUW3BP7ZK6RTZMKRDLTDIUSA5Y6HA3YXWDPB3MRD3RXDVYZKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813434571 (128h27m42.172093236s) [seg_pos={0 0}, upload=113070]: Segment reached optimal threshold of 26 pieces. Nov 16 18:11:56.813848675 (128h27m42.172507339s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813864049 (128h27m42.172522714s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813894323 (128h27m42.172552985s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813935878 (128h27m42.172594551s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813968221 (128h27m42.172626877s) [seg_pos={0 0}, upload=113070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 16 18:11:56.814067848 (128h27m42.172726507s) [upload=113070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 18:11:56.814075640 (128h27m42.172734299s) [upload=113070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 18:13:47.213254651 (128h29m32.571913308s) [upload=113070]: Done uploading segments... Nov 16 18:13:47.213331906 (128h29m32.57199057s) [upload=113070]: Upload finished. err: metaclient: context deadline exceeded Duration: 469.555766ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 560.218063ms trace: https://jaeger.dev.storj.io/trace/70994c85f165c3a0 FAIL (2025-11-16 18:20:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ce... Nov 16 18:11:56.176141393 (128h27m41.53480005s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176505694 (128h27m41.535164346s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176694869 (128h27m41.535353535s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176906560 (128h27m41.535565213s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178010924 (128h27m41.536669583s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178162029 (128h27m41.53682069s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178624747 (128h27m41.537283407s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.179822806 (128h27m41.538481511s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.181092396 (128h27m41.539751055s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.182267056 (128h27m41.540925728s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.298224599 (128h27m41.656883262s) [upload=113070]: Next returned nil segment Nov 16 18:11:56.298240977 (128h27m41.656899643s) [upload=113070]: Waiting for error group managing segments... Nov 16 18:11:56.322992573 (128h27m41.681651228s) [noise=true , address=81.171.14.70:4031 , piece_id=WMGGCRBCTQTVREBKXGD37YVKTKSK2P5SFJV5LH2MGFI3RJXONTKQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.326677846 (128h27m41.6853365s) [noise=true , address=81.171.14.70:4033 , piece_id=UR3UBNCMPQSDNMI44MKZ34MRQ4MYPCLHGWGWNDEPVNJ3GGFDULDQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.327086439 (128h27m41.685745102s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330596159 (128h27m41.689254822s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330894325 (128h27m41.689552985s) [noise=true , address=185.167.204.205:4016 , piece_id=XTDJJJHRNTXLU4D3HJ4OCP2UQR6RF6KHCU6OWSQAPI2Q6SONPKYA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331200302 (128h27m41.689858955s) [noise=true , address=185.167.204.205:4019 , piece_id=LJCJSOQLF4FSLVI7G753ZYDJW6VJJIQCOESIN4KVUHQ36NPZZOHQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331708487 (128h27m41.69036714s) [noise=true , address=185.167.204.206:4061 , piece_id=VWC7K7VZIUF3B47V3XQKWG4WRWFIZSCI4QOW24JI5RC77KZ2VKNQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331819138 (128h27m41.690477798s) [noise=true , address=185.167.204.205:4002 , piece_id=NL2FGXMOTSJRIS7O4ZBTIE2636T4UBL4TBWO3YMA7LCKNTSDGYSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334312534 (128h27m41.692971206s) [noise=true , address=185.167.204.205:4031 , piece_id=J2H4MRWYBVR3FYDVOB7BHQV3XJ2PBXPBKTGIS5SK6N76DDQI3VJA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334376547 (128h27m41.693035205s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334837583 (128h27m41.693496245s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.335363984 (128h27m41.694022646s) [noise=true , address=185.167.204.205:4005 , piece_id=VFVSPAC4QXC5RDOZWH5HS5GLIXCOSRJN67KX65IXBZD2WGULIAJQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.338489129 (128h27m41.697147785s) [noise=true , address=185.167.204.206:4039 , piece_id=S24WYTQCI7XAYTNW5CO6APLU3RRGU4AEJIANW3I5KT5CUHTP5WFQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.340260145 (128h27m41.698918799s) [noise=true , address=81.171.14.70:4040 , piece_id=4YLPSBJNFFRY4KJQRXR4FYNLV4DFOEO6SHIZGTJMSAUHRB4DTBWA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.342507930 (128h27m41.701166584s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.343055650 (128h27m41.701714293s) [noise=true , address=185.167.204.205:4024 , piece_id=EAG5CMHBVOSIFI2WFCQIXQFNGYU4ATNHUUIBQKYKM6B6XETO63YQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.346735021 (128h27m41.705393678s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.348599199 (128h27m41.707257859s) [noise=true , address=185.167.204.206:4051 , piece_id=3OFN2SAZSTHRRM4SNHQCU2IEVOCXMA3BZOLS2YNULUJ5IHFQQSPA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.349247455 (128h27m41.707906114s) [noise=true , address=89.149.201.147:4023 , piece_id=O533J4NWGNLHIAD4SA7VWCHP4WF7GGB33AG4EPFIUMZIVSMCZLSQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.355006125 (128h27m41.713664782s) [noise=true , address=185.167.204.205:4033 , piece_id=SWXEHUBOBIUDILZLZ2NC5G7MGHFPOWU2ZPODJAJV6NFUAYBUYZXQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.360471308 (128h27m41.719129963s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.374029787 (128h27m41.73268843s) [noise=true , address=89.149.201.147:4009 , piece_id=SEU3XXEIWPXZAES7EPYAZQTCKXCOEE7S6CLFL3SZZA3MTBNSMG2A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.789361201 (128h27m42.14801986s) [noise=true , address=81.171.14.70:4030 , piece_id=EPDZQGCWBKEEGW2EMHFP4FVUH4ZCOWFRRVJHH63TX7DF6BUELUIQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.804954891 (128h27m42.163613548s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.805830432 (128h27m42.164489092s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813392799 (128h27m42.172051464s) [noise=true , address=89.149.201.147:4015 , piece_id=W7BFUW3BP7ZK6RTZMKRDLTDIUSA5Y6HA3YXWDPB3MRD3RXDVYZKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813434571 (128h27m42.172093236s) [seg_pos={0 0}, upload=113070]: Segment reached optimal threshold of 26 pieces. Nov 16 18:11:56.813848675 (128h27m42.172507339s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813864049 (128h27m42.172522714s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813894323 (128h27m42.172552985s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813935878 (128h27m42.172594551s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813968221 (128h27m42.172626877s) [seg_pos={0 0}, upload=113070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 16 18:11:56.814067848 (128h27m42.172726507s) [upload=113070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 18:11:56.814075640 (128h27m42.172734299s) [upload=113070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 18:13:47.213254651 (128h29m32.571913308s) [upload=113070]: Done uploading segments... Nov 16 18:13:47.213331906 (128h29m32.57199057s) [upload=113070]: Upload finished. err: metaclient: context deadline exceeded Duration: 560.247032ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 884.498717ms trace: https://jaeger.dev.storj.io/trace/70994c85f165c3a0 FAIL (2025-11-16 18:19:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ce... Nov 16 18:11:56.176141393 (128h27m41.53480005s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176505694 (128h27m41.535164346s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176694869 (128h27m41.535353535s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176906560 (128h27m41.535565213s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178010924 (128h27m41.536669583s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178162029 (128h27m41.53682069s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178624747 (128h27m41.537283407s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.179822806 (128h27m41.538481511s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.181092396 (128h27m41.539751055s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.182267056 (128h27m41.540925728s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.298224599 (128h27m41.656883262s) [upload=113070]: Next returned nil segment Nov 16 18:11:56.298240977 (128h27m41.656899643s) [upload=113070]: Waiting for error group managing segments... Nov 16 18:11:56.322992573 (128h27m41.681651228s) [noise=true , address=81.171.14.70:4031 , piece_id=WMGGCRBCTQTVREBKXGD37YVKTKSK2P5SFJV5LH2MGFI3RJXONTKQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.326677846 (128h27m41.6853365s) [noise=true , address=81.171.14.70:4033 , piece_id=UR3UBNCMPQSDNMI44MKZ34MRQ4MYPCLHGWGWNDEPVNJ3GGFDULDQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.327086439 (128h27m41.685745102s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330596159 (128h27m41.689254822s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330894325 (128h27m41.689552985s) [noise=true , address=185.167.204.205:4016 , piece_id=XTDJJJHRNTXLU4D3HJ4OCP2UQR6RF6KHCU6OWSQAPI2Q6SONPKYA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331200302 (128h27m41.689858955s) [noise=true , address=185.167.204.205:4019 , piece_id=LJCJSOQLF4FSLVI7G753ZYDJW6VJJIQCOESIN4KVUHQ36NPZZOHQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331708487 (128h27m41.69036714s) [noise=true , address=185.167.204.206:4061 , piece_id=VWC7K7VZIUF3B47V3XQKWG4WRWFIZSCI4QOW24JI5RC77KZ2VKNQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331819138 (128h27m41.690477798s) [noise=true , address=185.167.204.205:4002 , piece_id=NL2FGXMOTSJRIS7O4ZBTIE2636T4UBL4TBWO3YMA7LCKNTSDGYSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334312534 (128h27m41.692971206s) [noise=true , address=185.167.204.205:4031 , piece_id=J2H4MRWYBVR3FYDVOB7BHQV3XJ2PBXPBKTGIS5SK6N76DDQI3VJA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334376547 (128h27m41.693035205s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334837583 (128h27m41.693496245s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.335363984 (128h27m41.694022646s) [noise=true , address=185.167.204.205:4005 , piece_id=VFVSPAC4QXC5RDOZWH5HS5GLIXCOSRJN67KX65IXBZD2WGULIAJQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.338489129 (128h27m41.697147785s) [noise=true , address=185.167.204.206:4039 , piece_id=S24WYTQCI7XAYTNW5CO6APLU3RRGU4AEJIANW3I5KT5CUHTP5WFQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.340260145 (128h27m41.698918799s) [noise=true , address=81.171.14.70:4040 , piece_id=4YLPSBJNFFRY4KJQRXR4FYNLV4DFOEO6SHIZGTJMSAUHRB4DTBWA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.342507930 (128h27m41.701166584s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.343055650 (128h27m41.701714293s) [noise=true , address=185.167.204.205:4024 , piece_id=EAG5CMHBVOSIFI2WFCQIXQFNGYU4ATNHUUIBQKYKM6B6XETO63YQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.346735021 (128h27m41.705393678s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.348599199 (128h27m41.707257859s) [noise=true , address=185.167.204.206:4051 , piece_id=3OFN2SAZSTHRRM4SNHQCU2IEVOCXMA3BZOLS2YNULUJ5IHFQQSPA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.349247455 (128h27m41.707906114s) [noise=true , address=89.149.201.147:4023 , piece_id=O533J4NWGNLHIAD4SA7VWCHP4WF7GGB33AG4EPFIUMZIVSMCZLSQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.355006125 (128h27m41.713664782s) [noise=true , address=185.167.204.205:4033 , piece_id=SWXEHUBOBIUDILZLZ2NC5G7MGHFPOWU2ZPODJAJV6NFUAYBUYZXQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.360471308 (128h27m41.719129963s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.374029787 (128h27m41.73268843s) [noise=true , address=89.149.201.147:4009 , piece_id=SEU3XXEIWPXZAES7EPYAZQTCKXCOEE7S6CLFL3SZZA3MTBNSMG2A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.789361201 (128h27m42.14801986s) [noise=true , address=81.171.14.70:4030 , piece_id=EPDZQGCWBKEEGW2EMHFP4FVUH4ZCOWFRRVJHH63TX7DF6BUELUIQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.804954891 (128h27m42.163613548s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.805830432 (128h27m42.164489092s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813392799 (128h27m42.172051464s) [noise=true , address=89.149.201.147:4015 , piece_id=W7BFUW3BP7ZK6RTZMKRDLTDIUSA5Y6HA3YXWDPB3MRD3RXDVYZKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813434571 (128h27m42.172093236s) [seg_pos={0 0}, upload=113070]: Segment reached optimal threshold of 26 pieces. Nov 16 18:11:56.813848675 (128h27m42.172507339s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813864049 (128h27m42.172522714s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813894323 (128h27m42.172552985s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813935878 (128h27m42.172594551s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813968221 (128h27m42.172626877s) [seg_pos={0 0}, upload=113070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 16 18:11:56.814067848 (128h27m42.172726507s) [upload=113070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 18:11:56.814075640 (128h27m42.172734299s) [upload=113070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 18:13:47.213254651 (128h29m32.571913308s) [upload=113070]: Done uploading segments... Nov 16 18:13:47.213331906 (128h29m32.57199057s) [upload=113070]: Upload finished. err: metaclient: context deadline exceeded Duration: 884.528058ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 434.768063ms trace: https://jaeger.dev.storj.io/trace/70994c85f165c3a0 FAIL (2025-11-16 18:19:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ce... Nov 16 18:11:56.176141393 (128h27m41.53480005s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176505694 (128h27m41.535164346s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176694869 (128h27m41.535353535s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176906560 (128h27m41.535565213s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178010924 (128h27m41.536669583s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178162029 (128h27m41.53682069s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178624747 (128h27m41.537283407s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.179822806 (128h27m41.538481511s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.181092396 (128h27m41.539751055s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.182267056 (128h27m41.540925728s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.298224599 (128h27m41.656883262s) [upload=113070]: Next returned nil segment Nov 16 18:11:56.298240977 (128h27m41.656899643s) [upload=113070]: Waiting for error group managing segments... Nov 16 18:11:56.322992573 (128h27m41.681651228s) [noise=true , address=81.171.14.70:4031 , piece_id=WMGGCRBCTQTVREBKXGD37YVKTKSK2P5SFJV5LH2MGFI3RJXONTKQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.326677846 (128h27m41.6853365s) [noise=true , address=81.171.14.70:4033 , piece_id=UR3UBNCMPQSDNMI44MKZ34MRQ4MYPCLHGWGWNDEPVNJ3GGFDULDQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.327086439 (128h27m41.685745102s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330596159 (128h27m41.689254822s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330894325 (128h27m41.689552985s) [noise=true , address=185.167.204.205:4016 , piece_id=XTDJJJHRNTXLU4D3HJ4OCP2UQR6RF6KHCU6OWSQAPI2Q6SONPKYA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331200302 (128h27m41.689858955s) [noise=true , address=185.167.204.205:4019 , piece_id=LJCJSOQLF4FSLVI7G753ZYDJW6VJJIQCOESIN4KVUHQ36NPZZOHQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331708487 (128h27m41.69036714s) [noise=true , address=185.167.204.206:4061 , piece_id=VWC7K7VZIUF3B47V3XQKWG4WRWFIZSCI4QOW24JI5RC77KZ2VKNQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331819138 (128h27m41.690477798s) [noise=true , address=185.167.204.205:4002 , piece_id=NL2FGXMOTSJRIS7O4ZBTIE2636T4UBL4TBWO3YMA7LCKNTSDGYSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334312534 (128h27m41.692971206s) [noise=true , address=185.167.204.205:4031 , piece_id=J2H4MRWYBVR3FYDVOB7BHQV3XJ2PBXPBKTGIS5SK6N76DDQI3VJA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334376547 (128h27m41.693035205s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334837583 (128h27m41.693496245s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.335363984 (128h27m41.694022646s) [noise=true , address=185.167.204.205:4005 , piece_id=VFVSPAC4QXC5RDOZWH5HS5GLIXCOSRJN67KX65IXBZD2WGULIAJQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.338489129 (128h27m41.697147785s) [noise=true , address=185.167.204.206:4039 , piece_id=S24WYTQCI7XAYTNW5CO6APLU3RRGU4AEJIANW3I5KT5CUHTP5WFQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.340260145 (128h27m41.698918799s) [noise=true , address=81.171.14.70:4040 , piece_id=4YLPSBJNFFRY4KJQRXR4FYNLV4DFOEO6SHIZGTJMSAUHRB4DTBWA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.342507930 (128h27m41.701166584s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.343055650 (128h27m41.701714293s) [noise=true , address=185.167.204.205:4024 , piece_id=EAG5CMHBVOSIFI2WFCQIXQFNGYU4ATNHUUIBQKYKM6B6XETO63YQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.346735021 (128h27m41.705393678s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.348599199 (128h27m41.707257859s) [noise=true , address=185.167.204.206:4051 , piece_id=3OFN2SAZSTHRRM4SNHQCU2IEVOCXMA3BZOLS2YNULUJ5IHFQQSPA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.349247455 (128h27m41.707906114s) [noise=true , address=89.149.201.147:4023 , piece_id=O533J4NWGNLHIAD4SA7VWCHP4WF7GGB33AG4EPFIUMZIVSMCZLSQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.355006125 (128h27m41.713664782s) [noise=true , address=185.167.204.205:4033 , piece_id=SWXEHUBOBIUDILZLZ2NC5G7MGHFPOWU2ZPODJAJV6NFUAYBUYZXQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.360471308 (128h27m41.719129963s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.374029787 (128h27m41.73268843s) [noise=true , address=89.149.201.147:4009 , piece_id=SEU3XXEIWPXZAES7EPYAZQTCKXCOEE7S6CLFL3SZZA3MTBNSMG2A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.789361201 (128h27m42.14801986s) [noise=true , address=81.171.14.70:4030 , piece_id=EPDZQGCWBKEEGW2EMHFP4FVUH4ZCOWFRRVJHH63TX7DF6BUELUIQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.804954891 (128h27m42.163613548s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.805830432 (128h27m42.164489092s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813392799 (128h27m42.172051464s) [noise=true , address=89.149.201.147:4015 , piece_id=W7BFUW3BP7ZK6RTZMKRDLTDIUSA5Y6HA3YXWDPB3MRD3RXDVYZKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813434571 (128h27m42.172093236s) [seg_pos={0 0}, upload=113070]: Segment reached optimal threshold of 26 pieces. Nov 16 18:11:56.813848675 (128h27m42.172507339s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813864049 (128h27m42.172522714s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813894323 (128h27m42.172552985s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813935878 (128h27m42.172594551s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813968221 (128h27m42.172626877s) [seg_pos={0 0}, upload=113070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 16 18:11:56.814067848 (128h27m42.172726507s) [upload=113070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 18:11:56.814075640 (128h27m42.172734299s) [upload=113070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 18:13:47.213254651 (128h29m32.571913308s) [upload=113070]: Done uploading segments... Nov 16 18:13:47.213331906 (128h29m32.57199057s) [upload=113070]: Upload finished. err: metaclient: context deadline exceeded Duration: 434.811891ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 857.599258ms trace: https://jaeger.dev.storj.io/trace/70994c85f165c3a0 FAIL (2025-11-16 18:19:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ce... Nov 16 18:11:56.176141393 (128h27m41.53480005s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176505694 (128h27m41.535164346s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176694869 (128h27m41.535353535s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176906560 (128h27m41.535565213s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178010924 (128h27m41.536669583s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178162029 (128h27m41.53682069s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178624747 (128h27m41.537283407s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.179822806 (128h27m41.538481511s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.181092396 (128h27m41.539751055s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.182267056 (128h27m41.540925728s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.298224599 (128h27m41.656883262s) [upload=113070]: Next returned nil segment Nov 16 18:11:56.298240977 (128h27m41.656899643s) [upload=113070]: Waiting for error group managing segments... Nov 16 18:11:56.322992573 (128h27m41.681651228s) [noise=true , address=81.171.14.70:4031 , piece_id=WMGGCRBCTQTVREBKXGD37YVKTKSK2P5SFJV5LH2MGFI3RJXONTKQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.326677846 (128h27m41.6853365s) [noise=true , address=81.171.14.70:4033 , piece_id=UR3UBNCMPQSDNMI44MKZ34MRQ4MYPCLHGWGWNDEPVNJ3GGFDULDQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.327086439 (128h27m41.685745102s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330596159 (128h27m41.689254822s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330894325 (128h27m41.689552985s) [noise=true , address=185.167.204.205:4016 , piece_id=XTDJJJHRNTXLU4D3HJ4OCP2UQR6RF6KHCU6OWSQAPI2Q6SONPKYA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331200302 (128h27m41.689858955s) [noise=true , address=185.167.204.205:4019 , piece_id=LJCJSOQLF4FSLVI7G753ZYDJW6VJJIQCOESIN4KVUHQ36NPZZOHQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331708487 (128h27m41.69036714s) [noise=true , address=185.167.204.206:4061 , piece_id=VWC7K7VZIUF3B47V3XQKWG4WRWFIZSCI4QOW24JI5RC77KZ2VKNQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331819138 (128h27m41.690477798s) [noise=true , address=185.167.204.205:4002 , piece_id=NL2FGXMOTSJRIS7O4ZBTIE2636T4UBL4TBWO3YMA7LCKNTSDGYSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334312534 (128h27m41.692971206s) [noise=true , address=185.167.204.205:4031 , piece_id=J2H4MRWYBVR3FYDVOB7BHQV3XJ2PBXPBKTGIS5SK6N76DDQI3VJA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334376547 (128h27m41.693035205s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334837583 (128h27m41.693496245s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.335363984 (128h27m41.694022646s) [noise=true , address=185.167.204.205:4005 , piece_id=VFVSPAC4QXC5RDOZWH5HS5GLIXCOSRJN67KX65IXBZD2WGULIAJQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.338489129 (128h27m41.697147785s) [noise=true , address=185.167.204.206:4039 , piece_id=S24WYTQCI7XAYTNW5CO6APLU3RRGU4AEJIANW3I5KT5CUHTP5WFQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.340260145 (128h27m41.698918799s) [noise=true , address=81.171.14.70:4040 , piece_id=4YLPSBJNFFRY4KJQRXR4FYNLV4DFOEO6SHIZGTJMSAUHRB4DTBWA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.342507930 (128h27m41.701166584s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.343055650 (128h27m41.701714293s) [noise=true , address=185.167.204.205:4024 , piece_id=EAG5CMHBVOSIFI2WFCQIXQFNGYU4ATNHUUIBQKYKM6B6XETO63YQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.346735021 (128h27m41.705393678s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.348599199 (128h27m41.707257859s) [noise=true , address=185.167.204.206:4051 , piece_id=3OFN2SAZSTHRRM4SNHQCU2IEVOCXMA3BZOLS2YNULUJ5IHFQQSPA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.349247455 (128h27m41.707906114s) [noise=true , address=89.149.201.147:4023 , piece_id=O533J4NWGNLHIAD4SA7VWCHP4WF7GGB33AG4EPFIUMZIVSMCZLSQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.355006125 (128h27m41.713664782s) [noise=true , address=185.167.204.205:4033 , piece_id=SWXEHUBOBIUDILZLZ2NC5G7MGHFPOWU2ZPODJAJV6NFUAYBUYZXQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.360471308 (128h27m41.719129963s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.374029787 (128h27m41.73268843s) [noise=true , address=89.149.201.147:4009 , piece_id=SEU3XXEIWPXZAES7EPYAZQTCKXCOEE7S6CLFL3SZZA3MTBNSMG2A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.789361201 (128h27m42.14801986s) [noise=true , address=81.171.14.70:4030 , piece_id=EPDZQGCWBKEEGW2EMHFP4FVUH4ZCOWFRRVJHH63TX7DF6BUELUIQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.804954891 (128h27m42.163613548s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.805830432 (128h27m42.164489092s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813392799 (128h27m42.172051464s) [noise=true , address=89.149.201.147:4015 , piece_id=W7BFUW3BP7ZK6RTZMKRDLTDIUSA5Y6HA3YXWDPB3MRD3RXDVYZKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813434571 (128h27m42.172093236s) [seg_pos={0 0}, upload=113070]: Segment reached optimal threshold of 26 pieces. Nov 16 18:11:56.813848675 (128h27m42.172507339s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813864049 (128h27m42.172522714s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813894323 (128h27m42.172552985s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813935878 (128h27m42.172594551s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813968221 (128h27m42.172626877s) [seg_pos={0 0}, upload=113070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 16 18:11:56.814067848 (128h27m42.172726507s) [upload=113070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 18:11:56.814075640 (128h27m42.172734299s) [upload=113070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 18:13:47.213254651 (128h29m32.571913308s) [upload=113070]: Done uploading segments... Nov 16 18:13:47.213331906 (128h29m32.57199057s) [upload=113070]: Upload finished. err: metaclient: context deadline exceeded Duration: 857.626546ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 861.886286ms trace: https://jaeger.dev.storj.io/trace/70994c85f165c3a0 FAIL (2025-11-16 18:18:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ce... Nov 16 18:11:56.176141393 (128h27m41.53480005s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176505694 (128h27m41.535164346s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176694869 (128h27m41.535353535s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176906560 (128h27m41.535565213s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178010924 (128h27m41.536669583s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178162029 (128h27m41.53682069s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178624747 (128h27m41.537283407s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.179822806 (128h27m41.538481511s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.181092396 (128h27m41.539751055s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.182267056 (128h27m41.540925728s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.298224599 (128h27m41.656883262s) [upload=113070]: Next returned nil segment Nov 16 18:11:56.298240977 (128h27m41.656899643s) [upload=113070]: Waiting for error group managing segments... Nov 16 18:11:56.322992573 (128h27m41.681651228s) [noise=true , address=81.171.14.70:4031 , piece_id=WMGGCRBCTQTVREBKXGD37YVKTKSK2P5SFJV5LH2MGFI3RJXONTKQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.326677846 (128h27m41.6853365s) [noise=true , address=81.171.14.70:4033 , piece_id=UR3UBNCMPQSDNMI44MKZ34MRQ4MYPCLHGWGWNDEPVNJ3GGFDULDQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.327086439 (128h27m41.685745102s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330596159 (128h27m41.689254822s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330894325 (128h27m41.689552985s) [noise=true , address=185.167.204.205:4016 , piece_id=XTDJJJHRNTXLU4D3HJ4OCP2UQR6RF6KHCU6OWSQAPI2Q6SONPKYA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331200302 (128h27m41.689858955s) [noise=true , address=185.167.204.205:4019 , piece_id=LJCJSOQLF4FSLVI7G753ZYDJW6VJJIQCOESIN4KVUHQ36NPZZOHQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331708487 (128h27m41.69036714s) [noise=true , address=185.167.204.206:4061 , piece_id=VWC7K7VZIUF3B47V3XQKWG4WRWFIZSCI4QOW24JI5RC77KZ2VKNQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331819138 (128h27m41.690477798s) [noise=true , address=185.167.204.205:4002 , piece_id=NL2FGXMOTSJRIS7O4ZBTIE2636T4UBL4TBWO3YMA7LCKNTSDGYSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334312534 (128h27m41.692971206s) [noise=true , address=185.167.204.205:4031 , piece_id=J2H4MRWYBVR3FYDVOB7BHQV3XJ2PBXPBKTGIS5SK6N76DDQI3VJA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334376547 (128h27m41.693035205s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334837583 (128h27m41.693496245s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.335363984 (128h27m41.694022646s) [noise=true , address=185.167.204.205:4005 , piece_id=VFVSPAC4QXC5RDOZWH5HS5GLIXCOSRJN67KX65IXBZD2WGULIAJQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.338489129 (128h27m41.697147785s) [noise=true , address=185.167.204.206:4039 , piece_id=S24WYTQCI7XAYTNW5CO6APLU3RRGU4AEJIANW3I5KT5CUHTP5WFQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.340260145 (128h27m41.698918799s) [noise=true , address=81.171.14.70:4040 , piece_id=4YLPSBJNFFRY4KJQRXR4FYNLV4DFOEO6SHIZGTJMSAUHRB4DTBWA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.342507930 (128h27m41.701166584s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.343055650 (128h27m41.701714293s) [noise=true , address=185.167.204.205:4024 , piece_id=EAG5CMHBVOSIFI2WFCQIXQFNGYU4ATNHUUIBQKYKM6B6XETO63YQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.346735021 (128h27m41.705393678s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.348599199 (128h27m41.707257859s) [noise=true , address=185.167.204.206:4051 , piece_id=3OFN2SAZSTHRRM4SNHQCU2IEVOCXMA3BZOLS2YNULUJ5IHFQQSPA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.349247455 (128h27m41.707906114s) [noise=true , address=89.149.201.147:4023 , piece_id=O533J4NWGNLHIAD4SA7VWCHP4WF7GGB33AG4EPFIUMZIVSMCZLSQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.355006125 (128h27m41.713664782s) [noise=true , address=185.167.204.205:4033 , piece_id=SWXEHUBOBIUDILZLZ2NC5G7MGHFPOWU2ZPODJAJV6NFUAYBUYZXQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.360471308 (128h27m41.719129963s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.374029787 (128h27m41.73268843s) [noise=true , address=89.149.201.147:4009 , piece_id=SEU3XXEIWPXZAES7EPYAZQTCKXCOEE7S6CLFL3SZZA3MTBNSMG2A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.789361201 (128h27m42.14801986s) [noise=true , address=81.171.14.70:4030 , piece_id=EPDZQGCWBKEEGW2EMHFP4FVUH4ZCOWFRRVJHH63TX7DF6BUELUIQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.804954891 (128h27m42.163613548s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.805830432 (128h27m42.164489092s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813392799 (128h27m42.172051464s) [noise=true , address=89.149.201.147:4015 , piece_id=W7BFUW3BP7ZK6RTZMKRDLTDIUSA5Y6HA3YXWDPB3MRD3RXDVYZKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813434571 (128h27m42.172093236s) [seg_pos={0 0}, upload=113070]: Segment reached optimal threshold of 26 pieces. Nov 16 18:11:56.813848675 (128h27m42.172507339s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813864049 (128h27m42.172522714s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813894323 (128h27m42.172552985s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813935878 (128h27m42.172594551s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813968221 (128h27m42.172626877s) [seg_pos={0 0}, upload=113070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 16 18:11:56.814067848 (128h27m42.172726507s) [upload=113070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 18:11:56.814075640 (128h27m42.172734299s) [upload=113070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 18:13:47.213254651 (128h29m32.571913308s) [upload=113070]: Done uploading segments... Nov 16 18:13:47.213331906 (128h29m32.57199057s) [upload=113070]: Upload finished. err: metaclient: context deadline exceeded Duration: 861.921911ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 941.877812ms trace: https://jaeger.dev.storj.io/trace/70994c85f165c3a0 FAIL (2025-11-16 18:18:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ce... Nov 16 18:11:56.176141393 (128h27m41.53480005s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176505694 (128h27m41.535164346s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176694869 (128h27m41.535353535s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176906560 (128h27m41.535565213s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178010924 (128h27m41.536669583s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178162029 (128h27m41.53682069s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178624747 (128h27m41.537283407s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.179822806 (128h27m41.538481511s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.181092396 (128h27m41.539751055s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.182267056 (128h27m41.540925728s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.298224599 (128h27m41.656883262s) [upload=113070]: Next returned nil segment Nov 16 18:11:56.298240977 (128h27m41.656899643s) [upload=113070]: Waiting for error group managing segments... Nov 16 18:11:56.322992573 (128h27m41.681651228s) [noise=true , address=81.171.14.70:4031 , piece_id=WMGGCRBCTQTVREBKXGD37YVKTKSK2P5SFJV5LH2MGFI3RJXONTKQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.326677846 (128h27m41.6853365s) [noise=true , address=81.171.14.70:4033 , piece_id=UR3UBNCMPQSDNMI44MKZ34MRQ4MYPCLHGWGWNDEPVNJ3GGFDULDQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.327086439 (128h27m41.685745102s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330596159 (128h27m41.689254822s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330894325 (128h27m41.689552985s) [noise=true , address=185.167.204.205:4016 , piece_id=XTDJJJHRNTXLU4D3HJ4OCP2UQR6RF6KHCU6OWSQAPI2Q6SONPKYA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331200302 (128h27m41.689858955s) [noise=true , address=185.167.204.205:4019 , piece_id=LJCJSOQLF4FSLVI7G753ZYDJW6VJJIQCOESIN4KVUHQ36NPZZOHQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331708487 (128h27m41.69036714s) [noise=true , address=185.167.204.206:4061 , piece_id=VWC7K7VZIUF3B47V3XQKWG4WRWFIZSCI4QOW24JI5RC77KZ2VKNQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331819138 (128h27m41.690477798s) [noise=true , address=185.167.204.205:4002 , piece_id=NL2FGXMOTSJRIS7O4ZBTIE2636T4UBL4TBWO3YMA7LCKNTSDGYSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334312534 (128h27m41.692971206s) [noise=true , address=185.167.204.205:4031 , piece_id=J2H4MRWYBVR3FYDVOB7BHQV3XJ2PBXPBKTGIS5SK6N76DDQI3VJA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334376547 (128h27m41.693035205s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334837583 (128h27m41.693496245s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.335363984 (128h27m41.694022646s) [noise=true , address=185.167.204.205:4005 , piece_id=VFVSPAC4QXC5RDOZWH5HS5GLIXCOSRJN67KX65IXBZD2WGULIAJQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.338489129 (128h27m41.697147785s) [noise=true , address=185.167.204.206:4039 , piece_id=S24WYTQCI7XAYTNW5CO6APLU3RRGU4AEJIANW3I5KT5CUHTP5WFQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.340260145 (128h27m41.698918799s) [noise=true , address=81.171.14.70:4040 , piece_id=4YLPSBJNFFRY4KJQRXR4FYNLV4DFOEO6SHIZGTJMSAUHRB4DTBWA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.342507930 (128h27m41.701166584s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.343055650 (128h27m41.701714293s) [noise=true , address=185.167.204.205:4024 , piece_id=EAG5CMHBVOSIFI2WFCQIXQFNGYU4ATNHUUIBQKYKM6B6XETO63YQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.346735021 (128h27m41.705393678s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.348599199 (128h27m41.707257859s) [noise=true , address=185.167.204.206:4051 , piece_id=3OFN2SAZSTHRRM4SNHQCU2IEVOCXMA3BZOLS2YNULUJ5IHFQQSPA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.349247455 (128h27m41.707906114s) [noise=true , address=89.149.201.147:4023 , piece_id=O533J4NWGNLHIAD4SA7VWCHP4WF7GGB33AG4EPFIUMZIVSMCZLSQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.355006125 (128h27m41.713664782s) [noise=true , address=185.167.204.205:4033 , piece_id=SWXEHUBOBIUDILZLZ2NC5G7MGHFPOWU2ZPODJAJV6NFUAYBUYZXQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.360471308 (128h27m41.719129963s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.374029787 (128h27m41.73268843s) [noise=true , address=89.149.201.147:4009 , piece_id=SEU3XXEIWPXZAES7EPYAZQTCKXCOEE7S6CLFL3SZZA3MTBNSMG2A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.789361201 (128h27m42.14801986s) [noise=true , address=81.171.14.70:4030 , piece_id=EPDZQGCWBKEEGW2EMHFP4FVUH4ZCOWFRRVJHH63TX7DF6BUELUIQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.804954891 (128h27m42.163613548s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.805830432 (128h27m42.164489092s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813392799 (128h27m42.172051464s) [noise=true , address=89.149.201.147:4015 , piece_id=W7BFUW3BP7ZK6RTZMKRDLTDIUSA5Y6HA3YXWDPB3MRD3RXDVYZKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813434571 (128h27m42.172093236s) [seg_pos={0 0}, upload=113070]: Segment reached optimal threshold of 26 pieces. Nov 16 18:11:56.813848675 (128h27m42.172507339s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813864049 (128h27m42.172522714s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813894323 (128h27m42.172552985s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813935878 (128h27m42.172594551s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813968221 (128h27m42.172626877s) [seg_pos={0 0}, upload=113070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 16 18:11:56.814067848 (128h27m42.172726507s) [upload=113070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 18:11:56.814075640 (128h27m42.172734299s) [upload=113070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 18:13:47.213254651 (128h29m32.571913308s) [upload=113070]: Done uploading segments... Nov 16 18:13:47.213331906 (128h29m32.57199057s) [upload=113070]: Upload finished. err: metaclient: context deadline exceeded Duration: 941.907783ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 844.8905ms trace: https://jaeger.dev.storj.io/trace/70994c85f165c3a0 FAIL (2025-11-16 18:17:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ce... Nov 16 18:11:56.176141393 (128h27m41.53480005s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176505694 (128h27m41.535164346s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176694869 (128h27m41.535353535s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176906560 (128h27m41.535565213s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178010924 (128h27m41.536669583s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178162029 (128h27m41.53682069s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178624747 (128h27m41.537283407s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.179822806 (128h27m41.538481511s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.181092396 (128h27m41.539751055s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.182267056 (128h27m41.540925728s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.298224599 (128h27m41.656883262s) [upload=113070]: Next returned nil segment Nov 16 18:11:56.298240977 (128h27m41.656899643s) [upload=113070]: Waiting for error group managing segments... Nov 16 18:11:56.322992573 (128h27m41.681651228s) [noise=true , address=81.171.14.70:4031 , piece_id=WMGGCRBCTQTVREBKXGD37YVKTKSK2P5SFJV5LH2MGFI3RJXONTKQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.326677846 (128h27m41.6853365s) [noise=true , address=81.171.14.70:4033 , piece_id=UR3UBNCMPQSDNMI44MKZ34MRQ4MYPCLHGWGWNDEPVNJ3GGFDULDQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.327086439 (128h27m41.685745102s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330596159 (128h27m41.689254822s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330894325 (128h27m41.689552985s) [noise=true , address=185.167.204.205:4016 , piece_id=XTDJJJHRNTXLU4D3HJ4OCP2UQR6RF6KHCU6OWSQAPI2Q6SONPKYA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331200302 (128h27m41.689858955s) [noise=true , address=185.167.204.205:4019 , piece_id=LJCJSOQLF4FSLVI7G753ZYDJW6VJJIQCOESIN4KVUHQ36NPZZOHQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331708487 (128h27m41.69036714s) [noise=true , address=185.167.204.206:4061 , piece_id=VWC7K7VZIUF3B47V3XQKWG4WRWFIZSCI4QOW24JI5RC77KZ2VKNQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331819138 (128h27m41.690477798s) [noise=true , address=185.167.204.205:4002 , piece_id=NL2FGXMOTSJRIS7O4ZBTIE2636T4UBL4TBWO3YMA7LCKNTSDGYSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334312534 (128h27m41.692971206s) [noise=true , address=185.167.204.205:4031 , piece_id=J2H4MRWYBVR3FYDVOB7BHQV3XJ2PBXPBKTGIS5SK6N76DDQI3VJA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334376547 (128h27m41.693035205s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334837583 (128h27m41.693496245s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.335363984 (128h27m41.694022646s) [noise=true , address=185.167.204.205:4005 , piece_id=VFVSPAC4QXC5RDOZWH5HS5GLIXCOSRJN67KX65IXBZD2WGULIAJQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.338489129 (128h27m41.697147785s) [noise=true , address=185.167.204.206:4039 , piece_id=S24WYTQCI7XAYTNW5CO6APLU3RRGU4AEJIANW3I5KT5CUHTP5WFQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.340260145 (128h27m41.698918799s) [noise=true , address=81.171.14.70:4040 , piece_id=4YLPSBJNFFRY4KJQRXR4FYNLV4DFOEO6SHIZGTJMSAUHRB4DTBWA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.342507930 (128h27m41.701166584s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.343055650 (128h27m41.701714293s) [noise=true , address=185.167.204.205:4024 , piece_id=EAG5CMHBVOSIFI2WFCQIXQFNGYU4ATNHUUIBQKYKM6B6XETO63YQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.346735021 (128h27m41.705393678s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.348599199 (128h27m41.707257859s) [noise=true , address=185.167.204.206:4051 , piece_id=3OFN2SAZSTHRRM4SNHQCU2IEVOCXMA3BZOLS2YNULUJ5IHFQQSPA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.349247455 (128h27m41.707906114s) [noise=true , address=89.149.201.147:4023 , piece_id=O533J4NWGNLHIAD4SA7VWCHP4WF7GGB33AG4EPFIUMZIVSMCZLSQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.355006125 (128h27m41.713664782s) [noise=true , address=185.167.204.205:4033 , piece_id=SWXEHUBOBIUDILZLZ2NC5G7MGHFPOWU2ZPODJAJV6NFUAYBUYZXQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.360471308 (128h27m41.719129963s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.374029787 (128h27m41.73268843s) [noise=true , address=89.149.201.147:4009 , piece_id=SEU3XXEIWPXZAES7EPYAZQTCKXCOEE7S6CLFL3SZZA3MTBNSMG2A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.789361201 (128h27m42.14801986s) [noise=true , address=81.171.14.70:4030 , piece_id=EPDZQGCWBKEEGW2EMHFP4FVUH4ZCOWFRRVJHH63TX7DF6BUELUIQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.804954891 (128h27m42.163613548s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.805830432 (128h27m42.164489092s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813392799 (128h27m42.172051464s) [noise=true , address=89.149.201.147:4015 , piece_id=W7BFUW3BP7ZK6RTZMKRDLTDIUSA5Y6HA3YXWDPB3MRD3RXDVYZKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813434571 (128h27m42.172093236s) [seg_pos={0 0}, upload=113070]: Segment reached optimal threshold of 26 pieces. Nov 16 18:11:56.813848675 (128h27m42.172507339s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813864049 (128h27m42.172522714s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813894323 (128h27m42.172552985s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813935878 (128h27m42.172594551s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813968221 (128h27m42.172626877s) [seg_pos={0 0}, upload=113070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 16 18:11:56.814067848 (128h27m42.172726507s) [upload=113070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 18:11:56.814075640 (128h27m42.172734299s) [upload=113070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 18:13:47.213254651 (128h29m32.571913308s) [upload=113070]: Done uploading segments... Nov 16 18:13:47.213331906 (128h29m32.57199057s) [upload=113070]: Upload finished. err: metaclient: context deadline exceeded Duration: 844.944164ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 473.986316ms trace: https://jaeger.dev.storj.io/trace/70994c85f165c3a0 FAIL (2025-11-16 18:17:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ce... Nov 16 18:11:56.176141393 (128h27m41.53480005s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176505694 (128h27m41.535164346s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176694869 (128h27m41.535353535s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176906560 (128h27m41.535565213s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178010924 (128h27m41.536669583s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178162029 (128h27m41.53682069s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178624747 (128h27m41.537283407s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.179822806 (128h27m41.538481511s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.181092396 (128h27m41.539751055s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.182267056 (128h27m41.540925728s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.298224599 (128h27m41.656883262s) [upload=113070]: Next returned nil segment Nov 16 18:11:56.298240977 (128h27m41.656899643s) [upload=113070]: Waiting for error group managing segments... Nov 16 18:11:56.322992573 (128h27m41.681651228s) [noise=true , address=81.171.14.70:4031 , piece_id=WMGGCRBCTQTVREBKXGD37YVKTKSK2P5SFJV5LH2MGFI3RJXONTKQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.326677846 (128h27m41.6853365s) [noise=true , address=81.171.14.70:4033 , piece_id=UR3UBNCMPQSDNMI44MKZ34MRQ4MYPCLHGWGWNDEPVNJ3GGFDULDQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.327086439 (128h27m41.685745102s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330596159 (128h27m41.689254822s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330894325 (128h27m41.689552985s) [noise=true , address=185.167.204.205:4016 , piece_id=XTDJJJHRNTXLU4D3HJ4OCP2UQR6RF6KHCU6OWSQAPI2Q6SONPKYA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331200302 (128h27m41.689858955s) [noise=true , address=185.167.204.205:4019 , piece_id=LJCJSOQLF4FSLVI7G753ZYDJW6VJJIQCOESIN4KVUHQ36NPZZOHQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331708487 (128h27m41.69036714s) [noise=true , address=185.167.204.206:4061 , piece_id=VWC7K7VZIUF3B47V3XQKWG4WRWFIZSCI4QOW24JI5RC77KZ2VKNQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331819138 (128h27m41.690477798s) [noise=true , address=185.167.204.205:4002 , piece_id=NL2FGXMOTSJRIS7O4ZBTIE2636T4UBL4TBWO3YMA7LCKNTSDGYSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334312534 (128h27m41.692971206s) [noise=true , address=185.167.204.205:4031 , piece_id=J2H4MRWYBVR3FYDVOB7BHQV3XJ2PBXPBKTGIS5SK6N76DDQI3VJA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334376547 (128h27m41.693035205s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334837583 (128h27m41.693496245s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.335363984 (128h27m41.694022646s) [noise=true , address=185.167.204.205:4005 , piece_id=VFVSPAC4QXC5RDOZWH5HS5GLIXCOSRJN67KX65IXBZD2WGULIAJQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.338489129 (128h27m41.697147785s) [noise=true , address=185.167.204.206:4039 , piece_id=S24WYTQCI7XAYTNW5CO6APLU3RRGU4AEJIANW3I5KT5CUHTP5WFQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.340260145 (128h27m41.698918799s) [noise=true , address=81.171.14.70:4040 , piece_id=4YLPSBJNFFRY4KJQRXR4FYNLV4DFOEO6SHIZGTJMSAUHRB4DTBWA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.342507930 (128h27m41.701166584s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.343055650 (128h27m41.701714293s) [noise=true , address=185.167.204.205:4024 , piece_id=EAG5CMHBVOSIFI2WFCQIXQFNGYU4ATNHUUIBQKYKM6B6XETO63YQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.346735021 (128h27m41.705393678s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.348599199 (128h27m41.707257859s) [noise=true , address=185.167.204.206:4051 , piece_id=3OFN2SAZSTHRRM4SNHQCU2IEVOCXMA3BZOLS2YNULUJ5IHFQQSPA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.349247455 (128h27m41.707906114s) [noise=true , address=89.149.201.147:4023 , piece_id=O533J4NWGNLHIAD4SA7VWCHP4WF7GGB33AG4EPFIUMZIVSMCZLSQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.355006125 (128h27m41.713664782s) [noise=true , address=185.167.204.205:4033 , piece_id=SWXEHUBOBIUDILZLZ2NC5G7MGHFPOWU2ZPODJAJV6NFUAYBUYZXQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.360471308 (128h27m41.719129963s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.374029787 (128h27m41.73268843s) [noise=true , address=89.149.201.147:4009 , piece_id=SEU3XXEIWPXZAES7EPYAZQTCKXCOEE7S6CLFL3SZZA3MTBNSMG2A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.789361201 (128h27m42.14801986s) [noise=true , address=81.171.14.70:4030 , piece_id=EPDZQGCWBKEEGW2EMHFP4FVUH4ZCOWFRRVJHH63TX7DF6BUELUIQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.804954891 (128h27m42.163613548s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.805830432 (128h27m42.164489092s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813392799 (128h27m42.172051464s) [noise=true , address=89.149.201.147:4015 , piece_id=W7BFUW3BP7ZK6RTZMKRDLTDIUSA5Y6HA3YXWDPB3MRD3RXDVYZKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813434571 (128h27m42.172093236s) [seg_pos={0 0}, upload=113070]: Segment reached optimal threshold of 26 pieces. Nov 16 18:11:56.813848675 (128h27m42.172507339s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813864049 (128h27m42.172522714s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813894323 (128h27m42.172552985s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813935878 (128h27m42.172594551s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813968221 (128h27m42.172626877s) [seg_pos={0 0}, upload=113070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 16 18:11:56.814067848 (128h27m42.172726507s) [upload=113070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 18:11:56.814075640 (128h27m42.172734299s) [upload=113070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 18:13:47.213254651 (128h29m32.571913308s) [upload=113070]: Done uploading segments... Nov 16 18:13:47.213331906 (128h29m32.57199057s) [upload=113070]: Upload finished. err: metaclient: context deadline exceeded Duration: 474.014615ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 406.955064ms trace: https://jaeger.dev.storj.io/trace/70994c85f165c3a0 FAIL (2025-11-16 18:16:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ce... Nov 16 18:11:56.176141393 (128h27m41.53480005s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176505694 (128h27m41.535164346s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176694869 (128h27m41.535353535s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176906560 (128h27m41.535565213s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178010924 (128h27m41.536669583s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178162029 (128h27m41.53682069s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178624747 (128h27m41.537283407s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.179822806 (128h27m41.538481511s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.181092396 (128h27m41.539751055s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.182267056 (128h27m41.540925728s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.298224599 (128h27m41.656883262s) [upload=113070]: Next returned nil segment Nov 16 18:11:56.298240977 (128h27m41.656899643s) [upload=113070]: Waiting for error group managing segments... Nov 16 18:11:56.322992573 (128h27m41.681651228s) [noise=true , address=81.171.14.70:4031 , piece_id=WMGGCRBCTQTVREBKXGD37YVKTKSK2P5SFJV5LH2MGFI3RJXONTKQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.326677846 (128h27m41.6853365s) [noise=true , address=81.171.14.70:4033 , piece_id=UR3UBNCMPQSDNMI44MKZ34MRQ4MYPCLHGWGWNDEPVNJ3GGFDULDQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.327086439 (128h27m41.685745102s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330596159 (128h27m41.689254822s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330894325 (128h27m41.689552985s) [noise=true , address=185.167.204.205:4016 , piece_id=XTDJJJHRNTXLU4D3HJ4OCP2UQR6RF6KHCU6OWSQAPI2Q6SONPKYA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331200302 (128h27m41.689858955s) [noise=true , address=185.167.204.205:4019 , piece_id=LJCJSOQLF4FSLVI7G753ZYDJW6VJJIQCOESIN4KVUHQ36NPZZOHQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331708487 (128h27m41.69036714s) [noise=true , address=185.167.204.206:4061 , piece_id=VWC7K7VZIUF3B47V3XQKWG4WRWFIZSCI4QOW24JI5RC77KZ2VKNQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331819138 (128h27m41.690477798s) [noise=true , address=185.167.204.205:4002 , piece_id=NL2FGXMOTSJRIS7O4ZBTIE2636T4UBL4TBWO3YMA7LCKNTSDGYSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334312534 (128h27m41.692971206s) [noise=true , address=185.167.204.205:4031 , piece_id=J2H4MRWYBVR3FYDVOB7BHQV3XJ2PBXPBKTGIS5SK6N76DDQI3VJA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334376547 (128h27m41.693035205s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334837583 (128h27m41.693496245s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.335363984 (128h27m41.694022646s) [noise=true , address=185.167.204.205:4005 , piece_id=VFVSPAC4QXC5RDOZWH5HS5GLIXCOSRJN67KX65IXBZD2WGULIAJQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.338489129 (128h27m41.697147785s) [noise=true , address=185.167.204.206:4039 , piece_id=S24WYTQCI7XAYTNW5CO6APLU3RRGU4AEJIANW3I5KT5CUHTP5WFQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.340260145 (128h27m41.698918799s) [noise=true , address=81.171.14.70:4040 , piece_id=4YLPSBJNFFRY4KJQRXR4FYNLV4DFOEO6SHIZGTJMSAUHRB4DTBWA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.342507930 (128h27m41.701166584s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.343055650 (128h27m41.701714293s) [noise=true , address=185.167.204.205:4024 , piece_id=EAG5CMHBVOSIFI2WFCQIXQFNGYU4ATNHUUIBQKYKM6B6XETO63YQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.346735021 (128h27m41.705393678s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.348599199 (128h27m41.707257859s) [noise=true , address=185.167.204.206:4051 , piece_id=3OFN2SAZSTHRRM4SNHQCU2IEVOCXMA3BZOLS2YNULUJ5IHFQQSPA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.349247455 (128h27m41.707906114s) [noise=true , address=89.149.201.147:4023 , piece_id=O533J4NWGNLHIAD4SA7VWCHP4WF7GGB33AG4EPFIUMZIVSMCZLSQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.355006125 (128h27m41.713664782s) [noise=true , address=185.167.204.205:4033 , piece_id=SWXEHUBOBIUDILZLZ2NC5G7MGHFPOWU2ZPODJAJV6NFUAYBUYZXQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.360471308 (128h27m41.719129963s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.374029787 (128h27m41.73268843s) [noise=true , address=89.149.201.147:4009 , piece_id=SEU3XXEIWPXZAES7EPYAZQTCKXCOEE7S6CLFL3SZZA3MTBNSMG2A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.789361201 (128h27m42.14801986s) [noise=true , address=81.171.14.70:4030 , piece_id=EPDZQGCWBKEEGW2EMHFP4FVUH4ZCOWFRRVJHH63TX7DF6BUELUIQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.804954891 (128h27m42.163613548s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.805830432 (128h27m42.164489092s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813392799 (128h27m42.172051464s) [noise=true , address=89.149.201.147:4015 , piece_id=W7BFUW3BP7ZK6RTZMKRDLTDIUSA5Y6HA3YXWDPB3MRD3RXDVYZKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813434571 (128h27m42.172093236s) [seg_pos={0 0}, upload=113070]: Segment reached optimal threshold of 26 pieces. Nov 16 18:11:56.813848675 (128h27m42.172507339s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813864049 (128h27m42.172522714s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813894323 (128h27m42.172552985s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813935878 (128h27m42.172594551s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813968221 (128h27m42.172626877s) [seg_pos={0 0}, upload=113070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 16 18:11:56.814067848 (128h27m42.172726507s) [upload=113070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 18:11:56.814075640 (128h27m42.172734299s) [upload=113070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 18:13:47.213254651 (128h29m32.571913308s) [upload=113070]: Done uploading segments... Nov 16 18:13:47.213331906 (128h29m32.57199057s) [upload=113070]: Upload finished. err: metaclient: context deadline exceeded Duration: 406.980527ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 411.017498ms trace: https://jaeger.dev.storj.io/trace/70994c85f165c3a0 FAIL (2025-11-16 18:15:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ce... Nov 16 18:11:56.176141393 (128h27m41.53480005s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176505694 (128h27m41.535164346s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176694869 (128h27m41.535353535s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176906560 (128h27m41.535565213s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178010924 (128h27m41.536669583s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178162029 (128h27m41.53682069s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178624747 (128h27m41.537283407s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.179822806 (128h27m41.538481511s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.181092396 (128h27m41.539751055s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.182267056 (128h27m41.540925728s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.298224599 (128h27m41.656883262s) [upload=113070]: Next returned nil segment Nov 16 18:11:56.298240977 (128h27m41.656899643s) [upload=113070]: Waiting for error group managing segments... Nov 16 18:11:56.322992573 (128h27m41.681651228s) [noise=true , address=81.171.14.70:4031 , piece_id=WMGGCRBCTQTVREBKXGD37YVKTKSK2P5SFJV5LH2MGFI3RJXONTKQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.326677846 (128h27m41.6853365s) [noise=true , address=81.171.14.70:4033 , piece_id=UR3UBNCMPQSDNMI44MKZ34MRQ4MYPCLHGWGWNDEPVNJ3GGFDULDQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.327086439 (128h27m41.685745102s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330596159 (128h27m41.689254822s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330894325 (128h27m41.689552985s) [noise=true , address=185.167.204.205:4016 , piece_id=XTDJJJHRNTXLU4D3HJ4OCP2UQR6RF6KHCU6OWSQAPI2Q6SONPKYA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331200302 (128h27m41.689858955s) [noise=true , address=185.167.204.205:4019 , piece_id=LJCJSOQLF4FSLVI7G753ZYDJW6VJJIQCOESIN4KVUHQ36NPZZOHQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331708487 (128h27m41.69036714s) [noise=true , address=185.167.204.206:4061 , piece_id=VWC7K7VZIUF3B47V3XQKWG4WRWFIZSCI4QOW24JI5RC77KZ2VKNQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331819138 (128h27m41.690477798s) [noise=true , address=185.167.204.205:4002 , piece_id=NL2FGXMOTSJRIS7O4ZBTIE2636T4UBL4TBWO3YMA7LCKNTSDGYSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334312534 (128h27m41.692971206s) [noise=true , address=185.167.204.205:4031 , piece_id=J2H4MRWYBVR3FYDVOB7BHQV3XJ2PBXPBKTGIS5SK6N76DDQI3VJA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334376547 (128h27m41.693035205s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334837583 (128h27m41.693496245s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.335363984 (128h27m41.694022646s) [noise=true , address=185.167.204.205:4005 , piece_id=VFVSPAC4QXC5RDOZWH5HS5GLIXCOSRJN67KX65IXBZD2WGULIAJQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.338489129 (128h27m41.697147785s) [noise=true , address=185.167.204.206:4039 , piece_id=S24WYTQCI7XAYTNW5CO6APLU3RRGU4AEJIANW3I5KT5CUHTP5WFQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.340260145 (128h27m41.698918799s) [noise=true , address=81.171.14.70:4040 , piece_id=4YLPSBJNFFRY4KJQRXR4FYNLV4DFOEO6SHIZGTJMSAUHRB4DTBWA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.342507930 (128h27m41.701166584s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.343055650 (128h27m41.701714293s) [noise=true , address=185.167.204.205:4024 , piece_id=EAG5CMHBVOSIFI2WFCQIXQFNGYU4ATNHUUIBQKYKM6B6XETO63YQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.346735021 (128h27m41.705393678s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.348599199 (128h27m41.707257859s) [noise=true , address=185.167.204.206:4051 , piece_id=3OFN2SAZSTHRRM4SNHQCU2IEVOCXMA3BZOLS2YNULUJ5IHFQQSPA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.349247455 (128h27m41.707906114s) [noise=true , address=89.149.201.147:4023 , piece_id=O533J4NWGNLHIAD4SA7VWCHP4WF7GGB33AG4EPFIUMZIVSMCZLSQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.355006125 (128h27m41.713664782s) [noise=true , address=185.167.204.205:4033 , piece_id=SWXEHUBOBIUDILZLZ2NC5G7MGHFPOWU2ZPODJAJV6NFUAYBUYZXQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.360471308 (128h27m41.719129963s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.374029787 (128h27m41.73268843s) [noise=true , address=89.149.201.147:4009 , piece_id=SEU3XXEIWPXZAES7EPYAZQTCKXCOEE7S6CLFL3SZZA3MTBNSMG2A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.789361201 (128h27m42.14801986s) [noise=true , address=81.171.14.70:4030 , piece_id=EPDZQGCWBKEEGW2EMHFP4FVUH4ZCOWFRRVJHH63TX7DF6BUELUIQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.804954891 (128h27m42.163613548s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.805830432 (128h27m42.164489092s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813392799 (128h27m42.172051464s) [noise=true , address=89.149.201.147:4015 , piece_id=W7BFUW3BP7ZK6RTZMKRDLTDIUSA5Y6HA3YXWDPB3MRD3RXDVYZKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813434571 (128h27m42.172093236s) [seg_pos={0 0}, upload=113070]: Segment reached optimal threshold of 26 pieces. Nov 16 18:11:56.813848675 (128h27m42.172507339s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813864049 (128h27m42.172522714s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813894323 (128h27m42.172552985s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813935878 (128h27m42.172594551s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813968221 (128h27m42.172626877s) [seg_pos={0 0}, upload=113070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 16 18:11:56.814067848 (128h27m42.172726507s) [upload=113070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 18:11:56.814075640 (128h27m42.172734299s) [upload=113070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 18:13:47.213254651 (128h29m32.571913308s) [upload=113070]: Done uploading segments... Nov 16 18:13:47.213331906 (128h29m32.57199057s) [upload=113070]: Upload finished. err: metaclient: context deadline exceeded Duration: 411.057025ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 987.257189ms trace: https://jaeger.dev.storj.io/trace/70994c85f165c3a0 FAIL (2025-11-16 18:14:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ce... Nov 16 18:11:56.176141393 (128h27m41.53480005s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176505694 (128h27m41.535164346s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176694869 (128h27m41.535353535s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176906560 (128h27m41.535565213s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178010924 (128h27m41.536669583s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178162029 (128h27m41.53682069s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178624747 (128h27m41.537283407s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.179822806 (128h27m41.538481511s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.181092396 (128h27m41.539751055s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.182267056 (128h27m41.540925728s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.298224599 (128h27m41.656883262s) [upload=113070]: Next returned nil segment Nov 16 18:11:56.298240977 (128h27m41.656899643s) [upload=113070]: Waiting for error group managing segments... Nov 16 18:11:56.322992573 (128h27m41.681651228s) [noise=true , address=81.171.14.70:4031 , piece_id=WMGGCRBCTQTVREBKXGD37YVKTKSK2P5SFJV5LH2MGFI3RJXONTKQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.326677846 (128h27m41.6853365s) [noise=true , address=81.171.14.70:4033 , piece_id=UR3UBNCMPQSDNMI44MKZ34MRQ4MYPCLHGWGWNDEPVNJ3GGFDULDQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.327086439 (128h27m41.685745102s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330596159 (128h27m41.689254822s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330894325 (128h27m41.689552985s) [noise=true , address=185.167.204.205:4016 , piece_id=XTDJJJHRNTXLU4D3HJ4OCP2UQR6RF6KHCU6OWSQAPI2Q6SONPKYA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331200302 (128h27m41.689858955s) [noise=true , address=185.167.204.205:4019 , piece_id=LJCJSOQLF4FSLVI7G753ZYDJW6VJJIQCOESIN4KVUHQ36NPZZOHQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331708487 (128h27m41.69036714s) [noise=true , address=185.167.204.206:4061 , piece_id=VWC7K7VZIUF3B47V3XQKWG4WRWFIZSCI4QOW24JI5RC77KZ2VKNQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331819138 (128h27m41.690477798s) [noise=true , address=185.167.204.205:4002 , piece_id=NL2FGXMOTSJRIS7O4ZBTIE2636T4UBL4TBWO3YMA7LCKNTSDGYSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334312534 (128h27m41.692971206s) [noise=true , address=185.167.204.205:4031 , piece_id=J2H4MRWYBVR3FYDVOB7BHQV3XJ2PBXPBKTGIS5SK6N76DDQI3VJA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334376547 (128h27m41.693035205s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334837583 (128h27m41.693496245s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.335363984 (128h27m41.694022646s) [noise=true , address=185.167.204.205:4005 , piece_id=VFVSPAC4QXC5RDOZWH5HS5GLIXCOSRJN67KX65IXBZD2WGULIAJQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.338489129 (128h27m41.697147785s) [noise=true , address=185.167.204.206:4039 , piece_id=S24WYTQCI7XAYTNW5CO6APLU3RRGU4AEJIANW3I5KT5CUHTP5WFQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.340260145 (128h27m41.698918799s) [noise=true , address=81.171.14.70:4040 , piece_id=4YLPSBJNFFRY4KJQRXR4FYNLV4DFOEO6SHIZGTJMSAUHRB4DTBWA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.342507930 (128h27m41.701166584s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.343055650 (128h27m41.701714293s) [noise=true , address=185.167.204.205:4024 , piece_id=EAG5CMHBVOSIFI2WFCQIXQFNGYU4ATNHUUIBQKYKM6B6XETO63YQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.346735021 (128h27m41.705393678s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.348599199 (128h27m41.707257859s) [noise=true , address=185.167.204.206:4051 , piece_id=3OFN2SAZSTHRRM4SNHQCU2IEVOCXMA3BZOLS2YNULUJ5IHFQQSPA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.349247455 (128h27m41.707906114s) [noise=true , address=89.149.201.147:4023 , piece_id=O533J4NWGNLHIAD4SA7VWCHP4WF7GGB33AG4EPFIUMZIVSMCZLSQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.355006125 (128h27m41.713664782s) [noise=true , address=185.167.204.205:4033 , piece_id=SWXEHUBOBIUDILZLZ2NC5G7MGHFPOWU2ZPODJAJV6NFUAYBUYZXQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.360471308 (128h27m41.719129963s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.374029787 (128h27m41.73268843s) [noise=true , address=89.149.201.147:4009 , piece_id=SEU3XXEIWPXZAES7EPYAZQTCKXCOEE7S6CLFL3SZZA3MTBNSMG2A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.789361201 (128h27m42.14801986s) [noise=true , address=81.171.14.70:4030 , piece_id=EPDZQGCWBKEEGW2EMHFP4FVUH4ZCOWFRRVJHH63TX7DF6BUELUIQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.804954891 (128h27m42.163613548s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.805830432 (128h27m42.164489092s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813392799 (128h27m42.172051464s) [noise=true , address=89.149.201.147:4015 , piece_id=W7BFUW3BP7ZK6RTZMKRDLTDIUSA5Y6HA3YXWDPB3MRD3RXDVYZKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813434571 (128h27m42.172093236s) [seg_pos={0 0}, upload=113070]: Segment reached optimal threshold of 26 pieces. Nov 16 18:11:56.813848675 (128h27m42.172507339s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813864049 (128h27m42.172522714s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813894323 (128h27m42.172552985s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813935878 (128h27m42.172594551s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813968221 (128h27m42.172626877s) [seg_pos={0 0}, upload=113070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 16 18:11:56.814067848 (128h27m42.172726507s) [upload=113070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 18:11:56.814075640 (128h27m42.172734299s) [upload=113070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 18:13:47.213254651 (128h29m32.571913308s) [upload=113070]: Done uploading segments... Nov 16 18:13:47.213331906 (128h29m32.57199057s) [upload=113070]: Upload finished. err: metaclient: context deadline exceeded Duration: 987.283553ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 938.849489ms trace: https://jaeger.dev.storj.io/trace/70994c85f165c3a0 FAIL (2025-11-16 18:14:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ce... Nov 16 18:11:56.176141393 (128h27m41.53480005s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176505694 (128h27m41.535164346s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176694869 (128h27m41.535353535s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176906560 (128h27m41.535565213s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178010924 (128h27m41.536669583s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178162029 (128h27m41.53682069s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178624747 (128h27m41.537283407s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.179822806 (128h27m41.538481511s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.181092396 (128h27m41.539751055s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.182267056 (128h27m41.540925728s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.298224599 (128h27m41.656883262s) [upload=113070]: Next returned nil segment Nov 16 18:11:56.298240977 (128h27m41.656899643s) [upload=113070]: Waiting for error group managing segments... Nov 16 18:11:56.322992573 (128h27m41.681651228s) [noise=true , address=81.171.14.70:4031 , piece_id=WMGGCRBCTQTVREBKXGD37YVKTKSK2P5SFJV5LH2MGFI3RJXONTKQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.326677846 (128h27m41.6853365s) [noise=true , address=81.171.14.70:4033 , piece_id=UR3UBNCMPQSDNMI44MKZ34MRQ4MYPCLHGWGWNDEPVNJ3GGFDULDQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.327086439 (128h27m41.685745102s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330596159 (128h27m41.689254822s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330894325 (128h27m41.689552985s) [noise=true , address=185.167.204.205:4016 , piece_id=XTDJJJHRNTXLU4D3HJ4OCP2UQR6RF6KHCU6OWSQAPI2Q6SONPKYA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331200302 (128h27m41.689858955s) [noise=true , address=185.167.204.205:4019 , piece_id=LJCJSOQLF4FSLVI7G753ZYDJW6VJJIQCOESIN4KVUHQ36NPZZOHQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331708487 (128h27m41.69036714s) [noise=true , address=185.167.204.206:4061 , piece_id=VWC7K7VZIUF3B47V3XQKWG4WRWFIZSCI4QOW24JI5RC77KZ2VKNQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331819138 (128h27m41.690477798s) [noise=true , address=185.167.204.205:4002 , piece_id=NL2FGXMOTSJRIS7O4ZBTIE2636T4UBL4TBWO3YMA7LCKNTSDGYSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334312534 (128h27m41.692971206s) [noise=true , address=185.167.204.205:4031 , piece_id=J2H4MRWYBVR3FYDVOB7BHQV3XJ2PBXPBKTGIS5SK6N76DDQI3VJA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334376547 (128h27m41.693035205s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334837583 (128h27m41.693496245s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.335363984 (128h27m41.694022646s) [noise=true , address=185.167.204.205:4005 , piece_id=VFVSPAC4QXC5RDOZWH5HS5GLIXCOSRJN67KX65IXBZD2WGULIAJQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.338489129 (128h27m41.697147785s) [noise=true , address=185.167.204.206:4039 , piece_id=S24WYTQCI7XAYTNW5CO6APLU3RRGU4AEJIANW3I5KT5CUHTP5WFQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.340260145 (128h27m41.698918799s) [noise=true , address=81.171.14.70:4040 , piece_id=4YLPSBJNFFRY4KJQRXR4FYNLV4DFOEO6SHIZGTJMSAUHRB4DTBWA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.342507930 (128h27m41.701166584s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.343055650 (128h27m41.701714293s) [noise=true , address=185.167.204.205:4024 , piece_id=EAG5CMHBVOSIFI2WFCQIXQFNGYU4ATNHUUIBQKYKM6B6XETO63YQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.346735021 (128h27m41.705393678s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.348599199 (128h27m41.707257859s) [noise=true , address=185.167.204.206:4051 , piece_id=3OFN2SAZSTHRRM4SNHQCU2IEVOCXMA3BZOLS2YNULUJ5IHFQQSPA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.349247455 (128h27m41.707906114s) [noise=true , address=89.149.201.147:4023 , piece_id=O533J4NWGNLHIAD4SA7VWCHP4WF7GGB33AG4EPFIUMZIVSMCZLSQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.355006125 (128h27m41.713664782s) [noise=true , address=185.167.204.205:4033 , piece_id=SWXEHUBOBIUDILZLZ2NC5G7MGHFPOWU2ZPODJAJV6NFUAYBUYZXQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.360471308 (128h27m41.719129963s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.374029787 (128h27m41.73268843s) [noise=true , address=89.149.201.147:4009 , piece_id=SEU3XXEIWPXZAES7EPYAZQTCKXCOEE7S6CLFL3SZZA3MTBNSMG2A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.789361201 (128h27m42.14801986s) [noise=true , address=81.171.14.70:4030 , piece_id=EPDZQGCWBKEEGW2EMHFP4FVUH4ZCOWFRRVJHH63TX7DF6BUELUIQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.804954891 (128h27m42.163613548s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.805830432 (128h27m42.164489092s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813392799 (128h27m42.172051464s) [noise=true , address=89.149.201.147:4015 , piece_id=W7BFUW3BP7ZK6RTZMKRDLTDIUSA5Y6HA3YXWDPB3MRD3RXDVYZKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813434571 (128h27m42.172093236s) [seg_pos={0 0}, upload=113070]: Segment reached optimal threshold of 26 pieces. Nov 16 18:11:56.813848675 (128h27m42.172507339s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813864049 (128h27m42.172522714s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813894323 (128h27m42.172552985s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813935878 (128h27m42.172594551s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813968221 (128h27m42.172626877s) [seg_pos={0 0}, upload=113070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 16 18:11:56.814067848 (128h27m42.172726507s) [upload=113070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 18:11:56.814075640 (128h27m42.172734299s) [upload=113070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 18:13:47.213254651 (128h29m32.571913308s) [upload=113070]: Done uploading segments... Nov 16 18:13:47.213331906 (128h29m32.57199057s) [upload=113070]: Upload finished. err: metaclient: context deadline exceeded Duration: 938.877917ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 904.927408ms trace: https://jaeger.dev.storj.io/trace/70994c85f165c3a0 FAIL (2025-11-16 18:13:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ce... Nov 16 18:11:56.176141393 (128h27m41.53480005s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176505694 (128h27m41.535164346s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176694869 (128h27m41.535353535s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176906560 (128h27m41.535565213s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178010924 (128h27m41.536669583s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178162029 (128h27m41.53682069s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178624747 (128h27m41.537283407s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.179822806 (128h27m41.538481511s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.181092396 (128h27m41.539751055s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.182267056 (128h27m41.540925728s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.298224599 (128h27m41.656883262s) [upload=113070]: Next returned nil segment Nov 16 18:11:56.298240977 (128h27m41.656899643s) [upload=113070]: Waiting for error group managing segments... Nov 16 18:11:56.322992573 (128h27m41.681651228s) [noise=true , address=81.171.14.70:4031 , piece_id=WMGGCRBCTQTVREBKXGD37YVKTKSK2P5SFJV5LH2MGFI3RJXONTKQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.326677846 (128h27m41.6853365s) [noise=true , address=81.171.14.70:4033 , piece_id=UR3UBNCMPQSDNMI44MKZ34MRQ4MYPCLHGWGWNDEPVNJ3GGFDULDQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.327086439 (128h27m41.685745102s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330596159 (128h27m41.689254822s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330894325 (128h27m41.689552985s) [noise=true , address=185.167.204.205:4016 , piece_id=XTDJJJHRNTXLU4D3HJ4OCP2UQR6RF6KHCU6OWSQAPI2Q6SONPKYA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331200302 (128h27m41.689858955s) [noise=true , address=185.167.204.205:4019 , piece_id=LJCJSOQLF4FSLVI7G753ZYDJW6VJJIQCOESIN4KVUHQ36NPZZOHQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331708487 (128h27m41.69036714s) [noise=true , address=185.167.204.206:4061 , piece_id=VWC7K7VZIUF3B47V3XQKWG4WRWFIZSCI4QOW24JI5RC77KZ2VKNQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331819138 (128h27m41.690477798s) [noise=true , address=185.167.204.205:4002 , piece_id=NL2FGXMOTSJRIS7O4ZBTIE2636T4UBL4TBWO3YMA7LCKNTSDGYSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334312534 (128h27m41.692971206s) [noise=true , address=185.167.204.205:4031 , piece_id=J2H4MRWYBVR3FYDVOB7BHQV3XJ2PBXPBKTGIS5SK6N76DDQI3VJA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334376547 (128h27m41.693035205s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334837583 (128h27m41.693496245s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.335363984 (128h27m41.694022646s) [noise=true , address=185.167.204.205:4005 , piece_id=VFVSPAC4QXC5RDOZWH5HS5GLIXCOSRJN67KX65IXBZD2WGULIAJQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.338489129 (128h27m41.697147785s) [noise=true , address=185.167.204.206:4039 , piece_id=S24WYTQCI7XAYTNW5CO6APLU3RRGU4AEJIANW3I5KT5CUHTP5WFQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.340260145 (128h27m41.698918799s) [noise=true , address=81.171.14.70:4040 , piece_id=4YLPSBJNFFRY4KJQRXR4FYNLV4DFOEO6SHIZGTJMSAUHRB4DTBWA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.342507930 (128h27m41.701166584s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.343055650 (128h27m41.701714293s) [noise=true , address=185.167.204.205:4024 , piece_id=EAG5CMHBVOSIFI2WFCQIXQFNGYU4ATNHUUIBQKYKM6B6XETO63YQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.346735021 (128h27m41.705393678s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.348599199 (128h27m41.707257859s) [noise=true , address=185.167.204.206:4051 , piece_id=3OFN2SAZSTHRRM4SNHQCU2IEVOCXMA3BZOLS2YNULUJ5IHFQQSPA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.349247455 (128h27m41.707906114s) [noise=true , address=89.149.201.147:4023 , piece_id=O533J4NWGNLHIAD4SA7VWCHP4WF7GGB33AG4EPFIUMZIVSMCZLSQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.355006125 (128h27m41.713664782s) [noise=true , address=185.167.204.205:4033 , piece_id=SWXEHUBOBIUDILZLZ2NC5G7MGHFPOWU2ZPODJAJV6NFUAYBUYZXQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.360471308 (128h27m41.719129963s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.374029787 (128h27m41.73268843s) [noise=true , address=89.149.201.147:4009 , piece_id=SEU3XXEIWPXZAES7EPYAZQTCKXCOEE7S6CLFL3SZZA3MTBNSMG2A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.789361201 (128h27m42.14801986s) [noise=true , address=81.171.14.70:4030 , piece_id=EPDZQGCWBKEEGW2EMHFP4FVUH4ZCOWFRRVJHH63TX7DF6BUELUIQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.804954891 (128h27m42.163613548s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.805830432 (128h27m42.164489092s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813392799 (128h27m42.172051464s) [noise=true , address=89.149.201.147:4015 , piece_id=W7BFUW3BP7ZK6RTZMKRDLTDIUSA5Y6HA3YXWDPB3MRD3RXDVYZKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813434571 (128h27m42.172093236s) [seg_pos={0 0}, upload=113070]: Segment reached optimal threshold of 26 pieces. Nov 16 18:11:56.813848675 (128h27m42.172507339s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813864049 (128h27m42.172522714s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813894323 (128h27m42.172552985s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813935878 (128h27m42.172594551s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813968221 (128h27m42.172626877s) [seg_pos={0 0}, upload=113070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 16 18:11:56.814067848 (128h27m42.172726507s) [upload=113070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 18:11:56.814075640 (128h27m42.172734299s) [upload=113070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 18:13:47.213254651 (128h29m32.571913308s) [upload=113070]: Done uploading segments... Nov 16 18:13:47.213331906 (128h29m32.57199057s) [upload=113070]: Upload finished. err: metaclient: context deadline exceeded Duration: 904.949296ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m51.077546009s trace: https://jaeger.dev.storj.io/trace/70994c85f165c3a0 FAIL (2025-11-16 18:13:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ce... Nov 16 18:11:56.176141393 (128h27m41.53480005s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176505694 (128h27m41.535164346s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176694869 (128h27m41.535353535s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.176906560 (128h27m41.535565213s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178010924 (128h27m41.536669583s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178162029 (128h27m41.53682069s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.178624747 (128h27m41.537283407s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.179822806 (128h27m41.538481511s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.181092396 (128h27m41.539751055s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.182267056 (128h27m41.540925728s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Uploading piece... Nov 16 18:11:56.298224599 (128h27m41.656883262s) [upload=113070]: Next returned nil segment Nov 16 18:11:56.298240977 (128h27m41.656899643s) [upload=113070]: Waiting for error group managing segments... Nov 16 18:11:56.322992573 (128h27m41.681651228s) [noise=true , address=81.171.14.70:4031 , piece_id=WMGGCRBCTQTVREBKXGD37YVKTKSK2P5SFJV5LH2MGFI3RJXONTKQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.326677846 (128h27m41.6853365s) [noise=true , address=81.171.14.70:4033 , piece_id=UR3UBNCMPQSDNMI44MKZ34MRQ4MYPCLHGWGWNDEPVNJ3GGFDULDQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.327086439 (128h27m41.685745102s) [noise=true , address=185.167.204.205:4028 , piece_id=G5BXCSGHR7FAQBHCZ3ENJOJZU2NRFD3LNW5GJQ6J4S7WR5WCNZLA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330596159 (128h27m41.689254822s) [noise=true , address=185.167.204.206:4046 , piece_id=N5F6QPOYULCUG2XN4X2VZ5NZWTJ6SEJCFUY5STUA4PCQOR35JIYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.330894325 (128h27m41.689552985s) [noise=true , address=185.167.204.205:4016 , piece_id=XTDJJJHRNTXLU4D3HJ4OCP2UQR6RF6KHCU6OWSQAPI2Q6SONPKYA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331200302 (128h27m41.689858955s) [noise=true , address=185.167.204.205:4019 , piece_id=LJCJSOQLF4FSLVI7G753ZYDJW6VJJIQCOESIN4KVUHQ36NPZZOHQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331708487 (128h27m41.69036714s) [noise=true , address=185.167.204.206:4061 , piece_id=VWC7K7VZIUF3B47V3XQKWG4WRWFIZSCI4QOW24JI5RC77KZ2VKNQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.331819138 (128h27m41.690477798s) [noise=true , address=185.167.204.205:4002 , piece_id=NL2FGXMOTSJRIS7O4ZBTIE2636T4UBL4TBWO3YMA7LCKNTSDGYSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334312534 (128h27m41.692971206s) [noise=true , address=185.167.204.205:4031 , piece_id=J2H4MRWYBVR3FYDVOB7BHQV3XJ2PBXPBKTGIS5SK6N76DDQI3VJA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334376547 (128h27m41.693035205s) [noise=true , address=81.171.14.70:4034 , piece_id=QLIKWTIUMSKIQOHGJ2DHXMKXK4Z5JT6X33SXEGSWP3ETIE634Z5A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.334837583 (128h27m41.693496245s) [noise=true , address=185.167.204.206:4066 , piece_id=CDSTTDZLRC2NNEJOO2Y4DO3V67N6FZIGMNZ3UFXUU2Q3AO34NRAQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.335363984 (128h27m41.694022646s) [noise=true , address=185.167.204.205:4005 , piece_id=VFVSPAC4QXC5RDOZWH5HS5GLIXCOSRJN67KX65IXBZD2WGULIAJQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.338489129 (128h27m41.697147785s) [noise=true , address=185.167.204.206:4039 , piece_id=S24WYTQCI7XAYTNW5CO6APLU3RRGU4AEJIANW3I5KT5CUHTP5WFQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.340260145 (128h27m41.698918799s) [noise=true , address=81.171.14.70:4040 , piece_id=4YLPSBJNFFRY4KJQRXR4FYNLV4DFOEO6SHIZGTJMSAUHRB4DTBWA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.342507930 (128h27m41.701166584s) [noise=true , address=185.167.204.206:4050 , piece_id=5G3GDM33B33GDIX4WC6DBM4NI2ZVINFVUYYZMRD22WB53XS2235A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.343055650 (128h27m41.701714293s) [noise=true , address=185.167.204.205:4024 , piece_id=EAG5CMHBVOSIFI2WFCQIXQFNGYU4ATNHUUIBQKYKM6B6XETO63YQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.346735021 (128h27m41.705393678s) [noise=true , address=185.167.204.205:4003 , piece_id=L4QSQNZALCUV6PQS4MPIAMD5SKPQOAFG374L7XWV7PPV7NTMPBSA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.348599199 (128h27m41.707257859s) [noise=true , address=185.167.204.206:4051 , piece_id=3OFN2SAZSTHRRM4SNHQCU2IEVOCXMA3BZOLS2YNULUJ5IHFQQSPA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.349247455 (128h27m41.707906114s) [noise=true , address=89.149.201.147:4023 , piece_id=O533J4NWGNLHIAD4SA7VWCHP4WF7GGB33AG4EPFIUMZIVSMCZLSQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.355006125 (128h27m41.713664782s) [noise=true , address=185.167.204.205:4033 , piece_id=SWXEHUBOBIUDILZLZ2NC5G7MGHFPOWU2ZPODJAJV6NFUAYBUYZXQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.360471308 (128h27m41.719129963s) [noise=true , address=185.167.204.205:4008 , piece_id=S2XUX5ZXG2Z2Q66H6BYIW2JDHITKILKQ3KLPUQBUAO3HYLLSIEYQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.374029787 (128h27m41.73268843s) [noise=true , address=89.149.201.147:4009 , piece_id=SEU3XXEIWPXZAES7EPYAZQTCKXCOEE7S6CLFL3SZZA3MTBNSMG2A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.789361201 (128h27m42.14801986s) [noise=true , address=81.171.14.70:4030 , piece_id=EPDZQGCWBKEEGW2EMHFP4FVUH4ZCOWFRRVJHH63TX7DF6BUELUIQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.804954891 (128h27m42.163613548s) [noise=true , address=81.171.14.70:4029 , piece_id=BA3OZUPTVGRPA2QMLBFSHBQOGYR2SJP4OTC4DPYRBD2OI44T4C7A, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.805830432 (128h27m42.164489092s) [noise=true , address=81.171.14.70:4041 , piece_id=3J7WHLNPTN5T7R4QOGUQPLY6JLGDK2ZD64PK7IREZHO6OT3GRPZA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813392799 (128h27m42.172051464s) [noise=true , address=89.149.201.147:4015 , piece_id=W7BFUW3BP7ZK6RTZMKRDLTDIUSA5Y6HA3YXWDPB3MRD3RXDVYZKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: <nil> Nov 16 18:11:56.813434571 (128h27m42.172093236s) [seg_pos={0 0}, upload=113070]: Segment reached optimal threshold of 26 pieces. Nov 16 18:11:56.813848675 (128h27m42.172507339s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813864049 (128h27m42.172522714s) [noise=true , address=89.149.201.147:4020 , piece_id=YDL724ONO53WQM4PEEXQLU6CHF2JXYMLIAADCDCGCGTODFOVT4AQ, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813894323 (128h27m42.172552985s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF Nov 16 18:11:56.813935878 (128h27m42.172594551s) [noise=true , address=89.149.201.147:4022 , piece_id=E3GDAJLEJKK624PX4XQJQQEW4XAHWT5HVXUPHHEVWBM45PIZBFKA, seg_pos={0 0}, upload=113070]: Long tail cancellation due to optimal threshold Nov 16 18:11:56.813968221 (128h27m42.172626877s) [seg_pos={0 0}, upload=113070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 16 18:11:56.814067848 (128h27m42.172726507s) [upload=113070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 18:11:56.814075640 (128h27m42.172734299s) [upload=113070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 18:13:47.213254651 (128h29m32.571913308s) [upload=113070]: Done uploading segments... Nov 16 18:13:47.213331906 (128h29m32.57199057s) [upload=113070]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m51.077569937s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [X...........................................................] duration: 1.63732773s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:30:21 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.637397297s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.X..........................................................] duration: 1.709973672s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:29:55 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.710005851s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..X.........................................................] duration: 1.580713501s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:29:36 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.580765209s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...X........................................................] duration: 1.679891497s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:29:05 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.679917941s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....X.......................................................] duration: 1.943931923s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:28:45 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.943987914s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....X......................................................] duration: 1.682885611s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:28:03 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.682914012s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......X.....................................................] duration: 1.646295672s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:26:56 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.64632154s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......X....................................................] duration: 1.619108986s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:25:42 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.619137941s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........X...................................................] duration: 2.238651688s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:25:21 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.238681076s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........X..................................................] duration: 2.137204789s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:24:28 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.137226841s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........X.................................................] duration: 1.643267772s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:23:34 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.643295053s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........X................................................] duration: 2.170959676s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:22:33 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.171421413s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............X...............................................] duration: 1.767114297s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:21:43 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.767221732s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............X..............................................] duration: 1.773161041s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:20:31 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.773187826s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............X.............................................] duration: 1.705355112s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:19:45 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.705409802s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............X............................................] duration: 1.683072977s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:18:40 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.683104185s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................X...........................................] duration: 1.579559107s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:17:26 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.579616176s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................X..........................................] duration: 1.707453478s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:16:53 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.707481918s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................X.........................................] duration: 1.672163999s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:16:06 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.672182637s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................X........................................] duration: 2.071453557s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:15:47 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.071472699s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................X.......................................] duration: 1.737784289s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:14:54 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.737805575s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................X......................................] duration: 1.54178649s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:13:47 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.541815522s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................X.....................................] duration: 1.680690661s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:12:57 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.680720633s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................X....................................] duration: 2.202308376s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:11:52 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.202336251s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................X...................................] duration: 1.683088733s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:11:31 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.683237273s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................X..................................] duration: 2.129583479s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:10:26 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.129745714s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................X.................................] duration: 1.609403254s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:09:54 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.609437632s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................X................................] duration: 2.115489072s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:09:09 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.115515476s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................X...............................] duration: 1.745430915s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:07:56 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.745453034s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................X..............................] duration: 1.623613613s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:07:31 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.623656565s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................X.............................] duration: 2.092543453s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:06:20 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.092571476s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................X............................] duration: 1.595383269s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:06:04 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.595413272s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................X...........................] duration: 1.676889919s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:05:48 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.676921938s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................X..........................] duration: 1.650712875s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:05:09 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.650844433s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................X.........................] duration: 1.625017888s trace: https://jaeger.dev.storj.io/trace/1044d785f9a6c033 FAIL (2025-11-15 22:04:26 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.625050761s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)