Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 3.230855034s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 16:02:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 3.230887552s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [......X........................................X............] duration: 2.091831679s trace: https://jaeger.dev.storj.io/trace/6972825919f640d8 FAIL (2026-04-03 16:02:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.091848319s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.......X........................................X...........] duration: 2.260941348s trace: https://jaeger.dev.storj.io/trace/6972825919f640d8 FAIL (2026-04-03 16:02:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.260992717s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.908225945s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 16:01:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 2.908258562s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........X........................................X..........] duration: 2.172936118s trace: https://jaeger.dev.storj.io/trace/6972825919f640d8 FAIL (2026-04-03 16:01:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.172961228s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.........X........................................X.........] duration: 1.632713775s trace: https://jaeger.dev.storj.io/trace/6972825919f640d8 FAIL (2026-04-03 16:01:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.632748221s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 3.444416542s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 16:00:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 3.444450294s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........X........................................X........] duration: 1.745963891s trace: https://jaeger.dev.storj.io/trace/6972825919f640d8 FAIL (2026-04-03 16:00:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.746000068s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.414066833s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 16:00:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 5.414106357s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 3.346563188s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:59:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 3.34660742s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...........X........................................X.......] duration: 1.699101335s trace: https://jaeger.dev.storj.io/trace/6972825919f640d8 FAIL (2026-04-03 15:59:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.699204105s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [............X........................................X......] duration: 2.093379282s trace: https://jaeger.dev.storj.io/trace/6972825919f640d8 FAIL (2026-04-03 15:59:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.093479303s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 3.467957058s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:58:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 3.468019718s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.............X........................................X.....] duration: 1.575949086s trace: https://jaeger.dev.storj.io/trace/6972825919f640d8 FAIL (2026-04-03 15:58:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.575970005s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 10.300294006s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:58:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 10.300330273s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 12.321158706s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:57:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 12.321192465s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..............X........................................X....] duration: 1.68987086s trace: https://jaeger.dev.storj.io/trace/6972825919f640d8 FAIL (2026-04-03 15:57:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.689904665s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...............X........................................X...] duration: 1.68917926s trace: https://jaeger.dev.storj.io/trace/6972825919f640d8 FAIL (2026-04-03 15:57:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.689229682s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 13.909931136s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:56:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 13.90997625s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 34.055455972s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:56:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 34.05548129s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [................X........................................X..] duration: 1.982091121s trace: https://jaeger.dev.storj.io/trace/6972825919f640d8 FAIL (2026-04-03 15:55:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.982196169s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 9.306634699s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:55:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 9.306669109s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.................X........................................X.] duration: 3.033224637s trace: https://jaeger.dev.storj.io/trace/6972825919f640d8 FAIL (2026-04-03 15:54:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.033325029s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..................X........................................X] duration: 5.001645425s trace: https://jaeger.dev.storj.io/trace/6972825919f640d8 FAIL (2026-04-03 15:54:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001669199s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 3.164653713s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:54:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 3.164692776s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................X........................................] duration: 1.640140845s trace: https://jaeger.dev.storj.io/trace/15001b0778f77a8 FAIL (2026-04-03 15:53:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.640169999s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.672768002s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:53:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 5.672875184s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................X.......................................] duration: 1.587409104s trace: https://jaeger.dev.storj.io/trace/15001b0778f77a8 FAIL (2026-04-03 15:52:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.587456362s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................X......................................] duration: 1.576980466s trace: https://jaeger.dev.storj.io/trace/15001b0778f77a8 FAIL (2026-04-03 15:52:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.577019893s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 3.587516704s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:52:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 3.587565093s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 3.561340035s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:51:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 3.561371957s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................X.....................................] duration: 1.585235182s trace: https://jaeger.dev.storj.io/trace/15001b0778f77a8 FAIL (2026-04-03 15:51:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.585265589s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................X....................................] duration: 1.569774279s trace: https://jaeger.dev.storj.io/trace/15001b0778f77a8 FAIL (2026-04-03 15:50:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.569805958s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 3.284536199s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:50:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 3.28456452s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 3.613112021s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:50:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 3.613204487s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................X...................................] duration: 1.527723016s trace: https://jaeger.dev.storj.io/trace/15001b0778f77a8 FAIL (2026-04-03 15:49:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.52775692s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 3.550996042s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:49:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 3.551036299s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................X..................................] duration: 1.620406252s trace: https://jaeger.dev.storj.io/trace/15001b0778f77a8 FAIL (2026-04-03 15:48:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.620444904s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 3.520109799s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:47:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 3.52014342s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................X.................................] duration: 1.614854949s trace: https://jaeger.dev.storj.io/trace/15001b0778f77a8 FAIL (2026-04-03 15:47:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.614894294s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................X................................] duration: 1.531013198s trace: https://jaeger.dev.storj.io/trace/15001b0778f77a8 FAIL (2026-04-03 15:47:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.531045419s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 3.646222341s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:47:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 3.646253067s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 3.800827368s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:47:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 3.800860822s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................X...............................] duration: 1.721487805s trace: https://jaeger.dev.storj.io/trace/15001b0778f77a8 FAIL (2026-04-03 15:46:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.721524462s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 3.240228536s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:46:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 3.240260585s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................X..............................] duration: 1.641162016s trace: https://jaeger.dev.storj.io/trace/15001b0778f77a8 FAIL (2026-04-03 15:45:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.641258219s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................X.............................] duration: 1.582019067s trace: https://jaeger.dev.storj.io/trace/15001b0778f77a8 FAIL (2026-04-03 15:45:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.582072322s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 4.959411209s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:45:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 4.959445531s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.221650138s trace: https://jaeger.dev.storj.io/trace/c036a4f2d71a778 FAIL (2026-04-03 15:44:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== 24:14.098265938 (575h13m17.113935551s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.098831891 (575h13m17.114501506s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099058352 (575h13m17.114727971s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099461616 (575h13m17.11513122s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099500194 (575h13m17.115169795s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.099989808 (575h13m17.11565942s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100032457 (575h13m17.115702071s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100377681 (575h13m17.116047288s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100513196 (575h13m17.116182799s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.100984246 (575h13m17.116653902s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101216587 (575h13m17.116886189s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101442365 (575h13m17.117111966s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101701277 (575h13m17.117370881s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.101844572 (575h13m17.117514184s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102128723 (575h13m17.117798352s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102543800 (575h13m17.118213402s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.102912469 (575h13m17.118582073s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Uploading piece... Apr 3 15:24:14.654255388 (575h13m17.66992499s) [upload=552424]: Next returned nil segment Apr 3 15:24:14.654276756 (575h13m17.669946361s) [upload=552424]: Waiting for error group managing segments... Apr 3 15:24:14.804527655 (575h13m17.820197259s) [noise=true , address=77.90.31.16:4745 , piece_id=A3WUTZNH4IBVVRCSVDJMBYHMMETDNQQRQ7UJBTDLMFLC4GQ6ROBA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:14.838399330 (575h13m17.854068932s) [noise=true , address=77.90.31.21:4903 , piece_id=36XTH3NLLEUBWFFQKVNHMUFK625M27I2SMS6Z2XG6IJIZMCY4DFQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:15.955952569 (575h13m18.971622182s) [noise=true , address=145.79.227.33:4897 , piece_id=WTVVSS2GM7YPZE3I52MIOYU4VPC6UX4N6QXTQK7FLTJSLMMEN4EA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:17.252252820 (575h13m20.267922421s) [noise=true , address=145.79.227.13:4818 , piece_id=EQ53LKD3QL6V52AJAELQBSR4EDULR32X2VY5EIV74JE4VTSEBKWQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.171247616 (575h13m22.186917217s) [noise=true , address=64.49.13.140:4506 , piece_id=3MCEL6K22E3ZR3RMRSOBBXFTRL7D2EJYGGTQSV6X3E2TJJYJBROA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.217203345 (575h13m22.232872952s) [noise=true , address=77.90.31.10:4542 , piece_id=2OEIKYWWNAR2HY4ILG2RTOTSYOP5PJHOW4PAUM2PRO7P6NMHT6MQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:19.612521688 (575h13m22.628191289s) [noise=true , address=45.91.116.13:4183 , piece_id=6LQYYMIZQMTVFE7PGX7ZL6V74DET6BG3NI4V74NVNVWLA3CQOP4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:22.414814811 (575h13m25.430484415s) [noise=true , address=45.91.116.7:4025 , piece_id=W4ALSH5GEHBRMFGP65P3UBKKZ7NTV2TBR7YPV5544TCG7J5XQWVA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:23.372257597 (575h13m26.387927189s) [noise=true , address=45.91.116.11:4133 , piece_id=NBNECP3KBWKATL7YZWQMPI3GVAUSNPN4NQNXF22BWCTW42WU73OQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:26.665681955 (575h13m29.681351555s) [noise=true , address=193.149.164.67:4010 , piece_id=JSL7EFOTNVHRNKXHP4UFSQLPOZPNMGALMGP62YBFT67BFOYTBRCQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:29.460066298 (575h13m32.475735909s) [noise=true , address=45.91.116.14:4210 , piece_id=TSDSSLPLD5OFIHMJXF7KOKUZIHF7JHRGAUZUUAXCUNCUUUCXEE4A, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.385795001 (575h13m33.401464603s) [noise=true , address=64.49.13.169:4706 , piece_id=EEYMZ4KZQXBGY4M3IVP72XY43DE4PWWQUVWJPMROFCTW243HUQVQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:30.927622980 (575h13m33.943292592s) [noise=true , address=193.149.164.69:4075 , piece_id=MZKPRYEXEDIY24H3UEPYJFYMZDFG3HSTKI43F37B64GMLTNWUL5Q, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.534990842 (575h13m36.550660448s) [noise=true , address=193.149.164.66:4980 , piece_id=AQJ3HP6H2IMCHPLZ7EFJQNFJKG2JCMXMOC24H4SHALYKUHG7FLIA, seg_pos={0 0}, upload=552424]: Done uploading piece. err: <nil> Apr 3 15:24:33.535026340 (575h13m36.550695943s) [seg_pos={0 0}, upload=552424]: Satisfied cohort requirements after 14 pieces Apr 3 15:24:33.535257561 (575h13m36.550927173s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gsNGvM8oKL9Fw1KLTrtpANYVozX2KGDDtL5BUmDhL9GDeFGof): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535265871 (575h13m36.550935474s) [noise=true , address=45.91.116.5:4319 , piece_id=FJCYG63AKPCE6UDM5NWNE6AEYRDK6HCP2SU32NZNQVMEMXI2LWRQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535403914 (575h13m36.551073528s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gozVE5KTcqGagY5veHpMqMQa3QHaNumynNSPvSb6LsKBHALuv): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535421314 (575h13m36.551090929s) [noise=true , address=145.79.227.38:4984 , piece_id=6ZHRHZRHOGK7T2VCQWQFLKZDH2EN5DIXWNKZ7MINSSPY3ARR5GBQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535725819 (575h13m36.55139543s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ziK1cLtCwq29Ep2kNhnuy7vSwr5Qk3tdgtERxnz2WRVS4XFvx): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535735892 (575h13m36.551405504s) [noise=true , address=193.149.164.68:4063 , piece_id=S23R6I4BKMZ34CDB3ZB6IKDJFAHAQN5FAIDGIPZFUFJ2T6UW4GHQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.535945878 (575h13m36.551615492s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vSW1FsGdLjuXrE6eBo5suP7RHKv9uTUUenQWbCMFWWRcqoP9T): protocol: expected piece hash; context canceled; EOF Apr 3 15:24:33.535961214 (575h13m36.551630824s) [noise=true , address=193.149.164.70:4137 , piece_id=2NJOIDG52XMYU3URSFRSEQE4LDAHIVQMC4WJQUYFLCB6FXIC47FQ, seg_pos={0 0}, upload=552424]: Long tail cancellation due to optimal threshold Apr 3 15:24:33.536141603 (575h13m36.551811196s) [seg_pos={0 0}, upload=552424]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 3 15:24:33.536421191 (575h13m36.552090785s) [upload=552424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 3 15:24:33.536463199 (575h13m36.552132807s) [upload=552424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 3 15:24:33.683231351 (575h13m36.698900952s) [upload=552424]: Done uploading segments... Apr 3 15:24:33.683270427 (575h13m36.698940027s) [upload=552424]: Upload finished. err: <nil> Duration: 5.221679873s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................X............................] duration: 2.183873994s trace: https://jaeger.dev.storj.io/trace/15001b0778f77a8 FAIL (2026-04-03 15:44:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.183906144s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)